[Oisf-users] Suricata threading

[Peter Manev]

On Thu, Feb 25, 2016 at 4:46 AM, Brandon Lattin <lattin at umn.edu> wrote:
> I'd like to pick the Suricata developer brains on what each cpu-set does,
> and how to best handle cpu pinning.
>
> I've noticed enormous performance gains by tweaking the following settings,
> but still feel as though I only have a partial picture.
>
> For those still getting up to speed, check out section 8.1.9 at:
> http://jasonish-suricata.readthedocs.org/en/latest/configuration/suricata-yaml.html

I have actually updated the docs with regards to this here -
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml#Relevant-cpu-affinity-settings-for-IDSIPS-modes
(thanks Eric for helping out through the code :) )


>
> I'd like approach this from the expectation that we're looking at many-core
> machines capable of handling a 10Gbps link at moderate levels of saturation.
>
> Ideally, this info might make it's way to the official docs. I'm going to
> enter this under the assumption that my assumptions on what each cpu-set
> does is wrong or misguided (which is so often the case)!
>
> So, here's what we have:
>
> - management-cpu-set:
> Description: ???
>
> - receive-cpu-set:
> Description: ???
>
> - decode-cpu-set:
> Description: ???
>
> - stream-cpu-set:
> Description: ???
>
> - detect-cpu-set:
> Description: ???
>
> - verdict-cpu-set:
> Description: ???
>
> - reject-cpu-set:
> Description: ???
>
> - output-cpu-set:
> Description: ???
>
>
> I don't want to derail the thread with tuning voodoo just yet, but it may
> help to have an understanding of where I'm coming from.
>
> Here's my current config settings. We're handling a max of about 1100MB/s
> over a Myricom (18 ring buffers, hence 18 pinned cores; kernel 2.6) with
> 19,000 ET Pro rules on a Dell R630 with 2x Xeon E5-2687W v3 @ 3.1GHz and
> 128GB RAM. I'll be bringing up mpm-context/detect-engine tuning in a later
> email thread, so don't jump the gun!
>
> threading:
>   set-cpu-affinity: yes
>   cpu-affinity:
>     - management-cpu-set:
>         cpu: [ 0,2 ]
>         mode: "exclusive"
>         prio:
>           default: "high"
>     - receive-cpu-set:
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>         mode: "exclusive"
>         prio:
>           default: "low"
>     - decode-cpu-set:
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>         mode: "exclusive"
>         prio:
>           default: "medium"
>     - stream-cpu-set:
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>         mode: "exclusive"
>         prio:
>           default: "medium"
>     - detect-cpu-set:
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>         mode: "exclusive"
>         prio:
>           default: "medium"
>     - verdict-cpu-set:
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>         mode: "exclusive"
>         prio:
>           default: "high"
>     - reject-cpu-set:
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>         mode: "exclusive"
>         prio:
>           default: "low"
>     - output-cpu-set:
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>         mode: "exclusive"
>         prio:
>            default: "medium"
>
>
> Victor, Eric, Peter, and everyone else who I've forgotten,
>
> What have you got for us?
>
> --
> Brandon Lattin
> Security Analyst
> University of Minnesota - University Information Security
> Office: 612-626-6672
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 9-11 in Washington, DC:
> http://oisfevents.net



-- 
Regards,
Peter Manev