[Oisf-users] Suricata threading

[Peter Manev]

On Mon, Feb 29, 2016 at 7:53 PM, Brandon Lattin <lattin at umn.edu> wrote:
> I knew I only had half the picture!
>
>> Runmode Workers
>> management-cpu-set - used for management (example - flow.managers,
>> flow.recyclers)
>> detect-cpu-set - used for
>> receive,streamtcp,decode,detect,output(logging),respond/reject
>
>
> I'm assuming I can just remove configurations options for unused cpu-sets?
> Time to make some adjustments to the configs!

Yes, np.

>
> Greatly appreciated!
>
> On Mon, Feb 29, 2016 at 11:14 AM, Peter Manev <petermanev at gmail.com> wrote:
>>
>> On Thu, Feb 25, 2016 at 4:46 AM, Brandon Lattin <lattin at umn.edu> wrote:
>> > I'd like to pick the Suricata developer brains on what each cpu-set
>> > does,
>> > and how to best handle cpu pinning.
>> >
>> > I've noticed enormous performance gains by tweaking the following
>> > settings,
>> > but still feel as though I only have a partial picture.
>> >
>> > For those still getting up to speed, check out section 8.1.9 at:
>> >
>> > http://jasonish-suricata.readthedocs.org/en/latest/configuration/suricata-yaml.html
>>
>> I have actually updated the docs with regards to this here -
>>
>> https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml#Relevant-cpu-affinity-settings-for-IDSIPS-modes
>> (thanks Eric for helping out through the code :) )
>>
>>
>> >
>> > I'd like approach this from the expectation that we're looking at
>> > many-core
>> > machines capable of handling a 10Gbps link at moderate levels of
>> > saturation.
>> >
>> > Ideally, this info might make it's way to the official docs. I'm going
>> > to
>> > enter this under the assumption that my assumptions on what each cpu-set
>> > does is wrong or misguided (which is so often the case)!
>> >
>> > So, here's what we have:
>> >
>> > - management-cpu-set:
>> > Description: ???
>> >
>> > - receive-cpu-set:
>> > Description: ???
>> >
>> > - decode-cpu-set:
>> > Description: ???
>> >
>> > - stream-cpu-set:
>> > Description: ???
>> >
>> > - detect-cpu-set:
>> > Description: ???
>> >
>> > - verdict-cpu-set:
>> > Description: ???
>> >
>> > - reject-cpu-set:
>> > Description: ???
>> >
>> > - output-cpu-set:
>> > Description: ???
>> >
>> >
>> > I don't want to derail the thread with tuning voodoo just yet, but it
>> > may
>> > help to have an understanding of where I'm coming from.
>> >
>> > Here's my current config settings. We're handling a max of about
>> > 1100MB/s
>> > over a Myricom (18 ring buffers, hence 18 pinned cores; kernel 2.6) with
>> > 19,000 ET Pro rules on a Dell R630 with 2x Xeon E5-2687W v3 @ 3.1GHz and
>> > 128GB RAM. I'll be bringing up mpm-context/detect-engine tuning in a
>> > later
>> > email thread, so don't jump the gun!
>> >
>> > threading:
>> >   set-cpu-affinity: yes
>> >   cpu-affinity:
>> >     - management-cpu-set:
>> >         cpu: [ 0,2 ]
>> >         mode: "exclusive"
>> >         prio:
>> >           default: "high"
>> >     - receive-cpu-set:
>> >         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>> >         mode: "exclusive"
>> >         prio:
>> >           default: "low"
>> >     - decode-cpu-set:
>> >         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>> >         mode: "exclusive"
>> >         prio:
>> >           default: "medium"
>> >     - stream-cpu-set:
>> >         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>> >         mode: "exclusive"
>> >         prio:
>> >           default: "medium"
>> >     - detect-cpu-set:
>> >         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>> >         mode: "exclusive"
>> >         prio:
>> >           default: "medium"
>> >     - verdict-cpu-set:
>> >         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>> >         mode: "exclusive"
>> >         prio:
>> >           default: "high"
>> >     - reject-cpu-set:
>> >         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>> >         mode: "exclusive"
>> >         prio:
>> >           default: "low"
>> >     - output-cpu-set:
>> >         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>> >         mode: "exclusive"
>> >         prio:
>> >            default: "medium"
>> >
>> >
>> > Victor, Eric, Peter, and everyone else who I've forgotten,
>> >
>> > What have you got for us?
>> >
>> > --
>> > Brandon Lattin
>> > Security Analyst
>> > University of Minnesota - University Information Security
>> > Office: 612-626-6672
>> >
>> > _______________________________________________
>> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> > Site: http://suricata-ids.org | Support:
>> > http://suricata-ids.org/support/
>> > List:
>> > https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> > Suricata User Conference November 9-11 in Washington, DC:
>> > http://oisfevents.net
>>
>>
>>
>> --
>> Regards,
>> Peter Manev
>
>



-- 
Regards,
Peter Manev