[Oisf-users] Suricata threading

[Brandon Lattin]

I'd like to pick the Suricata developer brains on what each cpu-set does,
and how to best handle cpu pinning.

I've noticed enormous performance gains by tweaking the following settings,
but still feel as though I only have a partial picture.

For those still getting up to speed, check out section 8.1.9 at:
http://jasonish-suricata.readthedocs.org/en/latest/configuration/suricata-yaml.html

I'd like approach this from the expectation that we're looking at many-core
machines capable of handling a 10Gbps link at moderate levels of saturation.

Ideally, this info might make it's way to the official docs. I'm going to
enter this under the assumption that my assumptions on what each cpu-set
does is wrong or misguided (which is so often the case)!

So, here's what we have:

- management-cpu-set:
Description: ???

- receive-cpu-set:
Description: ???

- decode-cpu-set:
Description: ???

- stream-cpu-set:
Description: ???

- detect-cpu-set:
Description: ???

- verdict-cpu-set:
Description: ???

- reject-cpu-set:
Description: ???

- output-cpu-set:
Description: ???


I don't want to derail the thread with tuning voodoo just yet, but it may
help to have an understanding of where I'm coming from.

Here's my current config settings. We're handling a max of about 1100MB/s
over a Myricom (18 ring buffers, hence 18 pinned cores; kernel 2.6) with
19,000 ET Pro rules on a Dell R630 with 2x Xeon E5-2687W v3 @ 3.1GHz and
128GB RAM. I'll be bringing up mpm-context/detect-engine tuning in a later
email thread, so don't jump the gun!

threading:
  set-cpu-affinity: yes
  cpu-affinity:
    - management-cpu-set:
        cpu: [ 0,2 ]
        mode: "exclusive"
        prio:
          default: "high"
    - receive-cpu-set:
        cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
        mode: "exclusive"
        prio:
          default: "low"
    - decode-cpu-set:
        cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
        mode: "exclusive"
        prio:
          default: "medium"
    - stream-cpu-set:
        cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
        mode: "exclusive"
        prio:
          default: "medium"
    - detect-cpu-set:
        cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
        mode: "exclusive"
        prio:
          default: "medium"
    - verdict-cpu-set:
        cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
        mode: "exclusive"
        prio:
          default: "high"
    - reject-cpu-set:
        cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
        mode: "exclusive"
        prio:
          default: "low"
    - output-cpu-set:
        cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
        mode: "exclusive"
        prio:
           default: "medium"


Victor, Eric, Peter, and everyone else who I've forgotten,

What have you got for us?

-- 
Brandon Lattin
Security Analyst
University of Minnesota - University Information Security
Office: 612-626-6672
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160224/0d6593af/attachment.html>