<div dir="ltr">I knew I only had half the picture!<div><br></div><div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Runmode Workers <br>management-cpu-set - used for management (example - flow.managers, flow.recyclers)<br>detect-cpu-set - used for receive,streamtcp,decode,detect,output(logging),respond/reject</blockquote></div><div><br></div><div>I'm assuming I can just remove configurations options for unused cpu-sets? Time to make some adjustments to the configs!</div><div><br></div><div>Greatly appreciated!</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 29, 2016 at 11:14 AM, Peter Manev <span dir="ltr"><<a href="mailto:petermanev@gmail.com" target="_blank">petermanev@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On Thu, Feb 25, 2016 at 4:46 AM, Brandon Lattin <<a href="mailto:lattin@umn.edu">lattin@umn.edu</a>> wrote:<br>
> I'd like to pick the Suricata developer brains on what each cpu-set does,<br>
> and how to best handle cpu pinning.<br>
><br>
> I've noticed enormous performance gains by tweaking the following settings,<br>
> but still feel as though I only have a partial picture.<br>
><br>
> For those still getting up to speed, check out section 8.1.9 at:<br>
> <a href="http://jasonish-suricata.readthedocs.org/en/latest/configuration/suricata-yaml.html" rel="noreferrer" target="_blank">http://jasonish-suricata.readthedocs.org/en/latest/configuration/suricata-yaml.html</a><br>
<br>
</span>I have actually updated the docs with regards to this here -<br>
<a href="https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml#Relevant-cpu-affinity-settings-for-IDSIPS-modes" rel="noreferrer" target="_blank">https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml#Relevant-cpu-affinity-settings-for-IDSIPS-modes</a><br>
(thanks Eric for helping out through the code :) )<br>
<div><div class="h5"><br>
<br>
><br>
> I'd like approach this from the expectation that we're looking at many-core<br>
> machines capable of handling a 10Gbps link at moderate levels of saturation.<br>
><br>
> Ideally, this info might make it's way to the official docs. I'm going to<br>
> enter this under the assumption that my assumptions on what each cpu-set<br>
> does is wrong or misguided (which is so often the case)!<br>
><br>
> So, here's what we have:<br>
><br>
> - management-cpu-set:<br>
> Description: ???<br>
><br>
> - receive-cpu-set:<br>
> Description: ???<br>
><br>
> - decode-cpu-set:<br>
> Description: ???<br>
><br>
> - stream-cpu-set:<br>
> Description: ???<br>
><br>
> - detect-cpu-set:<br>
> Description: ???<br>
><br>
> - verdict-cpu-set:<br>
> Description: ???<br>
><br>
> - reject-cpu-set:<br>
> Description: ???<br>
><br>
> - output-cpu-set:<br>
> Description: ???<br>
><br>
><br>
> I don't want to derail the thread with tuning voodoo just yet, but it may<br>
> help to have an understanding of where I'm coming from.<br>
><br>
> Here's my current config settings. We're handling a max of about 1100MB/s<br>
> over a Myricom (18 ring buffers, hence 18 pinned cores; kernel 2.6) with<br>
> 19,000 ET Pro rules on a Dell R630 with 2x Xeon E5-2687W v3 @ 3.1GHz and<br>
> 128GB RAM. I'll be bringing up mpm-context/detect-engine tuning in a later<br>
> email thread, so don't jump the gun!<br>
><br>
> threading:<br>
> set-cpu-affinity: yes<br>
> cpu-affinity:<br>
> - management-cpu-set:<br>
> cpu: [ 0,2 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "high"<br>
> - receive-cpu-set:<br>
> cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "low"<br>
> - decode-cpu-set:<br>
> cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "medium"<br>
> - stream-cpu-set:<br>
> cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "medium"<br>
> - detect-cpu-set:<br>
> cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "medium"<br>
> - verdict-cpu-set:<br>
> cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "high"<br>
> - reject-cpu-set:<br>
> cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "low"<br>
> - output-cpu-set:<br>
> cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]<br>
> mode: "exclusive"<br>
> prio:<br>
> default: "medium"<br>
><br>
><br>
> Victor, Eric, Peter, and everyone else who I've forgotten,<br>
><br>
> What have you got for us?<br>
><br>
> --<br>
> Brandon Lattin<br>
> Security Analyst<br>
> University of Minnesota - University Information Security<br>
> Office: 612-626-6672<br>
><br>
</div></div>> _______________________________________________<br>
> Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
> Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/support/</a><br>
> List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
> Suricata User Conference November 9-11 in Washington, DC:<br>
> <a href="http://oisfevents.net" rel="noreferrer" target="_blank">http://oisfevents.net</a><br>
<span class="HOEnZb"><font color="#888888"><br>
<br>
<br>
--<br>
Regards,<br>
Peter Manev<br>
</font></span></blockquote></div><br></div>