<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div></div><div>Can we include this calculator in the next Septun?</div><div><br>On Jul 11, 2018, at 7:53 AM, Cloherty, Sean E <<a href="mailto:scloherty@mitre.org">scloherty@mitre.org</a>> wrote:<br><br></div><blockquote type="cite"><div>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Lucida Sans";
panose-1:2 11 6 2 3 5 4 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{mso-style-priority:99;
mso-style-link:"Plain Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle19
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.PlainTextChar
{mso-style-name:"Plain Text Char";
mso-style-priority:99;
mso-style-link:"Plain Text";
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1573276107;
mso-list-type:hybrid;
mso-list-template-ids:1213868600 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoPlainText">Hello Fatema -<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">SEPTun is a great resource for sure and from that you might want to focus first on the CPU affinity and only include those in the same NUMA node as the NIC for workers. (See SEPTun page 14)
<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Some other quick hits – <o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoPlainText" style="mso-list:l0 level1 lfo1">Set threads to auto and specify which CPUs (by number or range of #s) instead of “all” for the workers to use. Also – I think you can choose to use CPUs not on the same NUMA node for the management-cpu-set
so you can save the rest for workers. <o:p></o:p></li><li class="MsoPlainText" style="mso-list:l0 level1 lfo1">Install the NIC driver from Intel<o:p></o:p></li><li class="MsoPlainText" style="mso-list:l0 level1 lfo1">In AF-PACKET – enable tpacketv3
<o:p></o:p></li><li class="MsoPlainText" style="mso-list:l0 level1 lfo1">Change the MPM-ALGO to AC-KS<o:p></o:p></li><li class="MsoPlainText" style="mso-list:l0 level1 lfo1">Stop automated IRQ balancing - Add “killall irqbalance” to your startup script<o:p></o:p></li><li class="MsoListParagraph" style="color:black;margin-left:0in;mso-list:l0 level1 lfo1">
<span style="color:windowtext">Check in the stats.log to see if you are running up against any memcap overruns and increase the RAM set aside for that especially
</span>STREAM MEMCAP<o:p></o:p></li></ul>
<p class="MsoNormal">and <span style="color:black">STREAM REASSEMBLY MEMCAP <o:p>
</o:p></span></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">As to RAM –I’ve attached a calculator which you can use to see how much memory will be used when you vary the values for various settings.<o:p></o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText"><o:p> </o:p></p>
<p class="MsoPlainText">Sean<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><b>From:</b> Oisf-users [<a href="mailto:oisf-users-bounces@lists.openinfosecfoundation.org">mailto:oisf-users-bounces@lists.openinfosecfoundation.org</a>]
<b>On Behalf Of </b>fatema bannatwala<br>
<b>Sent:</b> Tuesday, July 10, 2018 15:01 PM<br>
<b>To:</b> <a href="mailto:oisf-users@lists.openinfosecfoundation.org">oisf-users@lists.openinfosecfoundation.org</a><br>
<b>Subject:</b> [Oisf-users] High Suricata capture.kernel_drops<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I am pretty new to Suricata and started to play around with it.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I have Suricata 4.0.4 running on a CentOS7 box, that has 20 cores (40 on-line cpus) and an intel
<span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> X710 NIC, and 64GB RAM.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white">I am using AF_Packet with following settings, with some other mentioned settings:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"># Linux high speed capture support</span><span style="background:white"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white">af-packet:</span><span style="background:white"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> - interface: em1</span><span style="background:white"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> threads: 24</span><span style="background:white"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> cluster-id: 99</span><span style="background:white"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> cluster-type: cluster_cpu<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> defrag: yes<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> use-mmap: yes<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"> ring-size: 30000</span><span style="background:white"><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"><o:p> </o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white">......<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white"><o:p> </o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white">max-pending-packets: 10000</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;background:white">runmode: workers</span><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">mpm-algo: auto</span><span style="font-family:"Arial",sans-serif"><o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">threading:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> set-cpu-affinity: yes<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> cpu-affinity:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> - management-cpu-set:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> cpu: [ "all" ] # include only these cpus in affinity settings<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> mode: "balanced"<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> prio:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> default: "low"<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> - receive-cpu-set:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> cpu: [ 0 ] # include only these cpus in affinity settings<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> - worker-cpu-set:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> cpu: [ "all" ]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> mode: "exclusive"<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> prio:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> low: [ 0 ]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> medium: [ "1-2" ]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> high: [ 3 ]<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"> default: "medium"<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">detect-thread-ratio: 1.0</span><span style="font-family:"Arial",sans-serif"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">I am monitoring a ~5GBps link and getting high kernel_drop packets seen in stats.log:<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">capture.kernel_packets | Total | 301360376<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">capture.kernel_drops | Total | 67468903<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">Any idea how can I reduce the kernel drop rate of packets? or how can I check if af_packet threads are working correctly?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">I have also disabled the checksuming on the ethernet interface:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif">#
</span><span style="font-size:10.0pt;font-family:"Lucida Sans",sans-serif;color:#333333;background:white">ethtool -K em1 rx off tx off tso off sg off gso off gro off</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif">
<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal">Any help appreciated.<br>
<br>
Thanks,<br>
Fatema.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:10.0pt;font-family:"Arial",sans-serif"><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</div></blockquote><blockquote type="cite"><div><SuricataMemCalcNoPreAlloc.ods></div></blockquote><blockquote type="cite"><div><span>_______________________________________________</span><br><span>Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a></span><br><span>Site: <a href="http://suricata-ids.org">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/">http://suricata-ids.org/support/</a></span><br><span>List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a></span><br><span></span><br><span>Conference: <a href="https://suricon.net">https://suricon.net</a></span><br><span>Trainings: <a href="https://suricata-ids.org/training/">https://suricata-ids.org/training/</a></span></div></blockquote></body></html>