<div dir="ltr">Hi Alex<br><div class="gmail_quote"><div dir="ltr"><div><div><div><div><div><div><br></div>We seem to have a similar setup as of OS, hardware, traffic and myricom.<br></div>We experience almost no drops (usually less than 0.2%).<br><br></div><div>We start suri with these params:<br><br>SNF_NUM_RINGS=10 SNF_FLAGS=0x1 SNF_DATARING_SIZE=12884901888 SNF_DESCRING_SIZE=3221225472<br></div><div><br></div><div>We do start Suri with "<span style="font-size:11pt">-i snf0<span class="m_-2935738683436375521gmail-">", it throws a warning in suricata.log, but it works:<br><Warning> - [ERRCODE: SC_ERR_SYSCALL(50)] - Failure when trying to get feature via ioctl for 'snf0': No such device (19)<br></span></span></div><div><br><br></div>There are other factors which influence the performance:<br><br></div>- I set a BPF to bypass traffic I don't want to see anyway<br><br></div><div>- number of rules (we have activated about 22'000 rules)<br></div><div><br></div>- suricata.yaml, some configs greatly influence performance<br></div><div>Some excerpts:<br></div><div>...<br>pcap:<br> - interface: snf0<br> threads: 10 <<--- has to correspond with SNF_NUM_RINGS<br> buffer-size: 2gb<br> checksum-checks: no<br> promisc: no<br> snaplen: 1520<br> bpf-filter: "..."<br><br>...<br>stream:<br> memcap: 5gb<br>...<br> reassembly:<br> memcap: 10gb<br>...<br><br>detect:<br> profile: custom<br> custom-values:<br> toclient-groups: 200<br> toserver-groups: 200<br>...<br> # hyperscan<br>mpm-algo: hs<br>spm-algo: hs<br><br></div># pin to cores<br><div>threading:<br> set-cpu-affinity: yes<br> cpu-affinity:<br> - management-cpu-set:<br> cpu: [ ... ]<br> - worker-cpu-set:<br> cpu: [ ... ]<br> mode: "exclusive" # run detect threads in these cpus<br> threads: 10<br> prio:<br> high: [ ... ]<br> default: "medium"<br>...<br>max-pending-packets: 8192<br>...<br>flow:<br> memcap: 1024mb<br> hash-size: 524288<br> prealloc: 1048576<br> emergency-recovery: 30<br> managers: 2<br> recyclers: 2<br>...<br><br><br></div><div>- try to pin suri worker threads to the same NUMA node the myricom is attached to<br><br><br></div><div>HTH,<br></div><div>erich<br></div><div><br><div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">2018-02-20 18:58 GMT+01:00 Alexander Merck <span dir="ltr"><<a href="mailto:alexander.merck@duke.edu" target="_blank">alexander.merck@duke.edu</a>></span>:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="h5">
<div bgcolor="white" lang="EN-US">
<div class="m_-2935738683436375521gmail-m_-5345556061192901245WordSection1">
<p class="MsoNormal"><span style="font-size:11pt">Hello,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Hopefully someone can help shed some light on some issues we've been seeing. We just installed a new instance of Suricata on a fresh RHEL7 monitoring box with Myricom cards. However, we are seeing significant
packet loss (20-35%) on 2-3 Gbps traffic when attempting to use the SNF drivers.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">I'm suspecting that the Myricom SNF drivers are not functioning as expected. We're able to run tcpdump compiled against these drivers with no issue, including generating debug output. We've also found when
supplying the SNF_DEBUG_MASK environment variable when running Suricata, no debug output is generated.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Also, when using Suricata with the SNF drivers, should you be able to use the interface names specified by SNF (e.g. snf0)? When trying to run Suricata using the -i snf0, we get an "Unable to find iface snf0:
No such device" error message. We are only able to run Suricata against the interface names specified by the kernel (in our case, enp4s0)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">The version of Suricata is 4.0.4 and the version of SNF is 3.0.12. Running ldd shows that Suricata is linked against the SNF libraries.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"># ldd /usr/bin/suricata<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">...<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> libpcap.so.1 => /opt/snf/lib/libpcap.so.1 (0x00007f238ffb0000)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">...<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"> libsnf.so.0 => /opt/snf/lib/libsnf.so.0 (0x00007f238dae4000)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">We compiled Suricata per these instructions:
<a href="https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Myricom" target="_blank">
https://redmine.openinfosecfou<wbr>ndation.org/projects/suricata/<wbr>wiki/Myricom</a>. I did notice that this document is over five years old, but all of the configuration options seemed correct.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">./configure --with-libpcap-includes=/opt/s<wbr>nf/include/ --with-libpcap-libraries=/opt/<wbr>snf/lib/ --prefix=/usr --sysconfdir=/etc --localstatedir=/var<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">And we're running Suricata with the following command:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">SNF_NUM_RINGS=32 SNF_DATARING_SIZE=17179869184 SNF_DESCRING_SIZE=4294967296 SNF_FLAGS=0x1 SNF_DEBUG_MASK=3 SNF_DEBUG_FILENAME="/tmp/snf.o<wbr>ut" /usr/bin/suricata -c /etc/suricata/suricata.yaml -i enp4s0 --runmode=workers<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">The box we’re running this on has 64 cores and 256GB of RAM, so I doubt it’s a resource issue…but could potentially be a configuration issue.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">Are we missing something in the install process that may be causing these issues? Any recommendations or pointers would be greatly appreciated. Thanks!<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt">-Alex M<span class="m_-2935738683436375521gmail-HOEnZb"><font color="#888888"><u></u><u></u></font></span></span></p><span class="m_-2935738683436375521gmail-HOEnZb"><font color="#888888">
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">-- <u></u><u></u></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Alexander Merck<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">Duke University<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;color:black">IT Security Office<u></u><u></u></span></p>
</div>
</font></span></div>
</div>
<br></div></div>______________________________<wbr>_________________<br>
Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org" target="_blank">oisf-users@openinfosecfoundati<wbr>on.org</a><br>
Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/suppor<wbr>t/</a><br>
List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfound<wbr>ation.org/mailman/listinfo/<wbr>oisf-users</a><br>
<br>
Conference: <a href="https://suricon.net" rel="noreferrer" target="_blank">https://suricon.net</a><br>
Trainings: <a href="https://suricata-ids.org/training/" rel="noreferrer" target="_blank">https://suricata-ids.org/train<wbr>ing/</a><br></blockquote></div><br></div></div></div></div>
</div><br></div>