<div dir="ltr"><div>That's actually what we've seen so far - there might be 100Gbit interfaces but the real-world traffic is much less.</div><div><br></div><div>I'd highly (highly) recommend two cards per server if you're going 2 CPU (and 1 card if there's one CPU) for NUMA affinity, that's critically important for any kind of performance.</div><div><br></div><div>Intel x722-da2 (slightly preferred) or something from the Mellanox connectx-5 family will do the job.</div><div><br></div><div>Let me shamelessly say that a lot of people had lots of luck configuring systems according to a howto myself and Peter Manev (pevma) and Eric wrote a while ago. A couple of things changes since, but mostly on the software layer and the general direction is still correct.<br></div><div><br></div><div><a href="https://github.com/pevma/SEPTun">https://github.com/pevma/SEPTun</a></div><div><a href="https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst">https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst</a></div><div><br></div><div>I'd say 2 CPU with 1 NIC per CPU should be your basic building block. There's no overhead once things are configured correctly and the configuration should be relatively painless.</div><div><br></div><div>It's not the performance configuration you will spend most time on, but tuning the rule set, most likely.</div><div><br></div><div>I'd also recommend having some sort of "packet broker" in front of your cluster that distrbutes traffic among nodes and can be useful for filtering traffic you do not want to see, to service multiple taps, etc. We use Arista (ooold) 7150S but there are many more new models both in Arista land or from different vendors, like Gigamon. Arista tends to be cheaper and lighter on features.<br></div><div><br></div><div><br></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 18, 2019 at 2:40 PM Drew Dixon <<a href="mailto:dwdixon@umich.edu">dwdixon@umich.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">> I'd build a cluster instead of a single 100Gbit machine, for reliability reasons, unless your space is limited.<div><br></div><div>That's actually what I'm aiming to do : ) I need to accomodate for maybe something in the ballpark of approx. 50Gbit+ on average for starters I'd guess w/ room for future growth...rules probably shouldn't be anything too crazy I don't think.</div><div><br></div><div>I'd like to maybe find a 2x100G or 1x100G NIC that has somewhat low administrative overhead that I can hand off to operations folks, so once it's running I won't have to deal with tinkering w/ it too much after patching the server and updating to new kernel versions. I've used Myricom cards in the past with Suri on smaller ~1-10G links so I'm familiar but I'd like to move away from those and I'm not even sure they have anything in the 100G market at the moment anyhow. Are there similar (low admin. overhead) NIC vendor options out there now at 100G that folks know of?</div><div><br></div><div>What's the administrative overhead when patching/updating etc. when running Suri with Intel/Mellanox NIC's? Does Mellanox have a 100G option? I believe Intel's is either about to launch or may have already launched?</div><div><br></div><div>Best,</div><div><br></div><div>-Drew<br><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 18, 2019 at 5:13 PM Michał Purzyński <<a href="mailto:michalpurzynski1@gmail.com" target="_blank">michalpurzynski1@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>Let's argue for a moment that using dedicated capture cards is not necessary anymore, because your vanilla Linux has all you need, especially with Suricata 5.0 and XDP. How does that sound??</div><div><br></div><div>I'd build a cluster instead of a single 100Gbit machine, for reliability reasons, unless your space is limited.</div><div><br></div><div>Intel and Mellanox 40Gbit cards can handle 20-40Gbit/sec on a fairly commodity hardware. It totally depends on your rules, of course.<br></div><div><br></div><div>Yes, everyone was expecting that ;)</div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Oct 18, 2019 at 8:38 AM Drew Dixon <<a href="mailto:dwdixon@umich.edu" target="_blank">dwdixon@umich.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi, I wanted to revive this thread as I'm currently exploring the same- 100G+ w/ Suricata. I'm specifically interested in NIC recommendations, here I see a "Napatech NT100E3-1-PTP" is being used which I will look into a bit, I also saw they offer a compact "NT200A02-SCC-2×100/40" which may be ideal for my purposes, however I wanted to poll the community a bit-<div><br></div><div>Do folks have other 100G NIC recommendations that play very well with Suricata w/ minimal administrative overhead? It could maybe even be something like 2x40G if there are more options presently, but 1x100G would likely really be best looking forward.</div><div><br></div><div>I did see that Intel is about to (or may have by now) release their 800 series NIC's w/ a 100G option FWIW. In general I haven't heard much of anything on the top 100G NIC recommendations w/ Suricata.</div><div><br></div><div>Many thanks in advance-</div><div><br></div><div>Best,</div><div><br></div><div>-Drew</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Aug 1, 2019 at 5:28 PM Peter Manev <<a href="mailto:petermanev@gmail.com" target="_blank">petermanev@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">@Daniel <div>What type of traffic is that and what rules are you planing on using?</div><div><br></div><div><br></div><div>Thanks</div><div><br></div><div><br><div dir="ltr">On 1 Aug 2019, at 22:19, Nelson, Cooper <<a href="mailto:cnelson@ucsd.edu" target="_blank">cnelson@ucsd.edu</a>> wrote:<br><br></div><blockquote type="cite"><div dir="ltr">
<div>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">Should be fine for ISP traffic.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">We are doing 20Gbit with 48 worker threads on an older AMD Piledriver box and it’s around 10-15% loaded with the ‘ondemand’ CPU governor.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">Suricata is primarily I/O bound if you are using the Hyperscan matcher and given you have a more modern bus and caching sub-system than us you should be under 50% CPU @peak. This is my personal sizing recommendation
to keep packet drops under 1%.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">If you are having performance issues or packet loss; make sure you have flow bypass for tcp and tls.
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)">-Coop<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color:rgb(31,73,125)"><u></u> <u></u></span></p>
<div>
<div style="border-color:rgb(225,225,225) currentcolor currentcolor;border-style:solid none none;border-width:1pt medium medium;padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> Oisf-users <<a href="mailto:oisf-users-bounces@lists.openinfosecfoundation.org" target="_blank">oisf-users-bounces@lists.openinfosecfoundation.org</a>>
<b>On Behalf Of </b>Daniel Wallmeyer<br>
<b>Sent:</b> Thursday, August 1, 2019 1:14 PM<br>
<b>To:</b> '<a href="mailto:oisf-users@lists.openinfosecfoundation.org" target="_blank">oisf-users@lists.openinfosecfoundation.org</a>' <<a href="mailto:oisf-users@lists.openinfosecfoundation.org" target="_blank">oisf-users@lists.openinfosecfoundation.org</a>><br>
<b>Subject:</b> [Oisf-users] Hardware specs for monitoring 100GB<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">Hey fellow mobsters,<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Looking to verify that we have spec’d our hardware correctly for monitoring 100GB:<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">2 x Intel(R) Xeon(R) Gold 6136 CPU<u></u><u></u></p>
<p class="MsoNormal">256GB of RAM<u></u><u></u></p>
<p class="MsoNormal">Napatech NT100E3-1-PTP<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">The traffic will be fed via a single network tap.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Will this be enough hardware to deal with 100Gb/s of traffic?<u></u><u></u></p>
<p class="MsoNormal">At the very least it would be great to know if the CPU and RAM is enough, we can work with Napatech to get the right card.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10pt">Thanks,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:10pt">Dan</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:12pt;font-family:"Times New Roman",serif">This message and attachments may contain confidential information. If it appears that this message was sent to you by mistake, any retention, dissemination, distribution
or copying of this message and attachments is strictly prohibited. Please notify the sender immediately and permanently delete the message and any attachments.
<br>
<br>
. . . . .<u></u><u></u></span></p>
</div>
</div></blockquote><blockquote type="cite"><div dir="ltr"><span>_______________________________________________</span><br><span>Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org" target="_blank">oisf-users@openinfosecfoundation.org</a></span><br><span>Site: <a href="http://suricata-ids.org" target="_blank">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/" target="_blank">http://suricata-ids.org/support/</a></span><br><span>List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a></span><br><span></span><br><span>Conference: <a href="https://suricon.net" target="_blank">https://suricon.net</a></span><br><span>Trainings: <a href="https://suricata-ids.org/training/" target="_blank">https://suricata-ids.org/training/</a></span></div></blockquote></div></div>_______________________________________________<br>
Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org" target="_blank">oisf-users@openinfosecfoundation.org</a><br>
Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/support/</a><br>
List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
<br>
Conference: <a href="https://suricon.net" rel="noreferrer" target="_blank">https://suricon.net</a><br>
Trainings: <a href="https://suricata-ids.org/training/" rel="noreferrer" target="_blank">https://suricata-ids.org/training/</a></blockquote></div>
_______________________________________________<br>
Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org" target="_blank">oisf-users@openinfosecfoundation.org</a><br>
Site: <a href="http://suricata-ids.org" rel="noreferrer" target="_blank">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/" rel="noreferrer" target="_blank">http://suricata-ids.org/support/</a><br>
List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" rel="noreferrer" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
<br>
Conference: <a href="https://suricon.net" rel="noreferrer" target="_blank">https://suricon.net</a><br>
Trainings: <a href="https://suricata-ids.org/training/" rel="noreferrer" target="_blank">https://suricata-ids.org/training/</a></blockquote></div></div>
</blockquote></div>
</blockquote></div></div>