[Oisf-users] autofp vs workers - updated comparison?

[Peter Manev]

On Mon, Jul 20, 2015 at 9:53 PM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> None of the runmodes are not magic and suricata has a hard limit re: the
> number of packets any single core can process per second, depending on
> your configuration.  Additionally, I've found that there are two ways
> the suricata process can drop packets.  Either the packet socket buffers
> can be overflowed directly from the NIC, for example via a DOS attack,
> or the AF_PACKET ring buffer can fill up.  The latter is usually due to
> overhead from the ruleset or setting a flow inspection depth that is too
> big.
>
> On a 16 core system using the hardware RSS feature of a modern NIC and
> the workers+AF_PACKET/mmap is already the optimum solution (and I've
> done extensive testing of all runmodes).  If you are still dropping
> packets you should follow the following process.
>
> 1.  Look for SYN floods to/from hosts on your network.  This was our
> primary issue, which in our case appeared to be associated with
> something called the 'ChinaZ' botnet.
>
> 2.  If you allow streaming video on your network, particularly netflix,
> try filtering out your ISPs netflix caches via bpf filters or pass
> rules.  Bpf filters are the preferred option.
>
> 3.  If you want to do deep flow inspection on http traffic, particularly
> with the web_client rules and/or lua scripts, you are going to need at
> least 32 cores.  AFAIK there is a hard limit on 16 RSS queues per NIC
> (at least on the intel chipsets), so you will have to use a second NIC
> or interface and use an external load-balancer.

Just as an info note -
XL710  (http://www.intel.com/content/www/us/en/embedded/products/networking/xl710-10-40-controller-datasheet.html)
has 64 RSS per port.

>
> It's also a good idea to increase the packet socket buffers from the
> defaults.
>
> We are the largest network in San Diego and are running ~23k ETPRO rules
> on a single 10G/16 core sensor.  Packet loss is currently well under 1%,
> using the process defined above (plus some custom filters).  It's really
> more a manner of tuning your sensor/rules to manage your network traffic
> vs. the runmode.

Couldn't agree more here.

>
> - -Coop
>
> On 7/20/2015 8:53 AM, Rasmor, Zachary R wrote:
>> The conventional wisdom from users on this thread, as well as from the
>> Suricata training I attended in Virgina this year, seem to suggest that
>> ‘workers’ is the preferred runmode. However, my testing has shown
>> various circumstances where I’m dropping packets in workers mode due to
>> 1 or 2 (out of 16) threads pegged at ~99-100% CPU. Also, any costly Lua
>> add-ons raise the likelihood of holding up the pipeline and dropping
>> packets in workers mode. The load balancing aspects of autofp make it a
>> more appealing and logical option, in my mind.
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
>
> iQEcBAEBAgAGBQJVrUO1AAoJEKIFRYQsa8FWbpMH+gOT8m1dLHO+SST7pALCJS5A
> wjsX1fxIK7eIHEHV2fPX7+n5quo1twCnjSo40o73IWH3z9ZEyi/Svj5rXloycJBm
> +uxug3INpZaPEBkRdoC20HfJewVdS58M1I0wWe35WPdajzxggJ6aEO3M571FhsdP
> xUogy0dHoFYmjzRKVmWMDM7j64Mdr6E3qp2pM/ktg8DSLKbjseWS/qZvocICeCaP
> QIq2JkRuJX1m7bMoUc25f3xST18mpNxHxbHHR1srj7zsqv12ieluDML9TCJH/xnm
> ZKt+IXDK34Fv8zwvsol2LPwGKQ5wh1sL/9bFGjpvYyrJbqaKGREf7wQd3gvS6iY=
> =ykhJ
> -----END PGP SIGNATURE-----
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net



-- 
Regards,
Peter Manev