[Oisf-users] autofp vs workers - updated comparison?

[Cooper F. Nelson]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

None of the runmodes are not magic and suricata has a hard limit re: the
number of packets any single core can process per second, depending on
your configuration.  Additionally, I've found that there are two ways
the suricata process can drop packets.  Either the packet socket buffers
can be overflowed directly from the NIC, for example via a DOS attack,
or the AF_PACKET ring buffer can fill up.  The latter is usually due to
overhead from the ruleset or setting a flow inspection depth that is too
big.

On a 16 core system using the hardware RSS feature of a modern NIC and
the workers+AF_PACKET/mmap is already the optimum solution (and I've
done extensive testing of all runmodes).  If you are still dropping
packets you should follow the following process.

1.  Look for SYN floods to/from hosts on your network.  This was our
primary issue, which in our case appeared to be associated with
something called the 'ChinaZ' botnet.

2.  If you allow streaming video on your network, particularly netflix,
try filtering out your ISPs netflix caches via bpf filters or pass
rules.  Bpf filters are the preferred option.

3.  If you want to do deep flow inspection on http traffic, particularly
with the web_client rules and/or lua scripts, you are going to need at
least 32 cores.  AFAIK there is a hard limit on 16 RSS queues per NIC
(at least on the intel chipsets), so you will have to use a second NIC
or interface and use an external load-balancer.

It's also a good idea to increase the packet socket buffers from the
defaults.

We are the largest network in San Diego and are running ~23k ETPRO rules
on a single 10G/16 core sensor.  Packet loss is currently well under 1%,
using the process defined above (plus some custom filters).  It's really
more a manner of tuning your sensor/rules to manage your network traffic
vs. the runmode.

- -Coop

On 7/20/2015 8:53 AM, Rasmor, Zachary R wrote:
> The conventional wisdom from users on this thread, as well as from the
> Suricata training I attended in Virgina this year, seem to suggest that
> ‘workers’ is the preferred runmode. However, my testing has shown
> various circumstances where I’m dropping packets in workers mode due to
> 1 or 2 (out of 16) threads pegged at ~99-100% CPU. Also, any costly Lua
> add-ons raise the likelihood of holding up the pipeline and dropping
> packets in workers mode. The load balancing aspects of autofp make it a
> more appealing and logical option, in my mind.

- -- 
Cooper Nelson
Network Security Analyst
UCSD ACT Security Team
cnelson at ucsd.edu x41042
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)

iQEcBAEBAgAGBQJVrUO1AAoJEKIFRYQsa8FWbpMH+gOT8m1dLHO+SST7pALCJS5A
wjsX1fxIK7eIHEHV2fPX7+n5quo1twCnjSo40o73IWH3z9ZEyi/Svj5rXloycJBm
+uxug3INpZaPEBkRdoC20HfJewVdS58M1I0wWe35WPdajzxggJ6aEO3M571FhsdP
xUogy0dHoFYmjzRKVmWMDM7j64Mdr6E3qp2pM/ktg8DSLKbjseWS/qZvocICeCaP
QIq2JkRuJX1m7bMoUc25f3xST18mpNxHxbHHR1srj7zsqv12ieluDML9TCJH/xnm
ZKt+IXDK34Fv8zwvsol2LPwGKQ5wh1sL/9bFGjpvYyrJbqaKGREf7wQd3gvS6iY=
=ykhJ
-----END PGP SIGNATURE-----