[Oisf-users] 20Gbps - is it possible?!

[Cooper F. Nelson]

You need more cores if you want to run the full ET Pro ruleset on all
traffic.  I would recommend at least 24 Intel cores or 32 AMD cores per
10 Gbps interface.  And to be honest that may not even be enough, as
that is assuming filtering of "Top Talkers" like NetFlix and YouTube.

It's possible if/when suricata implements the Intel "HyperScan" SSE
optimizations that performance will improve.

A couple general hints (and to confirm what Peter Manev stated).

1.  You should be using suricata 3.0 at this point.  Performance is
improved compared to prior versions.

2.  Make sure you are using a 'fresh' kernel and NIC drivers.

3.  Get suricata running with all rules files disabled first, to ensure
that the packet capture and flow tracking is working properly.  Make
sure to disable all off-loading of the NIC.

4.  Assuming that is working properly, start enabling just the rule
files that have the highest value for your organization.  For example,
the web_client.rules are very expensive in terms of CPU cycles, so
consider leaving them disabled for now.

If you are still having performance problems contact me off-list and I
can give you some tips to increase performance on over-subscribed hardware.

-Coop

On 3/15/2016 5:11 AM, Matthew George wrote:
> Dear Suricata users - please help,
> 
> I am interested in getting Suricata running to rates up to and over 20Gbps.
> 
> We are using a fairly impressive server spec i.e. 20 cores and 128GB ram
> etc.
> I also have a signature offload card spliced into the bottom of our
> modified Suricata (based on 2.0.9) giving about a 20-30% reduction in
> CPU per worker thread without any negative impacts on alerts. The card
> also does 0 copy DMA and load sharing to 16 worker cores via a
> proprietary implementation not that dissimilar from PF_RING.
> 
> The throughput on the system however when running the full ET Pro
> ruleset is no where near what we'd like or it appears what you guys are
> getting so my questions is what are we doing wrong?
> 
> Should we use a different code base, a bigger server or tune the rules?
> 
> Any help would be greatly appreciated,  
> 
> Matt
> 
> 
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 9-11 in Washington, DC: http://oisfevents.net
> 


-- 
Cooper Nelson
Network Security Analyst
UCSD ITS Security Team
cnelson at ucsd.edu x41042

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160315/be7aaf0d/attachment-0002.sig>