[Oisf-users] High CPU Load with Small Ruleset at 10Gbit/s

[Peter Manev]

> On 19 Aug 2019, at 12:06, Eric Urban <eurban at umn.edu> wrote:
> 
> We use Myricom cards with about 35K rules loaded.  None of our cores run near 100% load.  I saw one period of 6Gbps traffic in the last week on one of our Suricata instances where one core had 43% usage but the other 8 were at about 12%.
> 
> Have you looked at the Suricata Extreme Performance Tuning guide at https://github.com/pevma/SEPTun?  The cpu-affinity settings seem to be covered more in depth there than at the link that you posted.  
> 
> Also, the section at https://suricata.readthedocs.io/en/latest/performance/high-performance-config.html could be of help.  We don't use the custom setting recommended there but do use "high" for the profile and "full" for the sgh-mpm-context.  Note the warning about significantly longer rule load times though.

+1 for “high” context.

Fabian:
What is your max-pending packets value ?
Also in some tests/live set ups it is common to see some CPUs busier than others.


> 
> -- 
> Eric Urban
> University Information Security | Office of Information Technology | it.umn.edu
> University of Minnesota | umn.edu
> eurban at umn.edu
> 
> 
>> On Fri, Aug 16, 2019 at 10:24 AM Fabian Franz <fabfaeb at googlemail.com> wrote:
>> Hi Everyone,
>> 
>> I am having a problem with my Suricata setup and hope that someone here as a hint for me:
>> I run suricata 4.1.4 together with a myricom card on a server with 128 gigs of RAM and two 16core(+HT) Intel CPUs.
>> The SNF settings are 30 rings and 32/8gig for ringsizes. 
>> 
>> As long as I do not deploy any rules, suricata runs smoothly with ~20% CPU load per (worker) core at 9-10 Gbit/s network traffic. However, when I deploy even small rulesets (e.g. et-shellcode) the CPU load skyrockets with 100% for 3-6 cores and the rest at around 50%. After a few moments, packets are dropped, with the SNF drop ring full counter increasing rapidly (at 9-10Gbit/s, as before). I use hyperscan as mpm-algo and tried to followed the recommendations at https://home.regit.org/2012/07/suricata-to-10gbps-and-beyond/ .
>> However, I was not able to follow the recommendations regarding IRQ, since those seemed pretty NIC specific. Is this setup also relevant for myricom cards? 
>> Additionally, I obviously do not use AF_PACKET but libpcap with 30 threads.  
>> 
>> To test the bandwidth I used iperf with 30 parallel connections. Could this be the reason why only some of the cores are running at 100% load? If so, are there any other possiblities to simulate the bandwidth more realistically?
>> 
>> Are there any myricom users here that could share performance hints for myricom+suricata? I feel that (hardware-wise) my setup should have no problem handling 10Gbit/s with a decent ruleset, right?
>> 
>> Thanks a lot
>> 
>> Fabian
>> 
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> 
>> Conference: https://suricon.net
>> Trainings: https://suricata-ids.org/training/
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> 
> Conference: https://suricon.net
> Trainings: https://suricata-ids.org/training/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190819/2bc74081/attachment.html>