[Oisf-users] Processing threads limit of 16?

[Barkley, Joey]

I do have the intel card and tried this. It doesn’t appear to have worked. Once I did the modprobe I re-ran the script to reset the irq balance stuff but I guess that isn’t how to set the balancing manually...


> On Feb 13, 2015, at 11:58 AM, Eric Leblond <eric at regit.org> wrote:
> 
> Hi Cooper,
> 
> On Fri, 2015-02-13 at 09:50 -0800, Cooper F. Nelson wrote:
>> What kind of NIC do you have?
>> 
>> If you have an Intel card with the ixgbe driver you can try setting the
>> number of RSS queues to 32:
>> 
>>> http://stackoverflow.com/questions/23730268/ixgbe-setting-the-number-of-rx-tx-queues
> 
> It won't work there is a limitation at 16 for the load balancing done by
> the card. You can have more RX queues but you need to do the "balancing"
> by yourself in that case.
> 
> ++
> 
>> 
>> -Coop
>> 
>> On 2/13/2015 9:45 AM, Barkley, Joey wrote:
>>> Yes, sort of…
>>> 
>>> I have 64 cores available. I set threads: 32 and cpu: [ 0-31].
>>> 
>>> It creates 32 threads, but only 16 of them show up as actually
>>> processing data in the stats log file. cores 15-31 always show 0 for
>>> kernel_packets and kernel_drops. Does that mean it just doesn’t need
>>> the extra cores? I do have some (very minimal) drops, but I’d think
>>> that if I had anything more than 0 it would start using more cores
>>> for processing.
>>> 
>>> And if it matters, this is 2.1beta3.
>>> 
>>> 
>>>> On Feb 13, 2015, at 11:36 AM, Cooper F. Nelson <cnelson at ucsd.edu>
>>>> wrote:
>>>> 
>>> The number of threads is governed by this configuration:
>>> 
>>>>>> af-packet: - interface: eth2 # Number of receive threads (>1
>>>>>> will enable experimental flow pinned # runmode) threads: 16
>>> 
>>> ... and this:
>>> 
>>>>>> - detect-cpu-set: cpu: [ 0-15 ] mode: "exclusive" # run detect
>>>>>> threads in these cpus # Use explicitely 3 threads and don't
>>>>>> compute number by using # detect-thread-ratio variable: 
>>>>>> #threads: 2 prio: default: "high"
>>> 
>>> Are they both set to use all available cores?
>>> 
>>> -Coop
>>> 
>>> On 2/13/2015 8:47 AM, Barkley, Joey wrote:
>>>>>> All,
>>>>>> 
>>>>>> I have made significant progress in tuning our suricata
>>>>>> instance to handle our network traffic. Thanks to everyone who
>>>>>> has helped me.
>>>>>> 
>>>>>> Question: Regardless of how many threads I configure, suricata
>>>>>> only shows kernel_packets and kernel_drops for the first 16
>>>>>> threads. Is there a hard limit of 16 “usable” threads? My
>>>>>> system has 64 cores but it doesn’t seem like I’m able to use
>>>>>> more than 16 cores. Have I just configured something
>>>>>> incorrectly? I have primarily followed advice on this list and
>>>>>> also on 
>>>>>> http://pevma.blogspot.se/2013/12/suricata-and-grand-slam-of-open-source_8.html
>>>>>> for AF_PACKET configuration. Would it help for me to assign my 
>>>>>> management-cpu-set to different cores than my detect-cpu-set? I
>>>>>> seem to remember reading that would not be good as it would
>>>>>> adversely impact performance. Or possibly, would increasing the
>>>>>> detect-thread-ratio work? I’m using cluster_cpu and not sure
>>>>>> how that would be affected by changing those settings.
>>>>>> 
>>>>>> Advice welcome.
>>>>>> 
>>>>>> Thanks, Joey
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________ Suricata IDS
>>>>>> Users mailing list: oisf-users at openinfosecfoundation.org Site:
>>>>>> http://suricata-ids.org | Support:
>>>>>> http://suricata-ids.org/support/ List:
>>>>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>>> 
>>>>>> 
>> Training now available: http://suricata-ids.org/training/
>>>>>> 
>>> 
>>> 
>>>> 
>>> 
>>> 
>>> _______________________________________________ Suricata IDS Users
>>> mailing list: oisf-users at openinfosecfoundation.org Site:
>>> http://suricata-ids.org | Support: http://suricata-ids.org/support/ 
>>> List:
>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users 
>>> Training now available: http://suricata-ids.org/training/
>>> 
>> 
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> Training now available: http://suricata-ids.org/training/
> 
> -- 
> Eric Leblond <eric at regit.org>
> 
>