[Oisf-users] Processing threads limit of 16?

[Eric Leblond]

Hi Cooper,

On Fri, 2015-02-13 at 09:50 -0800, Cooper F. Nelson wrote:
> What kind of NIC do you have?
> 
> If you have an Intel card with the ixgbe driver you can try setting the
> number of RSS queues to 32:
> 
> > http://stackoverflow.com/questions/23730268/ixgbe-setting-the-number-of-rx-tx-queues

It won't work there is a limitation at 16 for the load balancing done by
the card. You can have more RX queues but you need to do the "balancing"
by yourself in that case.

++

> 
> -Coop
> 
> On 2/13/2015 9:45 AM, Barkley, Joey wrote:
> > Yes, sort of…
> > 
> > I have 64 cores available. I set threads: 32 and cpu: [ 0-31].
> > 
> > It creates 32 threads, but only 16 of them show up as actually
> > processing data in the stats log file. cores 15-31 always show 0 for
> > kernel_packets and kernel_drops. Does that mean it just doesn’t need
> > the extra cores? I do have some (very minimal) drops, but I’d think
> > that if I had anything more than 0 it would start using more cores
> > for processing.
> > 
> > And if it matters, this is 2.1beta3.
> > 
> > 
> >> On Feb 13, 2015, at 11:36 AM, Cooper F. Nelson <cnelson at ucsd.edu>
> >> wrote:
> >> 
> > The number of threads is governed by this configuration:
> > 
> >>>> af-packet: - interface: eth2 # Number of receive threads (>1
> >>>> will enable experimental flow pinned # runmode) threads: 16
> > 
> > ... and this:
> > 
> >>>> - detect-cpu-set: cpu: [ 0-15 ] mode: "exclusive" # run detect
> >>>> threads in these cpus # Use explicitely 3 threads and don't
> >>>> compute number by using # detect-thread-ratio variable: 
> >>>> #threads: 2 prio: default: "high"
> > 
> > Are they both set to use all available cores?
> > 
> > -Coop
> > 
> > On 2/13/2015 8:47 AM, Barkley, Joey wrote:
> >>>> All,
> >>>> 
> >>>> I have made significant progress in tuning our suricata
> >>>> instance to handle our network traffic. Thanks to everyone who
> >>>> has helped me.
> >>>> 
> >>>> Question: Regardless of how many threads I configure, suricata
> >>>> only shows kernel_packets and kernel_drops for the first 16
> >>>> threads. Is there a hard limit of 16 “usable” threads? My
> >>>> system has 64 cores but it doesn’t seem like I’m able to use
> >>>> more than 16 cores. Have I just configured something
> >>>> incorrectly? I have primarily followed advice on this list and
> >>>> also on 
> >>>> http://pevma.blogspot.se/2013/12/suricata-and-grand-slam-of-open-source_8.html
> >>>> for AF_PACKET configuration. Would it help for me to assign my 
> >>>> management-cpu-set to different cores than my detect-cpu-set? I
> >>>> seem to remember reading that would not be good as it would
> >>>> adversely impact performance. Or possibly, would increasing the
> >>>> detect-thread-ratio work? I’m using cluster_cpu and not sure
> >>>> how that would be affected by changing those settings.
> >>>> 
> >>>> Advice welcome.
> >>>> 
> >>>> Thanks, Joey
> >>>> 
> >>>> 
> >>>> _______________________________________________ Suricata IDS
> >>>> Users mailing list: oisf-users at openinfosecfoundation.org Site:
> >>>> http://suricata-ids.org | Support:
> >>>> http://suricata-ids.org/support/ List:
> >>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> >>>>
> >>>> 
> Training now available: http://suricata-ids.org/training/
> >>>> 
> > 
> > 
> >> 
> > 
> > 
> > _______________________________________________ Suricata IDS Users
> > mailing list: oisf-users at openinfosecfoundation.org Site:
> > http://suricata-ids.org | Support: http://suricata-ids.org/support/ 
> > List:
> > https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users 
> > Training now available: http://suricata-ids.org/training/
> > 
> 
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Training now available: http://suricata-ids.org/training/

-- 
Eric Leblond <eric at regit.org>