[Oisf-devel] [PATCH 0/6] RFC: parametrable cpu affinity settings

[Eric Leblond]

Hi,

Le vendredi 12 novembre 2010 à 12:23 +0100, Victor Julien a écrit :
> Thanks Eric, for looking into this. Definitely an interesting approach.
> More inline.
> 
> Eric Leblond wrote:
> > This patchset is a proposal of implementation which enable the
> > configuration of cpu affinity (and priority) settings from the
> > suricata.yaml configuration file.
> > 
> > The idea is to separate the threads info families and to be able to set
> > affinity and priority of the treads on a per-family basis.
> > Given a rapid study of different runmodes, I've created the following
> > families:
> >  - receive_cpu_set: for receive threads
> >  - decode_cpu_set: for decode threads
> >  - stream_cpu_set: for stream threads
> >  - detect_cpu_set: for detect threads
> >  - verdict_cpu_set: for verdict and reject threads
> >  - output_cpu_set: for output threads
> 
> What would be the best way to deal with runmodes that run decode and
> stream in a single thread? (like RunModeFilePcapAuto for example) or
> stream and detect in a single thread (like RunModeFilePcapAutoFp for
> example).

Just assigned them to one family or the other. Ie, just choose an
arbitrary set between (decode or stream or even to another). You just
need when defining runmode to call
 TmThreadSetCPU(tv_decode, DECODE_CPU_SET);
for each threads you want to put on this set.

The thread affinity only contains generic information (cpu set and
priority). I've mapped them onto specific names but this was just for
convenience. This may be a mistake and we could define arbitrary
affinity type without too much work if necessary.

> 
> > For each family, it is possible to set:
> >  - cpu: define the cpu_set to attribute to the threads of the family
> >  	this is a list containing or single integer or range or the
> > 	keyword 'all' which is use to give all CPUs to the threads.
> >  - mode: [exclusive|balanced] If balanced is set, the thread of the family
> > 	are balanced on all cpus of the set. If exclusive is set, each thread
> > 	of the family is given a specific CPU.
> >  - prio: [high|medium|low]
> > 
> > An example of obtained configuration is the following:
> >   cpu_affinity:
> >     - receive_cpu_set:
> >         cpu: [ 0 ]  # include only these cpus in affinity settings
> >         prio: "high"
> >     - decode_cpu_set:
> >         cpu: [ 0, 1 ]
> >         mode: "balanced"
> 
> Would this mean that one or more "Decode" threads would be run where the
> OS balances them over 0 and 1?

Yes, this configuration assigns "Decode" threads to CPU 0 and 1. In
"balanced" mode, the system take care of the assignement of the threads.
In "exclusive" mode, each "Decode" thread is specifically assigned a
CPU.

>  So it would actually be a way to limit a
> (set of) thread(s) to certain cores/cpu's?

Yes, this is done for that.

> That would be interested as we could possible configure it to balance a
> set of threads on the cores of a single cpu in a multi-cpu system.

It could be easy to add a core_cpu_set family which could be used to set
CPUs used by core threads.

BR,
Eric
> 
> Cheers,
> Victor
> 
> >     - stream_cpu_set:
> >         cpu: [ "0-1" ]
> >     - detect_cpu_set:
> >         cpu: [ "all" ]
> >         mode: "exclusive" # run detect threads in these cpus
> >     - verdict_cpu_set:
> >         cpu: [ 0 ]
> >     - output_cpu_set:
> >         cpu: [ "all" ]
> > 
> > Patchset statistics:
> >  src/Makefile.am     |    2 +-
> >  src/runmodes.c      |   80 +++++++-----------
> >  src/runmodes.h      |    3 +
> >  src/threadvars.h    |    2 +
> >  src/tm-threads.c    |   76 ++++++++++++++---
> >  src/tm-threads.h    |    1 +
> >  src/util-affinity.c |  234 +++++++++++++++++++++++++++++++++++++++++++++++++++
> >  src/util-affinity.h |   62 ++++++++++++++
> >  suricata.yaml       |   17 ++++
> >  9 files changed, 413 insertions(+), 64 deletions(-)
> > 
> > The patchset is not complete as :
> >  * The WIN32 case may not be properly handled
> >  * I've only converted the NFQAuto runmode to the new system
> > But as I'm planning to do some work other these patches, I send it to the list to
> > check if I'm going in the good direction.
> > 
> > I've done some light testing on my laptop which has only dual core. And I've been
> > able to gain a second out of ten in a trivial test. I thus really think that we could
> > have some real performance increase by tuning the introduced parameters.
> > 
> > BR,
> > --
> > Eric Leblond <eleblond at edenwall.com>
> > 
> > 
> > _______________________________________________
> > Oisf-devel mailing list
> > Oisf-devel at openinfosecfoundation.org
> > http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
> 
>