[Oisf-users] Performance on multiple CPUs

[Victor Julien]

On 08/27/2011 12:23 AM, Gene Albin wrote:
> One of the smart guys here suggested a modification to suricata's source
> code that will allow it to grab the number of processors from an environment
> variable instead of from the system.  This would obviously not be something
> that the wider community would use, but I'd like to try it.
> 
> Basically the suggestion is to change:
> 
> nprocs = sysconf(_SC_NPROCESSORS_ONLN);
> 
> to:
> 
> nprocs = atoi( getenv("NPROCS") );
> 
> Then add an environment variable that is created every time I specify the
> number of processors when creating the job:
> 
> export NPROCS=$(cat $PBS_NODEFILE|wc –l)
> 
> This will ensure that Suricata runs with the specified number of processors,
> and not what it sees from the system (which will be more than have been
> allocated by the job controls.  Any thoughts on that idea?
> 
> My problem is that I don't know where or how to make that one line change to
> the Suricata source code.  Any hints on how to proceed?
> 
> Gene
> 
> 
> On Fri, Aug 26, 2011 at 12:52 AM, Victor Julien <victor at inliniac.net> wrote:
> 
>> On 08/25/2011 08:12 PM, Gene Albin wrote:
>>> Does anyone know how Suricata determines the number of CPU's available on
>>> the machine?  I'm asking because I need to know how to tell Suricata how
>>> many to use when submitting jobs on this research computer.
>>  /proc/cpuinfo
>>> always returns the total number of physical CPU's, and in my case I want
>> to
>>> test performance on a subset of the maximum (48).
>>
>>> What environment variable, or other parameter, do I need to pass to
>> Suricata
>>> in order to specify the number of CPUs available.
>>
>> It seems we get it from the system using:
>>
>> nprocs = sysconf(_SC_NPROCESSORS_ONLN);
>>
>> This can't be controlled in Suricata. Maybe it can in the system, but I
>> wouldn't know how. If you figure it out, please let us know.

Can you open up a feature request for this?

Btw, you are able to control some things already. In the "auto" runmode
(default), the number of detect threads in controlled by the
"threading.detect_thread_ratio" setting. Try setting it to 0.5 for 24
threads, 0.25 for for 12, etc.

Using the threading.cpu-affinity settings you should even be able to
control which cores are used, instead of having the kernel figure that
out automatically. Check the examples in the suricata.yaml and the
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml
doc.

Cheers,
Victor

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------