[Oisf-users] Performance on multiple CPUs

[Gene Albin]

Anoop,
  With max-pending-packets set to 50,000 and 48 CPU's I get performance
around 135,000 packets/sec.  With mpp at 50,000 and only 4 CPU's I get
performance around 31,343 packets/sec.  Both of these are with
--runmode=autofp enabled.

Interestingly enough, when I run 4 CPU's in autofp mode I get 31,343 pps,
and when I run 48 CPU's in auto mode I also get 31,343 pps.

  I have to admit that I don't quite follow your explanation about the
thread usage below.  In layman's terms how will this affect the performance
of suricata?  In my case I seem to be getting great performance increases,
but I can't see what downside there might be with the cache.

Thanks,
Gene

On Sat, Aug 13, 2011 at 10:05 PM, Anoop Saldanha <poonaatsoc at gmail.com>wrote:

>
>
> On Thu, Aug 11, 2011 at 12:24 AM, Gene Albin <gene.albin at gmail.com> wrote:
>
>> So I'm running in autofp mode and I increased the max-pending-packets from
>> 50 to 500, then 5000, then 50000.  I saw a dramatic increase from:
>> 50 to 500 (17000 packets/sec @ 450sec to 57000 pps at 140s)
>> not quite as dramatic from:
>> 500 to 5000 ( to 85000pps at 90s)
>> and about the same from:
>> 5000 to 50000 (to 135000pps at 60s)
>>
>> My question now is about the tradeoff mentioned in the config file.
>>  Mentions negatively impacting caching.  How does it impact caching?  Will I
>> see this when running pcaps or in live mode?
>>
>> Thanks,
>> Gene
>>
>>
> Probably polluting_the_cache/breaking_the_cache_coherency for the data used
> by other packets.  Either ways I wouldn't second guess the effects of cache
> usage when it comes to multiple threads probably ruining data loaded by some
> other thread.  I would just be interested about locality of reference with
> respect to data used by one thread for whatever time slice it is on the cpu.
>
> ** I see that you have tested with max-pending-packets set to 50,000.  Can
> you check how Suricata scales from 4 cpu cores to 32 cpu cores, with these
> 50,000 max-pending-packets, and post the results here?
>
>
>> On Thu, Aug 4, 2011 at 1:07 PM, saldanha <poonaatsoc at gmail.com> wrote:
>>
>>>  On 08/03/2011 08:50 AM, Gene Albin wrote:
>>>
>>> So I just installed Suricata on one of our research computers with lots
>>> of cores available.  I'm looking to see what kind of performance boost I get
>>> as I bump up the CPU's. After my first run I was surprised to see that I
>>> didn't get much of a boost when going from 8 to 32 CPUs.  I was running a
>>> 6GB pcap file with a about 17k rules loaded.  The first run on 8 cores took
>>> 190sec.  The second run on 32 cores took 170 sec.  Looks like something
>>> other than CPU is the bottle neck.
>>>
>>> My first guess is Disk IO.  Any recommendations on how I could
>>> check/verify that guess?
>>>
>>> Gene
>>>
>>> --
>>> Gene Albin
>>> gene.albin at gmail.com
>>>
>>>
>>>
>>> _______________________________________________
>>> Oisf-users mailing listOisf-users at openinfosecfoundation.orghttp://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>>
>>> * forgot to reply to the list previously
>>>
>>> Hey Gene.
>>>
>>> Can you test by increasing the max-pending-packets in the suricata.yaml
>>> file to a higher value.  You can try one run with a value of 500 and then
>>> try higher values(2000+ suggested.  More the better, as long as you don't
>>> hit swap).
>>>
>>> Once you have set a higher max-pending-packets you can try running
>>> suricata in autofp runmode.  autofp mode runs suricata in flow-pinned mode.
>>> To do this add this option to your suricata command line
>>> "--runmode=autofp.  "
>>>
>>> sudo suricata -c ./suricata.yaml -r your_pcap.pcap --runmode=autofp
>>>
>>> With max-pending-packets set to a higher value and with --runmode=autofp,
>>> you can test how suricata scales from 4 to 32 cores.
>>>
>>>
>>>
>>> _______________________________________________
>>> Oisf-users mailing list
>>> Oisf-users at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>>
>>
>>
>> --
>> Gene Albin
>> gene.albin at gmail.com
>>
>>
>> _______________________________________________
>> Oisf-users mailing list
>> Oisf-users at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>
>>
>
>
> --
> Anoop Saldanha
>
>


-- 
Gene Albin
gene.albin at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20110813/9d03f88b/attachment-0002.html>