Anoop,<br> With max-pending-packets set to 50,000 and 48 CPU's I get performance around 135,000 packets/sec. With mpp at 50,000 and only 4 CPU's I get performance around 31,343 packets/sec. Both of these are with --runmode=autofp enabled. <br>
<br>Interestingly enough, when I run 4 CPU's in autofp mode I get 31,343 pps, and when I run 48 CPU's in auto mode I also get 31,343 pps.<br><br> I have to admit that I don't quite follow your explanation about the thread usage below. In layman's terms how will this affect the performance of suricata? In my case I seem to be getting great performance increases, but I can't see what downside there might be with the cache.<br>
<br>Thanks,<br>Gene<br><br><div class="gmail_quote">On Sat, Aug 13, 2011 at 10:05 PM, Anoop Saldanha <span dir="ltr"><<a href="mailto:poonaatsoc@gmail.com">poonaatsoc@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br><br><div class="gmail_quote"><div class="im">On Thu, Aug 11, 2011 at 12:24 AM, Gene Albin <span dir="ltr"><<a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div>So I'm running in autofp mode and I increased the max-pending-packets from 50 to 500, then 5000, then 50000. I saw a dramatic increase from:</div><div>50 to 500 (17000 packets/sec @ 450sec to 57000 pps@140s)</div>
<div>not quite as dramatic from:</div><div>500 to 5000 ( to 85000pps@90s)</div><div>and about the same from:</div><div>5000 to 50000 (to 135000pps@60s)</div><div><br></div><div>My question now is about the tradeoff mentioned in the config file. Mentions negatively impacting caching. How does it impact caching? Will I see this when running pcaps or in live mode? </div>
<div><br></div><div>Thanks,</div><div>Gene</div><div><div></div><div><div><br></div></div></div></blockquote></div><div><br>Probably polluting_the_cache/breaking_the_cache_coherency for the data used by other packets. Either ways I wouldn't second guess the effects of cache usage when it comes to multiple threads probably ruining data loaded by some other thread. I would just be interested about locality of reference with respect to data used by one thread for whatever time slice it is on the cpu.<br>
<br>** I see that you have tested with max-pending-packets set to 50,000. Can you check how Suricata scales from 4 cpu cores to 32 cpu cores, with these 50,000 max-pending-packets, and post the results here?<br> </div><div class="im">
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div><div><div><div class="gmail_quote">On Thu, Aug 4, 2011 at 1:07 PM, saldanha <span dir="ltr"><<a href="mailto:poonaatsoc@gmail.com" target="_blank">poonaatsoc@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0pt 0pt 0pt 0.8ex;border-left:1px solid rgb(204, 204, 204);padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div>
On 08/03/2011 08:50 AM, Gene Albin wrote:
</div><blockquote type="cite"><div><div></div><div>So I just installed Suricata on one of our research
computers with lots of cores available. I'm looking to see what
kind of performance boost I get as I bump up the CPU's. After my
first run I was surprised to see that I didn't get much of a boost
when going from 8 to 32 CPUs. I was running a 6GB pcap file with
a about 17k rules loaded. The first run on 8 cores took 190sec.
The second run on 32 cores took 170 sec. Looks like something
other than CPU is the bottle neck. <br>
<br>
My first guess is Disk IO. Any recommendations on how I could
check/verify that guess?<br>
<br>
Gene<br>
<br>
-- <br>
Gene Albin<br>
<a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a><br>
<br>
<br>
<fieldset></fieldset>
<br>
</div></div><div><pre>_______________________________________________
Oisf-users mailing list
<a href="mailto:Oisf-users@openinfosecfoundation.org" target="_blank">Oisf-users@openinfosecfoundation.org</a>
<a href="http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a>
</pre>
</div></blockquote>
<tt><br>
* forgot to reply to the list previously<br>
<br>
Hey Gene.<br>
<br>
Can you test by increasing the max-pending-packets in the
suricata.yaml file to a higher value. You can try one run with a
value of 500 and then try higher values(2000+ suggested. More the
better, as long as you don't hit swap).</tt><br>
<br>
<tt>Once you have set a higher max-pending-packets you can try
running suricata in autofp runmode. autofp mode runs suricata in
flow-pinned mode</tt>. <tt>To do this add this option to your
suricata command line "--runmode=autofp. "<br>
<br>
sudo suricata -c ./suricata.yaml -r your_pcap.pcap
--runmode=autofp<br>
<br>
With max-pending-packets set to a higher value and with
--runmode=autofp, you can test how suricata scales from 4 to 32
cores.<br>
<br>
<br>
</tt>
</div>
<br>_______________________________________________<br>
Oisf-users mailing list<br>
<a href="mailto:Oisf-users@openinfosecfoundation.org" target="_blank">Oisf-users@openinfosecfoundation.org</a><br>
<a href="http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
<br></blockquote></div><br><br clear="all"><br>-- <br>Gene Albin<br><a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a><br><br>
</div>
</div></div><br>_______________________________________________<br>
Oisf-users mailing list<br>
<a href="mailto:Oisf-users@openinfosecfoundation.org" target="_blank">Oisf-users@openinfosecfoundation.org</a><br>
<a href="http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
<br></blockquote></div></div><font color="#888888"><br><br clear="all"><br>-- <br>Anoop Saldanha<br><br>
</font></blockquote></div><br><br clear="all"><br>-- <br>Gene Albin<br><a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a><br><br>