Dave,<div> Thanks for the reply. It's possible that I'm I/O limited. Quite simply I drew my conclusion from the fact that when I run the same 6GB pcap file through Suricata via tcpreplay the CPU utilization rises up to between 13 and 22 percent per core (4 cores). It completes in just over 2 minutes. Once complete it drops back down to 0%. Looking at the processes during the run I notice that Suricata and tcpreplay are both in the 60% range (using top the process table shows the average across all CPUs, I think). However, when I run Suricata with the -r <filename> option the CPU utilization on all 4 CPU's barely increases above 1, which is where it usually sits when I run a live capture on this interface and the run takes around 4 minutes to complete.</div>
<div><br></div><div> As for the hardware I'm running this in a VM hosted on an ESX server. OS is CentOS 5.6, 4 cores and 4GB ram. Pcaps are on a 1.5TB drive attached to the server via fiberchannel (I think). Not sure how I can measure the latency, but up to this point I haven't had an issue.</div>
<div><br></div><div> For ruleset I'm using just the open ET ruleset optimized for suricata. That's 46 rule files and 11357 rules loaded. My suricata.yaml file is for the most part stock. (attached for your viewing pleasure)</div>
<div><br></div><div> So I'm really at a loss here why the -r option runs slower than tcpreplay --topspeed. The only explanation I see is that -r replays the file at the same speed it was recorded.</div><div><br></div>
<div> Appreciate any insight you could offer...</div><div><br></div><div>Gene</div>
<div><br><br><div class="gmail_quote">On Thu, Jul 14, 2011 at 6:50 PM, Dave Remien <span dir="ltr"><<a href="mailto:dave.remien@gmail.com" target="_blank">dave.remien@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br><br><div class="gmail_quote"><div><div></div><div>On Thu, Jul 14, 2011 at 7:14 PM, Gene Albin <span dir="ltr"><<a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi all,<div> I'm experimenting with replaying various pcap files in Suricata. It appears that the pcap files are replaying at the same speed they were recorded. I'd like to be able to replay them faster so that 1) I can stress the detection engine, and 2) expedite post-event analysis. </div>
<div><br></div><div> One way to accomplish this is by using tcpreplay -t, but when running on the same machine that takes lots of cycles away from Suricata and sends the recorded pcap traffic onto an interface that already has live traffic. </div>
<div><br></div><div> Is there some other way to replay captured traffic through Suricata at an accelerated speed?<br clear="all"></div></blockquote><div><br></div></div></div><div>Hmm - I've done pretty extensive replay of pcaps with Suricata. I have a 750GB pcap that was recorded over a 9 hour time range, and takes about 3.5 hours to be replayed through Suricata. The alerts generated show the pcap time (i.e., over the 9 hour range). The machine replaying the pcap is a 16 core box with a RAID array.</div>
<div> </div><div>Is it possible that you're I/O limited?</div><div><br></div><div>So... I guess I'd ask about your configuration - # of CPUs, disk speeds, proc types, rule set, suricata.yaml?</div><div><br></div>
<div>
Cheers,</div><div><br></div><div>Dave</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><br>-- <br>Gene Albin<br><a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a><br>
<a href="mailto:gene_albin@bigfoot.com" target="_blank">gene_albin@bigfoot.com</a><br>
</div>
<br></div>_______________________________________________<br>
Oisf-users mailing list<br>
<a href="mailto:Oisf-users@openinfosecfoundation.org" target="_blank">Oisf-users@openinfosecfoundation.org</a><br>
<a href="http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
<br></blockquote></div><font color="#888888"><br><br clear="all"><br>-- <br>"Of course, someone who knows more about this will correct me if I'm<br>wrong, and someone who knows less will correct me if I'm right." <br>
David Palmer (<a href="mailto:palmer@tybalt.caltech.edu" target="_blank">palmer@tybalt.caltech.edu</a>)<br>
<br>
</font></blockquote></div><br><br clear="all"><br>-- <br>Gene Albin<br><a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a><br><a href="mailto:gene_albin@bigfoot.com" target="_blank">gene_albin@bigfoot.com</a><br>
</div>