[Oisf-users] Performance on multiple CPUs

Dave Remien dave.remien at gmail.com
Fri Aug 26 17:44:07 UTC 2011

On Fri, Aug 26, 2011 at 1:54 AM, Victor Julien <victor at inliniac.net> wrote:

> On 08/25/2011 08:18 PM, Gene Albin wrote:
> > Victor,
> >   Thanks for the explanation.  So do I understand correctly that for
> reading
> > pcap files I'm limited to only a single packet reader thread?  So at some
> > point my performance bottleneck would become the packet reading thread,
> > regardless of where the pcap file is stored?  If that's the case I could
> > probably use tcpreplay to resend the traffic through the interface and
> > pfring (thought I haven't begun to work with pfring yet).
> Correct. I don't think there is a sane way to have more pcap file
> readers. Replaying seems to be your best option.
I'm going to crawl out on a limb here.... Feel free to join me 8-).

I'll venture that reading a pcap is, for all practical purposes, limited by
the I/O to the pcap file. If you're reading the pcap from memory (either
from the cache or a RAM disk), you can probably read it at 1+gigabytes/sec,
assuming a modern machine with modern memory.

FWIW, the following is from a fairly elderly 8 cpu 3GHz Core2 machine, with
a 12 disk RAID6 array on a 4 port, 3Gbit/sec SAS backplane and modern LSI
PCiE controller:

space2:/y/pcaps # time dd if=tcpdump-2007-02-16-08-02 of=/dev/null bs=256k
2861036+1 records in
2861036+1 records out
750003609600 bytes (750 GB) copied, 994.034 s, 755 MB/s

real    16m34.051s
user    0m0.720s
sys     8m12.135s
space2:/y/pcaps # time tcpdump -nnr tcpdump-2007-02-16-08-02 -s0 -w
reading from file tcpdump-2007-02-16-08-02, link-type EN10MB (Ethernet)
tcpdump: pcap_loop: truncated dump file; tried to read 106 captured bytes,
only got 93

real    19m34.546s
user    5m46.591s
sys     9m31.326s
space2:/y/pcaps # ll tcpdump-2007-02-16-08-02
-rw-r--r-- 1 root root 750003609600 Mar 26  2007 tcpdump-2007-02-16-08-02

 So, this system can read a pcap from disk and write it (pcap formatted) to
/dev/null (tcpdump is single threaded) at
638MBytes/sec. Which is about 5Gbits/sec.

Chances are pretty good that a RAM drive would do better, but I don't have
enough memory to do that with this pcap (yet).

I haven't seen either snort or suricata process this pcap at anything
approaching this rate on this machine yet (using an actual fairly modern
rule set, I mean).

If Gene is still reading from a RAM drive, I wouldn't think that he's
"limited to only a single packet reader thread", pcap-read-performance-wise,
unless his pcap library was compiled without optimization.

(And welcome back from vacation, Victor!)


"Of course, someone who knows more about this will correct me if I'm
wron g, and someone who knows less will correct me if I'm right."
David Palmer (palmer at tybalt.caltech.edu)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20110826/b303b745/attachment-0002.html>

More information about the Oisf-users mailing list