[Oisf-users] Oisf-users Digest, Vol 19, Issue 1

[Dave Remien]

Folks,

Where Abhishek has lotsa RAM, I'd make a ramdisk (assuming that you've got a
64 bit kernel - if not, you should switch to a 64 bit distro) and copy the
pcap there; that'll take the disk out of the equation. Or you could just cp
the pcap to /dev/null just before running Suricata against it; that'll load
it into RAM and it should stay cached, assuming that you have the RAM free
to cache it. Top, htop and atop are your friends 8-).

Dave



> I'm doing some similar work and I'm also finding that the disk is
> definitely my bottleneck. On a 4core box I'm seeing cpu utilization around
> 25% per core when feeding, a pretty clear indication my disks are'nt able to
> over-feed suricata.
>
> Can you sample your cpu load for that 3.5 seconds and see where it is?
>
> Matt
>
>
>
> On Jun 2, 2011, at 2:09 AM, Abhishek Sharma wrote:
>
> > Hi Team,
> >
> > Firstly, I am mighty pleased and impressed with this tool!!! way better
> than snort!!
> >
> > What I am trying to achieve here is to parse pcap files at the rate of
> 500 MB Pcaps / Second. I have pcaps of the size of 1 GB available with me. I
> have close to 50 rules only. All TCP. Now, if I parse one file with Suricata
> it takes me approximately 3.5 seconds to do so. I am using a 24 core server
> with 47 GB RAM. I am running Ubuntu 10 platform. I believe the machine is
> strong enough.
> >
> > Now 3.5 secs for 1 GB file is good...no denying. But I have to achieve a
> speed of 500 Mbps and for that I have to parse a file in under 2 seconds. So
> what I did was to run two instances of Suricata in parallel (assuming two
> instances should finish in 3.5 seconds as its a fairly strong machine), but
> to my surprise (and dismay), it took me 7 seconds to process!!! for 3
> instnaces it takes close to 9 secs!! So basically running a instance in
> parallel just adds up the time. I dont understand this. I have disabled all
> logging...Tried all search algorithms...played with the multithreading
> concept but its not helping either....
> >
> > Please help this is my only hope...any suggestions are most
> appreciated...
> >
> > Cheers!
> > Abhi
> > _______________________________________________
> > Oisf-users mailing list
> > Oisf-users at openinfosecfoundation.org
> > http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
>
> ----------------------------------------------------
> Matthew Jonkman
> Emergingthreats.net
> Emerging Threats Pro
> Open Information Security Foundation (OISF)
> Phone 765-807-8630 x110
> Fax 312-264-0205
> http://www.emergingthreatspro.com
> http://www.openinfosecfoundation.org
> ----------------------------------------------------
>
> PGP: http://www.jonkmans.com/mattjonkman.asc
>
>
>
>
>
> ------------------------------
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
>
> End of Oisf-users Digest, Vol 19, Issue 1
> *****************************************
>



-- 
"Of course, someone who knows more about this will correct me if I'm
wrong, and someone who knows less will correct me if I'm right."
David Palmer (palmer at tybalt.caltech.edu)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20110603/aa01387f/attachment-0002.html>