[Oisf-users] Suricata performance in parallel instances!!!

[Matthew Jonkman]

I'm doing some similar work and I'm also finding that the disk is definitely my bottleneck. On a 4core box I'm seeing cpu utilization around 25% per core when feeding, a pretty clear indication my disks are'nt able to over-feed suricata.

Can you sample your cpu load for that 3.5 seconds and see where it is?

Matt



On Jun 2, 2011, at 2:09 AM, Abhishek Sharma wrote:

> Hi Team,
> 
> Firstly, I am mighty pleased and impressed with this tool!!! way better than snort!!
> 
> What I am trying to achieve here is to parse pcap files at the rate of 500 MB Pcaps / Second. I have pcaps of the size of 1 GB available with me. I have close to 50 rules only. All TCP. Now, if I parse one file with Suricata it takes me approximately 3.5 seconds to do so. I am using a 24 core server with 47 GB RAM. I am running Ubuntu 10 platform. I believe the machine is strong enough.
> 
> Now 3.5 secs for 1 GB file is good...no denying. But I have to achieve a speed of 500 Mbps and for that I have to parse a file in under 2 seconds. So what I did was to run two instances of Suricata in parallel (assuming two instances should finish in 3.5 seconds as its a fairly strong machine), but to my surprise (and dismay), it took me 7 seconds to process!!! for 3 instnaces it takes close to 9 secs!! So basically running a instance in parallel just adds up the time. I dont understand this. I have disabled all logging...Tried all search algorithms...played with the multithreading concept but its not helping either....
> 
> Please help this is my only hope...any suggestions are most appreciated...
> 
> Cheers!
> Abhi 
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users


----------------------------------------------------
Matthew Jonkman
Emergingthreats.net
Emerging Threats Pro
Open Information Security Foundation (OISF)
Phone 765-807-8630 x110
Fax 312-264-0205
http://www.emergingthreatspro.com
http://www.openinfosecfoundation.org
----------------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc