[Oisf-users] FW: Performance of pcap-log output

[Jake Gionet]

Suricata isn't actually reporting that its dropping the packets, but
I'll admit that I wouldn't know where to look if it isn't as apparent
as reading the stats.log values that include "drop" in the name.  When
I go back and read the pcaps that have been written, tcpdump reports
large gaps in the data.  I'm currently writing 1000 MB files that when
re-read report ~1500 MB of missing data.  If I set up a ring buffer
with tcpdump it appears to capture more data (the files roll almost
twice as fast) and it does not report any missing data while
re-reading the pcaps.


On Tue, Oct 23, 2012 at 2:39 PM, Martin Holste <mcholste at gmail.com> wrote:
> Jake,
>
> That seems very low indeed, though I don't use Suricata for collecting pcap.
>
> At anything less than a Gb/sec, the hard drive speeds will almost
> certainly not be a factor.  HD's write and read at rates not seen on
> normal networks during normal operation.  Modern SATA will
> theoretically write at 6 GB/sec, which is 48 Gb/sec.  Until you're
> pushing more than 1 Gb/sec, your bottleneck will be the pcap interface
> at all times, even if you have very slow disk.
>
> Do you get drops when you have disk writing turned off?
>


I'm seeing peaks of ~500 Mb/s but for the most part is stays under 120
Mb/s with an average of 85 Mb/s.  I'm using version 1.4beta2 of
Suricata.
I had actually planned on using that article as a guideline for this
device once I had packet capturing covered.  Unfortunately my NIC
isn't hashing the flows properly for RSS and it cannot be manually
configured to do it, otherwise I would have set it up almost
identically to that system.


On Tue, Oct 23, 2012 at 2:27 PM, Peter Manev <petermanev at gmail.com> wrote:
> Hi,
>
> The speeds that you are achieving are very low, almost impossible :).
> Please have a look here (although it uses advanced techniques for your
> network card drivers and such, it will be helpful to set up your
> suricata.yaml):
> https://home.regit.org/2012/07/suricata-to-10gbps-and-beyond/
>
> What speeds are you looking at .. on your network interface?
> Which version of Suricata are you using?
>
> I am not sure about pcap-log throughput, but i am assuming it will be hugely
> dependent on your HDD speed as well.
>
> thanks
>
>
> Regards,
> Peter Manev
>