[Oisf-users] 40GB inspection and I/O write speed concerns

[Nelson, Cooper]

I unfortunately didn't record it; its around 500 eps currently.  It's probably 10-20X that with full logging and students on campus.  A 100Gbs deployment on our core router would probably be 100X that (we have 12 million+ routable IPv4 addresses!).

I don't recall it impacting drops at all.  What did impact drops was setting the file extraction to 'unlimited', which resulted in about 2X drops.  This was expected and not surprising, btw.  Keep in mind our system is also massively overprovisioned and something like 80% idle at peak, even with the 'ondemand' cpu governor.  

Again, I highly recommend using btrfs with lzop compression enabled, as it effectively gives you 'free' disk and I/O improvements. 

-Coop


-----Original Message-----
From: Peter Manev <petermanev at gmail.com> 
Sent: Friday, July 26, 2019 5:51 AM
To: Nelson, Cooper <cnelson at ucsd.edu>
Cc: Jeremy A. Grove <jgrove at quadrantsec.com>; oisf-users <oisf-users at lists.openinfosecfoundation.org>
Subject: Re: 40GB inspection and I/O write speed concerns

On Mon, Jul 22, 2019 at 7:20 PM Nelson, Cooper <cnelson at ucsd.edu> wrote:
>
> Btw I’ve done 20Gbs w/full EVE logging to a SAS 10K RAID5 container, no problem.  Filesystem was btrfs with lzop compression enabled.
>

Whats your eps in that case? (out of curiosity :) ) and did it reflect on the drops (full vs non full logging ) ?