[Oisf-users] 40GB inspection and I/O write speed concerns

[Jeremy A. Grove]

Thanks to both Peter and Cooper! 

I do split out by event type already and I can certainly trim out some of this if needed. Ill let you know how it goes. 

Regards, 

Jeremy Grove, SSCP 
Security Engineer 
Quadrant Information Security 
o: [ callto:(904)296-9100 | (904)296-9100 ] x100 
t: [ callto:(800) 538-9357 | (800) 538-9357 ] x100 
e: [ mailto:soc at quadrantsec.com | soc at quadrantsec.com ] 

Learn more= about our managed SIEM [ https://a.quadrantsec.com/3D%22https://quadrantsec.com/SaganMSSP%22 | people + product ] 




From: "Peter Manev" <petermanev at gmail.com> 
To: "Jeremy A. Grove" <jgrove at quadrantsec.com> 
Cc: "oisf-users" <oisf-users at lists.openinfosecfoundation.org> 
Sent: Monday, July 22, 2019 11:13:46 AM 
Subject: Re: [Oisf-users] 40GB inspection and I/O write speed concerns 


On 22 Jul 2019, at 08:34, Jeremy A. Grove < [ mailto:jgrove at quadrantsec.com | jgrove at quadrantsec.com ] > wrote: 




Hi All, 

I am looking for advice. We are working on setting up a machine for potential 40GB a second on inspection. Our concern comes in the write speed of the I/O to disk as the meta-data that Suricata creates is important to us. Does anyone have experience with this? I have listed some of our set up below. Are there any suggestions or known issues that I should be aware of. 




My five cents to start with: 
I would split the logging per event_type and only log what is needed not just everything as some logs depending on the traffic could be very very verbose- like dns/fileinfo on university network etc.. 
Also filter out if possible with a bpf(or on the mirror /tap) what is not relevant too. 




BQ_BEGIN

DL360 Gen 10 
P408I-A (Raid Card) 
4 x 2TB SSD in RAID 10 (Part number 877788-B21) 
Mixed use SSDs 


Thanks! 

Jeremy Grove, SSCP 
Security Engineer 
Quadrant Information Security 

BQ_END

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190722/05fbdbee/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2131 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190722/05fbdbee/attachment.bin>