[Oisf-users] Hardware specs for monitoring 100GB

[Nelson, Cooper]

Indeed, we are running into associated IO and licensing bottlenecks with the torrent of metadata that is produced.  I had to write an asynchronous spooler to copy stored files from a tmpfs partition to long-term storage, for example.  Our JSON logging is to a tmpfs partition as well.  

-Coop

-----Original Message-----
From: Peter Manev <petermanev at gmail.com> 
Sent: Tuesday, November 5, 2019 12:15 AM
To: Nelson, Cooper <cnelson at ucsd.edu>
Cc: Michał Purzyński <michalpurzynski1 at gmail.com>; Drew Dixon <dwdixon at umich.edu>; Daniel Wallmeyer <Daniel.Wallmeyer at cisecurity.org>; oisf-users at lists.openinfosecfoundation.org
Subject: Re: [Oisf-users] Hardware specs for monitoring 100GB


We have recently experimented with AFPv2 IPS set up and Trex and were able to achieve 40Gbps throughput (Intel based CPU/NIC), (doc reminder for me) It is not always trivial esp at 100Gbps as it becomes a major single point of failure as well so there are a lot of caveats to consider and test(HA/Fail over/log writing/shipping etc..)


--
Regards,
Peter Manev