[Oisf-users] Massive kernel drops with HTTP traffic

[Konstantin Klinger]

On 18.08.2018 15:57, Peter Manev wrote:
> 
> 
>> On 17 Aug 2018, at 07:35, Michael Stone <mstone at mathom.us> wrote:
>>
>> On Fri, Aug 17, 2018 at 03:24:31PM +0200, you wrote:
>>>> Do you have filemagic enabled?
>>>
>>> Yes. We currently use filestore v1. And we use the filemagic value in
>>> our rules for filestoring.
>>
>> Unless you have customized the magic file it is very likely that you won't hit your performance target this way. I'd suggest rules specific to what you're trying to save rather than relying on libmagic (which is very inefficient).
>>
> 
> 
> That could be easy to test and confirm if it is contributing or creating the mess- Konstantin is it possible to try it out and see ?
> 
> 

We made some test runs without filestore enabled and after that only
without libmagic/filemagic (but filestore on) and that helped to
decrease the number of packet drops (~30% -> ~5% and ~50% -> ~10%).
Thank you. Our workaround will be not using filemagic rules anymore.

@Mike: Do you have further experience in a workaround to not use libmagic?
@all: Is someone using libmagic/filemagic on high traffic sensors
(>5Gb/sec) and has no performance issues? Is someone already using
filestore v2 (we are still using v1) and has any experience with it's
performance?

-- 
Konstantin Klinger
Security Content Engineer
Threat Detection & Hunting (TDH)

+49 160 95476260
konstantin.klinger at dcso.de

dcso.de
blog.dcso.de

PGP: 180D C5B3 3C68 5C9A FB58 6F33 400E 5A35 3307 8D46
 
DCSO Deutsche Cyber-Sicherheitsorganisation GmbH • EUREF-Campus
22 • D-10829 Berlin
Geschäftsführer: Dr.-Ing. Gunnar Siebert, Sitz der Gesellschaft: Berlin,
Amtsgericht Charlottenburg HRB 172382