[Oisf-users] Massive kernel drops with HTTP traffic

[Peter Manev]

> On 16 Aug 2018, at 08:00, Konstantin Klinger <konstantin.klinger at dcso.de> wrote:
> 
> 
> 
>> On 16.08.2018 15:02, Victor Julien wrote:
>>> On 16-08-18 14:49, Konstantin Klinger wrote:
>>> Hello OISF users,
>>> 
>>> we have some issues with massive capture.kernel_drops (~30-50%) on some
>>> of our high traffic (>5Gbit/s per interface) 4.1.0dev Suricata instances
>>> (af_packet). What we found curious about the issue is that there is no
>>> associated heavy CPU load.
>>> 
>>> We were able to determine that the problem is related by large volumes
>>> of HTTP traffic on the interface (such as, for example, huge backups,
>>> huge file downloads, etc.). Without HTTP traffic (for example after
>>> filtering port 80/8080 via bpf before inspection) the packets drops
>>> decreased below 5%. This is also the case after deactivating the HTTP
>>> parser in the suricata.yaml config.
>>> 
>>> So our question is if anyone has or had the same issue? Any experience
>>> to share?
>>> 
>>> We will do further debugging on this issue and we will try to make the
>>> problem reproducible by tcpreplaying a captured pcap, but we are not at
>>> this point yet.
>> 
>> Which git rev? I'm just analyzing a recently added perf regression.
>> Added in 7e004f52c60c5e4d7cd8f5ed09491291d18f42d2
> 
> We have the same problems with 4.0.* stable.
> 

When you enable http/stream events during a run do they show something abnormal ?


>> 
> 
> -- 
> Konstantin Klinger
> Security Content Engineer
> Threat Detection & Hunting (TDH)
> 
> +49 160 95476260
> konstantin.klinger at dcso.de
> 
> dcso.de
> blog.dcso.de
> 
> PGP: 180D C5B3 3C68 5C9A FB58 6F33 400E 5A35 3307 8D46
>  
> DCSO Deutsche Cyber-Sicherheitsorganisation GmbH • EUREF-Campus
> 22 • D-10829 Berlin
> Geschäftsführer: Dr.-Ing. Gunnar Siebert, Sitz der Gesellschaft: Berlin,
> Amtsgericht Charlottenburg HRB 172382
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> 
> Conference: https://suricon.net
> Trainings: https://suricata-ids.org/training/