[Oisf-users] Suricata causes massive packet loss

[Nelson, Cooper]

Hi again Peter,

I am absolutely not recommending shipping a beta release!  Rather, I'm suggesting you try it to see if it's an issue that has already been resolved as a sanity check.  

Have you tried using libpcap instead of AF_PACKET as the capture mechanism?  

-Coop

-----Original Message-----
From: peter.mueller at ipfire.org <peter.mueller at ipfire.org> 
Sent: Thursday, September 5, 2019 11:56 AM
To: Nelson, Cooper <cnelson at ucsd.edu>; Peter Manev <petermanev at gmail.com>
Cc: oisf-users at lists.openinfosecfoundation.org; IPFire: Development-List <development at lists.ipfire.org>; Stefan Schantl <stefan.schantl at ipfire.org>
Subject: Re: [Oisf-users] Suricata causes massive packet loss

Hello Nelson, hello Peter, hello *,

thank you for your replies.

Upgrading to Suricata 5.0-beta is a difficult task, as we cannot simply ship beta releases in our firewall distribution. Personally, I rather doubt this is an issue due to a kernel/library/... combination, as we use Suricata for quite a while now and are upgrading IPFire's distribution kernel on a regular basis.

Anyway, Stefan (see CC) is currently working on Rust for the distribution, so we hope to take advantage of some more features soon. But since our issue is regarding packet loss for at least DNS and TLS traffic, I rather doubt Rust will make a big difference here.

Changing from "workers" to "autofp" mode unfortunately did not solve the problem. It is good to know the latter is recommended for inline deployments, "workers" was about 0.5 % faster in our benchmarks.

In IPFire, Suricata is started by a custom init script (please refer to https://git.ipfire.org/?p=ipfire-2.x.git;a=blob;f=src/initscripts/system/suricata;h=5a567f2d7f4bfef90fabb11438bc5065e731f21c;hb=HEAD
for its content) and appears like this in the process list:
> [root at maverick ~]# ps aux | grep suricata
> suricata  4882 10.9  7.3 1419868 289192 ?      Ssl  20:38   1:37 /usr/bin/suricata -c /etc/suricata/suricata.yaml -D -q 0 -q 1 -q 2 -q 3

I am not sure what the number behind "tcp.pkt_on_wrong_thread" should read like normally. @Peter: Is it too low or too high?

We will ship an update for libhtp as soon as possible, thank you for catching this.

Thanks, and best regards,
Peter Müller