[Oisf-users] Suricata causes massive packet loss
peter.mueller at ipfire.org
peter.mueller at ipfire.org
Tue Sep 3 13:41:00 UTC 2019
Dear OSIF/Suricata users,
earlier this year, the linux-based open source firewall distribution IPFire
has migrated from Snort to Suricata for a number of reasons (further information
is available at https://blog.ipfire.org/post/introducing-ipfire-s-new-intrusion-prevention-system).
While we are quite pleased with some of its features (multi-threading, ability
to monitor several interfaces per process, etc.), we experienced some problems
ever since we are running it. Not being reproducible everywhere, we initially
thought they were corner cases in obscure network scenarios.
Ultimately, they were not. Even worse, no dropped packets were logged although
we can tell for sure there were some.
For example, several IPFire users - including myself - report very slow DNS
resolution when trying to access a website, while "normal" lookups using
dig or host commands perform fine.
Another issue is reduced OpenVPN tunnel throughput, which seems to be caused
by massive packet loss when Suricata is enabled (~ 800 kB/s, ~ 2 MB/s if
Suricata is turned off). In order to get closer to its origin, we spend a lot
of time on testing and debugging, eventually left without any idea what the
solution might be.
Both issues - possibly being related to each other - can be reproduced using
Suricata 4.1.4 without any rules or packet decoders enabled. Unfortunately, our
setup, where Suricata runs inline via Netfilter queue, does not seem to be
documented very well.
That's why I am asking here if anybody is able to tell us what we are doing
wrong. Perhaps this just might be a configuration problem, but we are out
of ideas where to look for it.
Please find our suricata.yaml (decoders enabled, but disabling it does not matter)
and the stats.log file enclosed.
Details regarding a testing machine:
> [root at maverick ~]# suricata -V
> This is Suricata version 4.1.4 RELEASE
> [root at maverick ~]# uname -a
> Linux maverick 4.14.138-ipfire #1 SMP Sat Aug 10 00:53:30 GMT 2019 x86_64 Intel(R) Celeron(R) CPU N3150 @ 1.60GHz GenuineIntel GNU/Linux
> [root at maverick ~]# ldd /usr/bin/suricata
> linux-vdso.so.1 (0x00007ffdd77df000)
> libdl.so.2 => /lib/libdl.so.2 (0x00007ba2d3e7f000)
> librt.so.1 => /lib/librt.so.1 (0x00007ba2d3e75000)
> libm.so.6 => /lib/libm.so.6 (0x00007ba2d3d26000)
> libmagic.so.1 => /usr/lib/libmagic.so.1 (0x00007ba2d3afc000)
> libcap-ng.so.0 => /usr/lib/libcap-ng.so.0 (0x00007ba2d38f6000)
> libpcap.so.1 => /usr/lib/libpcap.so.1 (0x00007ba2d36b5000)
> libnet.so.1 => /usr/lib/libnet.so.1 (0x00007ba2d3498000)
> libnetfilter_queue.so.1 => /usr/lib/libnetfilter_queue.so.1 (0x00007ba2d3291000)
> libnfnetlink.so.0 => /usr/lib/libnfnetlink.so.0 (0x00007ba2d308a000)
> libjansson.so.4 => /usr/lib/libjansson.so.4 (0x00007ba2d307b000)
> libpthread.so.0 => /lib/libpthread.so.0 (0x00007ba2d305a000)
> libyaml-0.so.2 => /usr/lib/libyaml-0.so.2 (0x00007ba2d2e3c000)
> libpcre.so.1 => /usr/lib/libpcre.so.1 (0x00007ba2d2bc7000)
> liblzma.so.5 => /usr/lib/liblzma.so.5 (0x00007ba2d29a0000)
> libhs.so.5 => /usr/lib/libhs.so.5 (0x00007ba2d221a000)
> libhtp.so.2 => /usr/lib/libhtp.so.2 (0x00007ba2d1ff2000)
> libc.so.6 => /lib/libc.so.6 (0x00007ba2d1e0d000)
> /lib64/ld-linux-x86-64.so.2 (0x00007ba2d3e8f000)
> libmnl.so.0 => /usr/lib/libmnl.so.0 (0x00007ba2d1c07000)
> libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007ba2d1a6c000)
> libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007ba2d1855000)
> [root at maverick ~]# suricata --build-info
> This is Suricata version 4.1.4 RELEASE
> Features: NFQ PCAP_SET_BUFF AF_PACKET HAVE_PACKET_FANOUT LIBCAP_NG LIBNET1.1 HAVE_HTP_URI_NORMALIZE_HOOK PCRE_JIT HAVE_LIBJANSSON TLS MAGIC
> SIMD support: none
> Atomic intrisics: 1 2 4 8 byte(s)
> 64-bits, Little-endian architecture
> GCC version 8.3.0, C version 199901
> compiled with _FORTIFY_SOURCE=2
> L1 cache line size (CLS)=64
> thread local storage method: __thread
> compiled with LibHTP v0.5.28, linked against LibHTP v0.5.28
>
> Suricata Configuration:
> AF_PACKET support: yes
> eBPF support: no
> XDP support: no
> PF_RING support: no
> NFQueue support: yes
> NFLOG support: no
> IPFW support: no
> Netmap support: no
> DAG enabled: no
> Napatech enabled: no
> WinDivert enabled: no
>
> Unix socket enabled: yes
> Detection enabled: yes
>
> Libmagic support: yes
> libnss support: no
> libnspr support: no
> libjansson support: yes
> liblzma support: yes
> hiredis support: no
> hiredis async with libevent: no
> Prelude support: no
> PCRE jit: yes
> LUA support: no
> libluajit: no
> libgeoip: no
> Non-bundled htp: yes
> Old barnyard2 support: no
> Hyperscan support: yes
> Libnet support: yes
> liblz4 support: no
>
> Rust support: no
> Rust strict mode: no
> Rust debug mode: no
> Rust compiler: not set
> Rust cargo: not set
>
> Install suricatasc: no
> Install suricata-update: no
>
> Profiling enabled: no
> Profiling locks enabled: no
>
> Development settings:
> Coccinelle / spatch: no
> Unit tests enabled: no
> Debug output enabled: no
> Debug validation enabled: no
>
> Generic build parameters:
> Installation prefix: /usr
> Configuration directory: /etc/suricata/
> Log directory: /var/log/suricata/
>
> --prefix /usr
> --sysconfdir /etc
> --localstatedir /var
> --datarootdir /usr/share
>
> Host: x86_64-pc-linux-gnu
> Compiler: gcc (exec name) / gcc (real)
> GCC Protect enabled: yes
> GCC march native enabled: no
> GCC Profile enabled: no
> Position Independent Executable enabled: no
> CFLAGS -O2 -pipe -Wall -fexceptions -fPIC -m64 -mindirect-branch=thunk -mfunction-return=thunk -mtune=generic -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fstack-protector-strong
> PCAP_CFLAGS -I/usr/include
> SECCFLAGS -fstack-protector -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security
Please let me know if further information is needed. Any help is highly appreciated.
Thanks, and best regards,
Peter Müller
-------------- next part --------------
A non-text attachment was scrubbed...
Name: suricata.yaml
Type: application/x-yaml
Size: 26059 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190903/acd06f3e/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: stats.log
Type: text/x-log
Size: 18673 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190903/acd06f3e/attachment-0003.bin>
More information about the Oisf-users
mailing list