[Oisf-users] TCP connection timeouts with suricata 4.0.3

[Gareth Parks]

Hi,

I have a situation where there are occasional layer 4 connection timeouts on a centos 7 server running haproxy and suricata (4.0.3). Suricata is being run in IPS mode using nfqueue. The original complaint was that the backend server checks in haproxy were timing out and servers were being ejected from the load balancer as a result however I can reproduce the case using curl in a loop, in 100 tries it will usually fail at least once.

I have observed the situation using tcpdump on both the server running suricata and the server it is trying to reach and neither registers the packet which suggests that the packet is put on the nfqueue for suricata to consume, it does so and never outputs it. Given that it is running in IPS mode I thought the packets were being dropped but on enabling the drop log no events were logged to it.

A few things that have been noted throughout the troubleshooting process is that restarting suricata will cause the problem to immediately stop occurring for a period of time; after removing the server from being exposed to production traffic and without restarting suricata the frequency of the timeouts will go down over time and if the server is run without suricata then the problem does not occur.

I'm stumped as to what to look at next to continue troubleshooting so any suggestions would be appreciated.

Thanks,
Gareth