[Oisf-users] Suricata load/latency spikes

[Oliver Humpage]

Hi all,

Well I built using --enable-profiling and --enable-debug, and used almost exactly the same config as on the live router.

I had a 76MB pcap (100k packets) which had taken around 90 seconds to collect on the router. In pcap mode, suricata processed the entire thing in under a second.

The only difference is that the "workers" runmode isn't available in pcap mode, so I had to use autofp. I don't use autofp in production because when used with ipfw divert, it only has a throughput of around 130Kb. I'm not sure why.

Do you think the results above suggest that there's an issue with autofp vs workers? Or perhaps how suricata is getting the packets from ipfw? (Relevant output from pcap processing below).

Thanks again for all your help, and apologies I don't understand as much as I should about the inner workings of the suricata engine.

Oliver.


29/6/2015 -- 17:56:48 - <Info> - time elapsed 0.849s
29/6/2015 -- 17:56:48 - <Notice> - Pcap-file module read 100000 packets, 78536202 bytes
29/6/2015 -- 17:56:48 - <Info> - AutoFP - Total flow handler queues - 6
29/6/2015 -- 17:56:48 - <Info> - AutoFP - Queue 0  - pkts: 16570        flows: 125         
29/6/2015 -- 17:56:48 - <Info> - AutoFP - Queue 1  - pkts: 20936        flows: 137         
29/6/2015 -- 17:56:48 - <Info> - AutoFP - Queue 2  - pkts: 19015        flows: 243         
29/6/2015 -- 17:56:48 - <Info> - AutoFP - Queue 3  - pkts: 16499        flows: 85          
29/6/2015 -- 17:56:48 - <Info> - AutoFP - Queue 4  - pkts: 13556        flows: 193         
29/6/2015 -- 17:56:48 - <Info> - AutoFP - Queue 5  - pkts: 13424        flows: 227         
29/6/2015 -- 17:56:48 - <Info> - Stream TCP processed 8831 TCP packets
29/6/2015 -- 17:56:48 - <Info> - TLS logger logged 8 requests
29/6/2015 -- 17:56:48 - <Info> - HTTP logger logged 55 requests
29/6/2015 -- 17:56:48 - <Info> - DNS logger logged 0 transactions
29/6/2015 -- 17:56:48 - <Info> - Stream TCP processed 13670 TCP packets
29/6/2015 -- 17:56:48 - <Info> - TLS logger logged 0 requests
29/6/2015 -- 17:56:48 - <Info> - HTTP logger logged 25 requests
29/6/2015 -- 17:56:48 - <Info> - DNS logger logged 0 transactions
29/6/2015 -- 17:56:48 - <Info> - Stream TCP processed 11105 TCP packets
29/6/2015 -- 17:56:48 - <Info> - TLS logger logged 19 requests
29/6/2015 -- 17:56:48 - <Info> - HTTP logger logged 48 requests
29/6/2015 -- 17:56:48 - <Info> - DNS logger logged 0 transactions
29/6/2015 -- 17:56:48 - <Info> - Stream TCP processed 9217 TCP packets
29/6/2015 -- 17:56:48 - <Info> - TLS logger logged 0 requests
29/6/2015 -- 17:56:48 - <Info> - HTTP logger logged 35 requests
29/6/2015 -- 17:56:48 - <Info> - DNS logger logged 0 transactions
29/6/2015 -- 17:56:48 - <Info> - Stream TCP processed 6097 TCP packets
29/6/2015 -- 17:56:48 - <Info> - TLS logger logged 12 requests
29/6/2015 -- 17:56:48 - <Info> - HTTP logger logged 32 requests
29/6/2015 -- 17:56:48 - <Info> - DNS logger logged 0 transactions
29/6/2015 -- 17:56:48 - <Info> - Stream TCP processed 5184 TCP packets
29/6/2015 -- 17:56:48 - <Info> - TLS logger logged 4 requests
29/6/2015 -- 17:56:48 - <Info> - HTTP logger logged 34 requests
29/6/2015 -- 17:56:48 - <Info> - DNS logger logged 2 transactions
29/6/2015 -- 17:56:48 - <Info> - host memory usage: 1216000 bytes, maximum: 33554432