[Oisf-users] Suricata running out of memory

[Theodore Elhourani]

I have a Suricata instance running on a machine with 2 CPUs, and 4GB of RAM. Suricata is in workers mode and is using af-packet with 2 threads for receive/detect.


I am generating http traffic using httperf. TCP connections are established. In each connection, a total of 6 bursts, with 5 http requests in each, are made every 1 second. The connection is then torn down. The resulting bit rate is 226Mbps (30k packets/sec). The total number of TCP connections in the test is 80k.

With the "old values" below, suricata keeps on using more memory until all 4GB are occupied. This is even though my connections are completing correctly, with zero resets or session timeouts. In contrast, with the "new values", it uses a maximum of 1500MB of memory. CPU utilization is always below 90% in both cases. The tcp.reassembly_gap in both cases is around 2500 for the 80k connections, and there are no packet drops. Note that the stats file reports roughly 39k tcp.sessions per thread. I am attaching the build-info and stats for both old and new configs.

--------------------------------------------------------------------------------------------
The following is the only change made to the configuration:
tcp:
    new: 30
    established: 30 # old value: 3600, new value:30
    closed: 30 # old value: 120, new value: 30
    emergency-new: 10
    emergency-established: 30 # old value: 300, new value: 30
    emergency-closed: 20
----------------------------------------------------------------------------------------------

It appears suricata is not releasing memory for closed connections when the "old values" are used.




 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130903/5d58619c/attachment-0004.html>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130903/5d58619c/attachment-0005.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build-info.rtf
Type: text/rtf
Size: 2485 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130903/5d58619c/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: new-values-stats.rtf
Type: text/rtf
Size: 6536 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130903/5d58619c/attachment-0004.bin>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130903/5d58619c/attachment-0006.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: old-values-stats.rtf
Type: text/rtf
Size: 6461 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130903/5d58619c/attachment-0005.bin>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130903/5d58619c/attachment-0007.html>