[Oisf-users] Suricata running out of memory

[Peter Manev]

On Wed, Sep 4, 2013 at 2:52 AM, Theodore Elhourani
<theodore.elhourani at gmail.com> wrote:
> I have a Suricata instance running on a machine with 2 CPUs, and 4GB of RAM.
> Suricata is in workers mode and is using af-packet with 2 threads for
> receive/detect.
>
>
> I am generating http traffic using httperf. TCP connections are established.
> In each connection, a total of 6 bursts, with 5 http requests in each, are
> made every 1 second. The connection is then torn down.

What is the duration of the tests?
Are the connections torn down properly?

>The resulting bit
> rate is 226Mbps (30k packets/sec). The total number of TCP connections in
> the test is 80k.
>
> With the "old values" below, suricata keeps on using more memory until all
> 4GB are occupied. This is even though my connections are completing
> correctly, with zero resets or session timeouts. In contrast, with the "new
> values", it uses a maximum of 1500MB of memory. CPU utilization is always
> below 90% in both cases. The tcp.reassembly_gap in both cases is around 2500
> for the 80k connections, and there are no packet drops. Note that the stats
> file reports roughly 39k tcp.sessions per thread. I am attaching the
> build-info and stats for both old and new configs.
>
> --------------------------------------------------------------------------------------------
> The following is the only change made to the configuration:
> tcp:
>     new: 30
>     established: 30 # old value: 3600, new value:30

The value of 3600 means that Suricata will wait 1 hr before it drops
an established connection if it does not see a proper teardown of the
tcp session.
If there is no proper teardown  the memory consumption would
accumulate overtime (until the timeout value is reached for that
specific connection) unless more aggressive tcp timeout values are
configured.

>     closed: 30 # old value: 120, new value: 30
>     emergency-new: 10
>     emergency-established: 30 # old value: 300, new value: 30
>     emergency-closed: 20
> ----------------------------------------------------------------------------------------------
>
> It appears suricata is not releasing memory for closed connections when the
> "old values" are used.
>