[Oisf-devel] Live rule swap testing

[Victor Julien]

We've fixed the biggest leaks in the current git master. Some memory
still gets lost it seems, probably due to fragmentation. Dealing with
this will be a 1.4 effort.

With 15k rules I now get:

17691 root      20   0 1973m 1.3g 3532 S 35.9 17.0   0:44.32 lt-suricata
17691 root      20   0 2945m 2.2g 3756 S 41.9 28.6   1:51.01 lt-suricata
17691 root      20   0 3335m 2.6g 3756 S 42.5 33.8   2:55.43 lt-suricata
17691 root      20   0 3343m 2.6g 3756 S 41.5 33.8   4:08.54 lt-suricata
17691 root      20   0 3351m 2.6g 3756 S 38.2 33.9   5:12.40 lt-suricata

First entry is cold start, every one after is another reload. It's clear
that mem usage stabilizes at some point.

Interestingly, using tcmalloc gives a lot better results for the exact
same ruleset:

17865 root      20   0 1418m 1.2g 3964 S 37.2 16.0   0:36.16 lt-suricata
17865 root      20   0 2194m 2.0g 4016 S 36.9 25.6   1:30.87 lt-suricata
17865 root      20   0 2509m 2.0g 4088 S 37.5 26.0   2:28.65 lt-suricata
17865 root      20   0 2517m 2.0g 4088 S 44.2 26.1   3:24.68 lt-suricata

I've added a tcmalloc page to the performance section of the wiki:
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Tcmalloc

On 06/30/2012 04:15 PM, Martin Holste wrote:
> Great, thanks.
> 
> On Sat, Jun 30, 2012 at 4:16 AM, Victor Julien <victor at inliniac.net> wrote:
>> On 06/29/2012 09:44 PM, Martin Holste wrote:
>>> Suricata's not giving back the RAM after doing the live swap, which is
>>> a big (99G in my case) problem:
>>>
>>>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>>>  8513 root      20   0 99.7g  96g  67m S  445 68.3  36:01.06 suricata
>>>
>>> It also appears to do this for each live swap, so by the third swap, I'm OOM.
>>>
>>> This is on the latest from this morning, commit:
>>> 4cf6bb3f4cbdab8a0cd57964be801cf676d2ec26.
>>>
>>> Suggestions?
>>
>> Seeing the same. We're tracking the issue here:
>> https://redmine.openinfosecfoundation.org/issues/492
>>
>> --
>> ---------------------------------------------
>> Victor Julien
>> http://www.inliniac.net/
>> PGP: http://www.inliniac.net/victorjulien.asc
>> ---------------------------------------------
>>
>>
>>
>> _______________________________________________
>> Oisf-devel mailing list
>> Oisf-devel at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
> 
> 
> 


-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------