[Oisf-devel] Live rule swap testing

[Martin Holste]

Ok, giving this a shot.  Also, I should note that since build
e3764b90c3bdfffaf8d98ec5bd71543a37b3f407 (or around there in early
June) Suricata crashes regularly.  I'll start running under gdb to
hopefully get you the backtrace (can't do core dumps, they're too
large).

On Tue, Jul 3, 2012 at 6:33 AM, Victor Julien <victor at inliniac.net> wrote:
> We've fixed the biggest leaks in the current git master. Some memory
> still gets lost it seems, probably due to fragmentation. Dealing with
> this will be a 1.4 effort.
>
> With 15k rules I now get:
>
> 17691 root      20   0 1973m 1.3g 3532 S 35.9 17.0   0:44.32 lt-suricata
> 17691 root      20   0 2945m 2.2g 3756 S 41.9 28.6   1:51.01 lt-suricata
> 17691 root      20   0 3335m 2.6g 3756 S 42.5 33.8   2:55.43 lt-suricata
> 17691 root      20   0 3343m 2.6g 3756 S 41.5 33.8   4:08.54 lt-suricata
> 17691 root      20   0 3351m 2.6g 3756 S 38.2 33.9   5:12.40 lt-suricata
>
> First entry is cold start, every one after is another reload. It's clear
> that mem usage stabilizes at some point.
>
> Interestingly, using tcmalloc gives a lot better results for the exact
> same ruleset:
>
> 17865 root      20   0 1418m 1.2g 3964 S 37.2 16.0   0:36.16 lt-suricata
> 17865 root      20   0 2194m 2.0g 4016 S 36.9 25.6   1:30.87 lt-suricata
> 17865 root      20   0 2509m 2.0g 4088 S 37.5 26.0   2:28.65 lt-suricata
> 17865 root      20   0 2517m 2.0g 4088 S 44.2 26.1   3:24.68 lt-suricata
>
> I've added a tcmalloc page to the performance section of the wiki:
> https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Tcmalloc
>
> On 06/30/2012 04:15 PM, Martin Holste wrote:
>> Great, thanks.
>>
>> On Sat, Jun 30, 2012 at 4:16 AM, Victor Julien <victor at inliniac.net> wrote:
>>> On 06/29/2012 09:44 PM, Martin Holste wrote:
>>>> Suricata's not giving back the RAM after doing the live swap, which is
>>>> a big (99G in my case) problem:
>>>>
>>>>  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>>>>  8513 root      20   0 99.7g  96g  67m S  445 68.3  36:01.06 suricata
>>>>
>>>> It also appears to do this for each live swap, so by the third swap, I'm OOM.
>>>>
>>>> This is on the latest from this morning, commit:
>>>> 4cf6bb3f4cbdab8a0cd57964be801cf676d2ec26.
>>>>
>>>> Suggestions?
>>>
>>> Seeing the same. We're tracking the issue here:
>>> https://redmine.openinfosecfoundation.org/issues/492
>>>
>>> --
>>> ---------------------------------------------
>>> Victor Julien
>>> http://www.inliniac.net/
>>> PGP: http://www.inliniac.net/victorjulien.asc
>>> ---------------------------------------------
>>>
>>>
>>>
>>> _______________________________________________
>>> Oisf-devel mailing list
>>> Oisf-devel at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>
>>
>>
>
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
>
>