[Oisf-devel] Memory pb on Suricata git today
rmkml
rmkml at free.fr
Fri Jul 30 16:58:54 UTC 2010
Hi,
Congratulations for Suricata v1.0.1!
but this new release not fix my memory usage pb please.
v101 - mem usage: 621M
Im not continue my testing on your open source product because my linux kernel kill suricata process...
Regards
Rmkml
On Mon, 26 Jul 2010, rmkml wrote:
> It's ok, but with my commercial sig, suricata use 1.2G and killed by linux
> kernel (on my personnal laptop).
> Anyone test with vrt sigs please? (v2.8.5.3 or old)
> Regards
> Rmkml
>
>
> On Mon, 26 Jul 2010, rmkml wrote:
>
>> Hi Victor,
>> ok I have tested with theses suricata versions: (same conf, same pcap file
>> is 27Mo)
>> v100 - mem usage: 400M
>> git13jul- mem usage: 400M
>> git21jul- mem usage: 630M
>> git25jul- mem usage: 649M
>> All test with emerging all sigs daily
>> (http://www.emergingthreats.net/rules/emerging-all.rules.zip)
>> Anyone confirm increase 50% memory please?
>> Regards
>> Rmkml
>>
>>
>> On Mon, 26 Jul 2010, Victor Julien wrote:
>>
>>> I think the increased mem usage is caused by fixing some accuracy
>>> issues. As far as I can tell, it's not a bug of some kind.
>>>
>>> Cheers,
>>> Victor
>>>
>>> rmkml wrote:
>>>> Thx Anoop and Victor,
>>>> ok crash/segfault fixed,
>>>> but mem usage increase always exist on git
>>>> c25921edf01c9f2d2e3c639037528ef5440c566e.
>>>> Regards
>>>> Rmkml
>>>>
>>>>
>>>> On Sun, 25 Jul 2010, Victor Julien wrote:
>>>>
>>>>> Should be fixed in current master. Thanks guys!
>>>>>
>>>>> Anoop Saldanha wrote:
>>>>>> Attached a new patch. Please don't apply the older one. Fixed a small
>>>>>> typo in the unittest. It should pass now.
>>>>>>
>>>>>> On Sun, Jul 25, 2010 at 10:48 AM, Anoop Saldanha <poonaatsoc at gmail.com
>>>>>> <mailto:poonaatsoc at gmail.com>> wrote:
>>>>>>
>>>>>> Hi rmkml. Can you please check it with this attached patch.
>>>>>> Should
>>>>>> fix it. Added an unittest to the patch as well.
>>>>>>
>>>>>>
>>>>>> On Sun, Jul 25, 2010 at 1:21 AM, <rmkml at free.fr
>>>>>> <mailto:rmkml at free.fr>> wrote:
>>>>>>
>>>>>> Ok Im found my "crash" sig:
>>>>>> alert udp any any -> any any (msg:"crash"; byte_test:4,>,2,0;
>>>>>> byte_jump:1,0,relative; sid:11; )
>>>>>> Regards
>>>>>> Rmkml
>>>>>>
>>>>>>
>>>>>> Selon rmkml <rmkml at free.fr <mailto:rmkml at free.fr>>:
>>>>>>
>>>>>> > thx for reply Victor,
>>>>>> > no problemo:
>>>>>> >
>>>>>> > ...
>>>>>> > [20560] 24/7/2010 -- 16:23:13 - (detect.c:302) <Error>
>>>>>> (DetectLoadSigFile) --
>>>>>> > [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - Error parsing
>>>>>> signature "alert tcp
>>>>>> > $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"WEB-PHP
>>>>>> shoutbox.php
>>>>>> > access"; flow:to_server,established;
>>>>>> uricontent:"/shoutbox.php";
>>>>>> > reference:nessus,11668; classtype:web-application-activity;
>>>>>> sid:2142;
>>>>>> > rev:1;)" from file /home/test/snort/rules/web-php.rules at
>>>>>> line 94
>>>>>> > [20560] 24/7/2010 -- 16:23:14 - (detect-bytejump.c:594) <Error>
>>>>>> > (DetectBytejumpSetup) -- [ERRCODE:
>>>>>> SC_ERR_INVALID_SIGNATURE(39)] - No
>>>>>> > preceding content or uricontent or pcre option
>>>>>> > *** glibc detected ***
>>>>>> > /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul201
>>>>>> > 0/src/.libs/suricata: corrupted double-linked list:
>>>>>> 0x0a51dea8 ***
>>>>>> > ======= Backtrace: =========
>>>>>> > /lib/libc.so.6[0xa9d06d]
>>>>>> > ...
>>>>>> >
>>>>>> > Regards
>>>>>> > Rmkml
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Sat, 24 Jul 2010, Victor Julien wrote:
>>>>>> >
>>>>>> >> Can you share the signature this is happening with?
>>>>>> Privately if you
>>>>>> > prefer.
>>>>>> >>
>>>>>> >> Cheers,
>>>>>> >> Victor
>>>>>> >>
>>>>>> >> rmkml wrote:
>>>>>> >>> Hi Victor,
>>>>>> >>> Thx for your work and your time on this project!
>>>>>> >>>
>>>>>> >>> I have "downloaded" (git clone) last Suricata version,
>>>>>> >>> but I have a glibc error (git
>>>>>> ead29dc6918f4524f1fae7e892d3f86dac117c0a):
>>>>>> >>> ...
>>>>>> >>> [20560] 24/7/2010 -- 16:23:14 - (detect-bytejump.c:594)
>>>>>> <Error>
>>>>>> >>> (DetectBytejumpSetup) -- [ERRCODE:
>>>>>> SC_ERR_INVALID_SIGNATURE(39)] - No
>>>>>> >>> preceding content or uricontent or pcre option
>>>>>> >>> *** glibc detected ***
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata:
>>>>>>
>>>>>> >>> corrupted double-linked list: 0x0a51dea8 ***
>>>>>> >>> ======= Backtrace: =========
>>>>>> >>> /lib/libc.so.6[0xa9d06d]
>>>>>> >>> /lib/libc.so.6[0xa9eb2b]
>>>>>> >>> /lib/libc.so.6(cfree+0x90)[0xaa2430]
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata[0x807b0dd]
>>>>>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata[0x807c04a]
>>>>>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata[0x807c1fb]
>>>>>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata[0x806586e]
>>>>>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata[0x8065d4b]
>>>>>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata[0x804bc70]
>>>>>>
>>>>>> >>>
>>>>>> >>> /lib/libc.so.6(__libc_start_main+0xe0)[0xa4cf70]
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata[0x804aa01]
>>>>>>
>>>>>> >>>
>>>>>> >>> ======= Memory map: ========
>>>>>> >>> 0072c000-0073e000 r-xp 00000000 08:02 3700508
>>>>>> /lib/libz.so.1.2.3
>>>>>> >>> 0073e000-0073f000 rw-p 00011000 08:02 3700508
>>>>>> /lib/libz.so.1.2.3
>>>>>> >>> 00a18000-00a33000 r-xp 00000000 08:02 11817698
>>>>>> /lib/ld-2.6.so <http://ld-2.6.so>
>>>>>> >>> 00a33000-00a34000 r--p 0001a000 08:02 11817698
>>>>>> /lib/ld-2.6.so <http://ld-2.6.so>
>>>>>> >>> 00a34000-00a35000 rw-p 0001b000 08:02 11817698
>>>>>> /lib/ld-2.6.so <http://ld-2.6.so>
>>>>>> >>> 00a37000-00b85000 r-xp 00000000 08:02 11817699
>>>>>> /lib/libc-2.6.so <http://libc-2.6.so>
>>>>>> >>> 00b85000-00b87000 r--p 0014e000 08:02 11817699
>>>>>> /lib/libc-2.6.so <http://libc-2.6.so>
>>>>>> >>> 00b87000-00b88000 rw-p 00150000 08:02 11817699
>>>>>> /lib/libc-2.6.so <http://libc-2.6.so>
>>>>>> >>> 00b88000-00b8b000 rw-p 00000000 00:00 0
>>>>>> >>> 00bbf000-00bd3000 r-xp 00000000 08:02 5434178
>>>>>> /lib/libpthread-2.6.so <http://libpthread-2.6.so>
>>>>>> >>> 00bd3000-00bd4000 r--p 00013000 08:02 5434178
>>>>>> /lib/libpthread-2.6.so <http://libpthread-2.6.so>
>>>>>> >>> 00bd4000-00bd5000 rw-p 00014000 08:02 5434178
>>>>>> /lib/libpthread-2.6.so <http://libpthread-2.6.so>
>>>>>> >>> 00bd5000-00bd7000 rw-p 00000000 00:00 0
>>>>>> >>> 00bee000-00c17000 r-xp 00000000 08:02 2078837
>>>>>> /usr/lib/libpcap.so.0.9.7
>>>>>> >>> 00c17000-00c19000 rw-p 00028000 08:02 2078837
>>>>>> /usr/lib/libpcap.so.0.9.7
>>>>>> >>> 00c58000-00c7f000 r-xp 00000000 08:02 5434342
>>>>>> /lib/libpcre.so.0.0.1
>>>>>> >>> 00c7f000-00c80000 rw-p 00026000 08:02 5434342
>>>>>> /lib/libpcre.so.0.0.1
>>>>>> >>> 05db4000-05dbf000 r-xp 00000000 08:02 5434249
>>>>>> >>> /lib/libgcc_s-4.1.2-20070925.so.1
>>>>>> >>> 05dbf000-05dc0000 rw-p 0000a000 08:02 5434249
>>>>>> >>> /lib/libgcc_s-4.1.2-20070925.so.1
>>>>>> >>> 08048000-08100000 r-xp 00000000 08:02 1244073
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata
>>>>>>
>>>>>> >>>
>>>>>> >>> 08100000-08101000 rw-p 000b8000 08:02 1244073
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/src/.libs/suricata
>>>>>>
>>>>>> >>>
>>>>>> >>> 08101000-0a53d000 rw-p 00000000 00:00 0 [heap]
>>>>>> >>> b7400000-b7421000 rw-p 00000000 00:00 0
>>>>>> >>> b7421000-b7500000 ---p 00000000 00:00 0
>>>>>> >>> b7594000-b771c000 rw-p 00000000 00:00 0
>>>>>> >>> b771c000-b7737000 r-xp 00000000 08:02 11261710
>>>>>> >>>
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/yaml-0.1.3/src/.libs/libyaml-0.so.2.0.1
>>>>>> >>> b7737000-b7738000 rw-p 0001a000 08:02 11261710
>>>>>> >>>
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/yaml-0.1.3/src/.libs/libyaml-0.so.2.0.1
>>>>>> >>> b7748000-b7749000 rw-p 00000000 00:00 0
>>>>>> >>> b7749000-b7758000 r-xp 00000000 08:02 654980
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/libhtp/htp/.libs/libhtp-0.2.so.1.0.2
>>>>>>
>>>>>> >>>
>>>>>> >>> b7758000-b7759000 rw-p 0000e000 08:02 654980
>>>>>> >>>
>>>>>> >
>>>>>>
>>>>>> /home/test/oisf_suricata_ids/suricata-1.0.1pregit24jul2010/libhtp/htp/.libs/libhtp-0.2.so.1.0.2
>>>>>>
>>>>>> >>>
>>>>>> >>> b7759000-b775a000 rw-p 00000000 00:00 0
>>>>>> >>> b775a000-b775b000 r-xp 00000000 00:00 0 [vdso]
>>>>>> >>> bf96c000-bf98d000 rw-p 00000000 00:00 0 [stack]
>>>>>> >>> Abandon
>>>>>> >>>
>>>>>> >>> Regards
>>>>>> >>> Rmkml
>>>>>> >>>
>>>>>> >>>
>>>>>> >>>
>>>>>> >>> On Sat, 24 Jul 2010, Victor Julien wrote:
>>>>>> >>>
>>>>>> >>>> rmkml at free.fr <mailto:rmkml at free.fr> wrote:
>>>>>> >>>>> I have new:
>>>>>> >>>>> On git 21 jul, mem usage pb appear, but I have a small
>>>>>> (revert)
>>>>>> >>>>> change "resolv"
>>>>>> >>>>> my pb, Move (back) this Line on if loop /* content */:
>>>>>> >>>>> PatternMatchPreparePopulateMpm(de_ctx, sh);
>>>>>> >>>>> #line 1081 in src/detect-engine-mpm.c
>>>>>> >>>>
>>>>>> >>>> Thanks Rmkml. At this point I don't think there is
>>>>>> anything wrong in the
>>>>>> >>>> code there. The changes were done to fix some accuracy
>>>>>> issues we were
>>>>>> >>>> seeing.
>>>>>> >>>>
>>>>>> >>>> I did cleanup the code a bit in the latest git master. I
>>>>>> don't expect
>>>>>> >>>> anything to change, but maybe you can try anyway :)
>>>>>> >>>>
>>>>>> >>>> Cheers,
>>>>>> >>>> Victor
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>>> --
>>>>>> >>>> ---------------------------------------------
>>>>>> >>>> Victor Julien
>>>>>> >>>> http://www.inliniac.net/
>>>>>> >>>> PGP: http://www.inliniac.net/victorjulien.asc
>>>>>> >>>> ---------------------------------------------
>>>>>> >>>>
>>>>>> >>>>
>>>>>> >>
>>>>>> >>
>>>>>> >> --
>>>>>> >> ---------------------------------------------
>>>>>> >> Victor Julien
>>>>>> >> http://www.inliniac.net/
>>>>>> >> PGP: http://www.inliniac.net/victorjulien.asc
>>>>>> >> ---------------------------------------------
>>>>>> >>
>>>>>> >>
>>>>>> >
>>>>>> >
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Oisf-devel mailing list
>>>>>> Oisf-devel at openinfosecfoundation.org
>>>>>> <mailto:Oisf-devel at openinfosecfoundation.org>
>>>>>>
>>>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Anoop Saldanha
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Regards,
>>>>>> Anoop Saldanha
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>> _______________________________________________
>>>>>> Oisf-devel mailing list
>>>>>> Oisf-devel at openinfosecfoundation.org
>>>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>>>
>>>>>
>>>>> --
>>>>> ---------------------------------------------
>>>>> Victor Julien
>>>>> http://www.inliniac.net/
>>>>> PGP: http://www.inliniac.net/victorjulien.asc
>>>>> ---------------------------------------------
>>>>>
>>>>>
>>>
>>>
>>> --
>>> ---------------------------------------------
>>> Victor Julien
>>> http://www.inliniac.net/
>>> PGP: http://www.inliniac.net/victorjulien.asc
>>> ---------------------------------------------
>>>
>>> _______________________________________________
>>> Oisf-devel mailing list
>>> Oisf-devel at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>
>>
>>
>
More information about the Oisf-devel
mailing list