[Oisf-users] Suricata runs out of memory on startup

[Matt Jonkman]

Are you on the most recent git version by chance? The resolution of
flow bits in ip only rules solved a similar problem for me in the dev
version, just this week.

Thanks gene!


----------------------------------------------------
Matthew Jonkman
Emerging Threats
Open Information Security Foundation (OISF)
Phone 765-429-0398
Fax 312-264-0205
http://www.emergingthreats.net
http://www.openinfosecfoundation.org
----------------------------------------------------

On Jul 27, 2011, at 7:02 PM, Will Metcalf <william.metcalf at gmail.com> wrote:

> Can you create a redmine ticket and attach a scrubbed version of your
> suricata.yaml?  Along with output of free -m prior to starting suri?
>
> https://redmine.openinfosecfoundation.org/projects/suricata
>
> Regards,
>
> Will
> On Wed, Jul 27, 2011 at 4:35 PM, Gene Albin <gene.albin at gmail.com> wrote:
>> Ok,  I'm probably doing something wrong here, but every time I try to load a
>> combined rule file with all of the VRT and ET rules enabled (~30K rules) it
>> fails following stage 3:
>>
>> [7069] 27/7/2011 -- 14:14:09 - (detect.c:631) <Info> (SigLoadSignatures) --
>> 102 rule files processed. 30183 rules succesfully loaded, 164 rules failed
>> [7069] 27/7/2011 -- 14:14:47 - (detect.c:2161) <Info>
>> (SigAddressPrepareStage1) -- 30701 signatures processed. 1800 are IP-only
>> rules, 20152 are inspecting packet payload, 11088 inspect application layer,
>> 0 are decoder event only
>> [7069] 27/7/2011 -- 14:14:47 - (detect.c:2164) <Info>
>> (SigAddressPrepareStage1) -- building signature grouping structure, stage 1:
>> adding signatures to signature source addresses... complete
>> [7069] 27/7/2011 -- 14:14:48 - (detect.c:2806) <Info>
>> (SigAddressPrepareStage2) -- building signature grouping structure, stage 2:
>> building source address list... complete
>> [7069] 27/7/2011 -- 14:16:40 - (detect.c:3363) <Info>
>> (SigAddressPrepareStage3) -- MPM memory 1801173581 (dynamic 1801173581, ctxs
>> 0, avg per ctx 0)
>> [7069] 27/7/2011 -- 14:16:40 - (detect.c:3365) <Info>
>> (SigAddressPrepareStage3) -- max sig id 30701, array size 3838
>> [7069] 27/7/2011 -- 14:16:40 - (detect.c:3376) <Info>
>> (SigAddressPrepareStage3) -- building signature grouping structure, stage 3:
>> building destination address lists... complete
>> [7069] 27/7/2011 -- 14:16:43 - (detect-engine-siggroup.c:1583) <Error>
>> (SigGroupHeadBuildHeadArray) -- [ERRCODE: SC_ERR_MEM_ALLOC(1)] - SCMalloc
>> failed: Cannot allocate memory, while trying to allocate 558852 bytes
>>
>> [7069] 27/7/2011 -- 14:16:43 - (detect-engine-siggroup.c:1583) <Error>
>> (SigGroupHeadBuildHeadArray) -- [ERRCODE: SC_ERR_FATAL(169)] - Out of
>> memory. The engine cannot be initialized. Exiting...
>>
>> I have done this while watching the memory useage in top (set to refresh
>> every .2 seconds).  Initially when this happened I only had 4GB allocated to
>> the VM.  Useage never gets beyond 2GB so that left almost 2GB available.  I
>> decided to bump the VM up to 8GB but the problem didn't go away.  It still
>> exits when the memory useage gets to around 2GB.
>>
>> Everything works fine when I load a reduced ruleset, i.e. just VRT or just
>> ET, but for my tests I want to load both.  Before I go back to the VM
>> administrator and ask for 16 GB (and wait several days for the allocation) I
>> was wondering if there might be a config setting that is limiting the size
>> of memory allocated to the rules.
>>
>> Running 1.1b2 on CentOS 5.6 - 4core VMWare ESXi.
>>
>> Any suggestions are welcome.
>>
>> Gene
>>
>> --
>> Gene Albin
>> gene.albin at gmail.com
>>
>>
>> _______________________________________________
>> Oisf-users mailing list
>> Oisf-users at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>
>>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users