[Oisf-users] Suricata runs out of memory on startup

[Will Metcalf]

Can you create a redmine ticket and attach a scrubbed version of your
suricata.yaml?  Along with output of free -m prior to starting suri?

https://redmine.openinfosecfoundation.org/projects/suricata

Regards,

Will
On Wed, Jul 27, 2011 at 4:35 PM, Gene Albin <gene.albin at gmail.com> wrote:
> Ok,  I'm probably doing something wrong here, but every time I try to load a
> combined rule file with all of the VRT and ET rules enabled (~30K rules) it
> fails following stage 3:
>
> [7069] 27/7/2011 -- 14:14:09 - (detect.c:631) <Info> (SigLoadSignatures) --
> 102 rule files processed. 30183 rules succesfully loaded, 164 rules failed
> [7069] 27/7/2011 -- 14:14:47 - (detect.c:2161) <Info>
> (SigAddressPrepareStage1) -- 30701 signatures processed. 1800 are IP-only
> rules, 20152 are inspecting packet payload, 11088 inspect application layer,
> 0 are decoder event only
> [7069] 27/7/2011 -- 14:14:47 - (detect.c:2164) <Info>
> (SigAddressPrepareStage1) -- building signature grouping structure, stage 1:
> adding signatures to signature source addresses... complete
> [7069] 27/7/2011 -- 14:14:48 - (detect.c:2806) <Info>
> (SigAddressPrepareStage2) -- building signature grouping structure, stage 2:
> building source address list... complete
> [7069] 27/7/2011 -- 14:16:40 - (detect.c:3363) <Info>
> (SigAddressPrepareStage3) -- MPM memory 1801173581 (dynamic 1801173581, ctxs
> 0, avg per ctx 0)
> [7069] 27/7/2011 -- 14:16:40 - (detect.c:3365) <Info>
> (SigAddressPrepareStage3) -- max sig id 30701, array size 3838
> [7069] 27/7/2011 -- 14:16:40 - (detect.c:3376) <Info>
> (SigAddressPrepareStage3) -- building signature grouping structure, stage 3:
> building destination address lists... complete
> [7069] 27/7/2011 -- 14:16:43 - (detect-engine-siggroup.c:1583) <Error>
> (SigGroupHeadBuildHeadArray) -- [ERRCODE: SC_ERR_MEM_ALLOC(1)] - SCMalloc
> failed: Cannot allocate memory, while trying to allocate 558852 bytes
>
> [7069] 27/7/2011 -- 14:16:43 - (detect-engine-siggroup.c:1583) <Error>
> (SigGroupHeadBuildHeadArray) -- [ERRCODE: SC_ERR_FATAL(169)] - Out of
> memory. The engine cannot be initialized. Exiting...
>
> I have done this while watching the memory useage in top (set to refresh
> every .2 seconds).  Initially when this happened I only had 4GB allocated to
> the VM.  Useage never gets beyond 2GB so that left almost 2GB available.  I
> decided to bump the VM up to 8GB but the problem didn't go away.  It still
> exits when the memory useage gets to around 2GB.
>
> Everything works fine when I load a reduced ruleset, i.e. just VRT or just
> ET, but for my tests I want to load both.  Before I go back to the VM
> administrator and ask for 16 GB (and wait several days for the allocation) I
> was wondering if there might be a config setting that is limiting the size
> of memory allocated to the rules.
>
> Running 1.1b2 on CentOS 5.6 - 4core VMWare ESXi.
>
> Any suggestions are welcome.
>
> Gene
>
> --
> Gene Albin
> gene.albin at gmail.com
>
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
>