[Oisf-users] Suricata runs out of memory on startup

[Dave Remien]

On Wed, Jul 27, 2011 at 5:02 PM, Will Metcalf <william.metcalf at gmail.com>wrote:

> Can you create a redmine ticket and attach a scrubbed version of your
> suricata.yaml?  Along with output of free -m prior to starting suri?
>

Are you running a 32 bit kernel with a 2GB/2GB memory split, by any chance??

Cheers,

Dave


>
> https://redmine.openinfosecfoundation.org/projects/suricata
>
> Regards,
>
> Will
> On Wed, Jul 27, 2011 at 4:35 PM, Gene Albin <gene.albin at gmail.com> wrote:
> > Ok,  I'm probably doing something wrong here, but every time I try to
> load a
> > combined rule file with all of the VRT and ET rules enabled (~30K rules)
> it
> > fails following stage 3:
> >
> > [7069] 27/7/2011 -- 14:14:09 - (detect.c:631) <Info> (SigLoadSignatures)
> --
> > 102 rule files processed. 30183 rules succesfully loaded, 164 rules
> failed
> > [7069] 27/7/2011 -- 14:14:47 - (detect.c:2161) <Info>
> > (SigAddressPrepareStage1) -- 30701 signatures processed. 1800 are IP-only
> > rules, 20152 are inspecting packet payload, 11088 inspect application
> layer,
> > 0 are decoder event only
> > [7069] 27/7/2011 -- 14:14:47 - (detect.c:2164) <Info>
> > (SigAddressPrepareStage1) -- building signature grouping structure, stage
> 1:
> > adding signatures to signature source addresses... complete
> > [7069] 27/7/2011 -- 14:14:48 - (detect.c:2806) <Info>
> > (SigAddressPrepareStage2) -- building signature grouping structure, stage
> 2:
> > building source address list... complete
> > [7069] 27/7/2011 -- 14:16:40 - (detect.c:3363) <Info>
> > (SigAddressPrepareStage3) -- MPM memory 1801173581 (dynamic 1801173581,
> ctxs
> > 0, avg per ctx 0)
> > [7069] 27/7/2011 -- 14:16:40 - (detect.c:3365) <Info>
> > (SigAddressPrepareStage3) -- max sig id 30701, array size 3838
> > [7069] 27/7/2011 -- 14:16:40 - (detect.c:3376) <Info>
> > (SigAddressPrepareStage3) -- building signature grouping structure, stage
> 3:
> > building destination address lists... complete
> > [7069] 27/7/2011 -- 14:16:43 - (detect-engine-siggroup.c:1583) <Error>
> > (SigGroupHeadBuildHeadArray) -- [ERRCODE: SC_ERR_MEM_ALLOC(1)] - SCMalloc
> > failed: Cannot allocate memory, while trying to allocate 558852 bytes
> >
> > [7069] 27/7/2011 -- 14:16:43 - (detect-engine-siggroup.c:1583) <Error>
> > (SigGroupHeadBuildHeadArray) -- [ERRCODE: SC_ERR_FATAL(169)] - Out of
> > memory. The engine cannot be initialized. Exiting...
> >
> > I have done this while watching the memory useage in top (set to refresh
> > every .2 seconds).  Initially when this happened I only had 4GB allocated
> to
> > the VM.  Useage never gets beyond 2GB so that left almost 2GB available.
> I
> > decided to bump the VM up to 8GB but the problem didn't go away.  It
> still
> > exits when the memory useage gets to around 2GB.
> >
> > Everything works fine when I load a reduced ruleset, i.e. just VRT or
> just
> > ET, but for my tests I want to load both.  Before I go back to the VM
> > administrator and ask for 16 GB (and wait several days for the
> allocation) I
> > was wondering if there might be a config setting that is limiting the
> size
> > of memory allocated to the rules.
> >
> > Running 1.1b2 on CentOS 5.6 - 4core VMWare ESXi.
> >
> > Any suggestions are welcome.
> >
> > Gene
> >
> > --
> > Gene Albin
> > gene.albin at gmail.com
> >
> >
> > _______________________________________________
> > Oisf-users mailing list
> > Oisf-users at openinfosecfoundation.org
> > http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> >
> >
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>



-- 
"Of course, someone who knows more about this will correct me if I'm
wrong, and someone who knows less will correct me if I'm right."
David Palmer (palmer at tybalt.caltech.edu)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20110727/e61ebcc3/attachment-0002.html>