[Oisf-users] What limit to remove?

[Mark Ashley]

To answer my own question, since it's a 32bit compiled binary, the limit
for the process address map is 4GB.

I'll have to prune the resource usage, or re-compile the binary and
libraries 64bit.


On Wed, Dec 18, 2013 at 4:33 PM, Mark Ashley <mark at ibiblio.org> wrote:

> Solaris 11 x86
> X4600 128GB RAM, 16 x AMD8220s
> Suricata master-2013-12-02
>
> I'm seeing memory "exhaustion" issues with starting suricata with pretty
> much all of the rules switched on, including the emerging ones. The process
> gets up to about 3.85GB RSS and then crashes with the errors below.
>
> I'm looking at the suricata.yaml file and trying to find out where I can
> increase limits to allow the rules to be there, and suricata to cache them.
> There's no issue with available RAM or a ulimit setting. It can have it all.
>
> ta,
> Mark.
>
>
> [1] 18/12/2013 -- 15:01:53 - (suricata.c:1877) <Info>
> (PostConfLoadedSetup) -- No 'host-mode': suricata is in IDS mode, using
> default setting 'sniffer-only'
> [1] 18/12/2013 -- 15:01:53 - (defrag-hash.c:209) <Info> (DefragInitConfig)
> -- allocated 1280000 bytes of memory for the defrag hash... 40000 buckets
> of size 32
> [1] 18/12/2013 -- 15:01:53 - (defrag-hash.c:236) <Info> (DefragInitConfig)
> -- preallocated 50000 defrag trackers of size 136
> [1] 18/12/2013 -- 15:01:53 - (defrag-hash.c:243) <Info> (DefragInitConfig)
> -- defrag memory usage: 8080000 bytes, maximum: 4294967296
> [1] 18/12/2013 -- 15:01:53 - (tmqh-flow.c:61) <Info> (TmqhFlowRegister) --
> AutoFP mode using "Round Robin" flow load balancer
> [1] 18/12/2013 -- 15:01:53 - (suricata.c:1910) <Info>
> (PostConfLoadedSetup) -- Will use direct allocation instead of packet pool
> [1] 18/12/2013 -- 15:01:53 - (host.c:202) <Info> (HostInitConfig) --
> allocated 131072 bytes of memory for the host hash... 4096 buckets of size
> 32
> [1] 18/12/2013 -- 15:01:53 - (host.c:227) <Info> (HostInitConfig) --
> preallocated 5000 hosts of size 96
> [1] 18/12/2013 -- 15:01:53 - (host.c:229) <Info> (HostInitConfig) -- host
> memory usage: 651072 bytes, maximum: 17179869184
> [1] 18/12/2013 -- 15:01:53 - (flow.c:383) <Info> (FlowInitConfig) --
> allocated 2097152 bytes of memory for the flow hash... 65536 buckets of
> size 32
> [1] 18/12/2013 -- 15:01:55 - (flow.c:409) <Info> (FlowInitConfig) --
> preallocated 1000000 flows of size 236
> [1] 18/12/2013 -- 15:01:55 - (flow.c:411) <Info> (FlowInitConfig) -- flow
> memory usage: 242097152 bytes, maximum: 4294967296
> [1] 18/12/2013 -- 15:01:55 - (reputation.c:459) <Info> (SRepInit) -- IP
> reputation disabled
> [1] 18/12/2013 -- 15:01:55 - (util-magic.c:62) <Info> (MagicInit) -- using
> magic-file /usr/local/share/misc/magic.mgc
> [1] 18/12/2013 -- 15:01:55 - (suricata.c:1753) <Info> (SetupDelayedDetect)
> -- Delayed detect disabled
> [1] 18/12/2013 -- 15:02:11 - (detect.c:453) <Info> (SigLoadSignatures) --
> 49 rule files processed. 15023 rules successfully loaded, 0 rules failed
> [1] 18/12/2013 -- 15:02:11 - (detect.c:2564) <Info>
> (SigAddressPrepareStage1) -- 15040 signatures processed. 1039 are IP-only
> rules, 5182 are inspecting packet payload, 10662 inspect application layer,
> 83 are decoder event only
> [1] 18/12/2013 -- 15:02:11 - (detect.c:2570) <Info>
> (SigAddressPrepareStage1) -- building signature grouping structure, stage
> 1: preprocessing rules... complete
> [1] 18/12/2013 -- 15:02:13 - (detect.c:3194) <Info>
> (SigAddressPrepareStage2) -- building signature grouping structure, stage
> 2: building source address list... complete
> [1] 18/12/2013 -- 15:03:39 - (util-mpm-ac.c:430) <Error>
> (SCACInitNewState) -- [ERRCODE: SC_ERR_MEM_ALLOC(1)] - SCRealloc failed:
> Not enough space, while trying to allocate 15066112 bytes
> [1] 18/12/2013 -- 15:03:39 - (util-mpm-ac.c:430) <Error>
> (SCACInitNewState) -- [ERRCODE: SC_ERR_FATAL(171)] - Out of memory. The
> engine cannot be initialized. Exiting...
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20131218/33ddf2a9/attachment-0002.html>