[Oisf-users] Suricata, 10k rules, 10Gbit/sec and lots of RAM

[Peter Manev]

> On 5 nov 2014, at 01:08, Michał Purzyński <michalpurzynski1 at gmail.com> wrote:
> 
> Suricata 2.0.4, 128GB memory, around 10.5k rules from ET. The startup
> process is loooong, then it fails, eating all the memory. Is it
> expected? I've tried using ac-bs but gave up after like >20 minutes
> waiting for it to start.
> 
> detect-engine:
>  - profile: custom
>  - custom-values:
>      toclient-src-groups: 200
>      toclient-dst-groups: 200
>      toclient-sp-groups: 200
>      toclient-dp-groups: 300
>      toserver-src-groups: 200
>      toserver-dst-groups: 400
>      toserver-sp-groups: 200
>      toserver-dp-groups: 200
>  - sgh-mpm-context: full
>  - inspection-recursion-limit: 3000
> 
> mpm-algo: ac
> 
> Now, if I change the sgh-mpm-context to 'auto' it can start, using
> around 40GB of memory. Does it mean that auto = single?

Usually yes.

> 
> I'm kind of concerned that rules cannot fit in the memory with
> sgh-mpm-context set to full and the settings presented. Should I be?
> :)

There is a bug at the moment when using full with over 10k rules - it just ends up eating all the memory. 
Can you try under 10k rules to see if any diff?


> 
> What is better - use profile: high and context: full (if it fits) or
> profile: custom with settings presented and sgh-mpm-context: auto?

Profile: high and context: full are enough in my opinion - but you can try also high and auto. (with just ac)

> 
> -- 
> Michał Purzyński
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Training now available: http://suricata-ids.org/training/