[Oisf-users] Tuning Suricata with Myricom Sniffer10g Hardware

[Michał Purzyński]

> In particular, I was wondering if anyone had any insight into Suricata’s
> memory usage with regards to DATARING_SIZE, as you can see here from the
> output of ‘top’, the Suricata memory increases *rapidly* with respect to the
> DATARING_SIZE.

Simple.

 4577 root      20   0 2585g 2.5t 2.5t S 1417 2052.9  16386:51
Suricata-Main
No, my Suricata sensor does not have 2.5TB RAM. I use 32GB dataring
and 8GB descriptor ring sizes (the later should be x4 size of the
former)

export LD_LIBRARY_PATH=/opt/snf/lib
export SNF_NUM_RINGS=32
export SNF_DATARING_SIZE=34359738368
export SNF_DESCRING_SIZE=8589934592
export SNF_DEBUG_MASK=0x3

             total       used       free     shared    buffers     cached
Mem:        128903      73597      55305          0        201        646
-/+ buffers/cache:      72749      56154
Swap:            0          0          0

That means I have around 70GB used and that includes both rings.
Here's why - tools like "top" sees the same 32+8GB buffer mapped in
each of the worker threads so when you add 1GB to buffer you see
memory usage going up by 32GB (with 32 workers), which is not true.

When tuning the ring sizes you want it as small as possible but still
a couple of GB. This is a tradeoff between the amount of packets you
can buffer and the dTLB pressure.

>
> **The yaml file was constant throughout all 3 tests – only the SNF ring
> parameters changed**
>
>
>
> (DATARING_SIZE = 256MB (default), DESCRING_SIZE=64MB (default) – 8
> rings/threads)
>
> 61300 user  20   0 15.4g  13g 5.0g S 221.7  5.2  36:43.21 Suricata-Main
>
>
>
> (DATARING_SIZE = 4GB, DESCRING_SIZE=1GB – 8 rings/threads)
>
> 28208 user  20   0 90.4g  88g  80g S 281.3 35.1   8:07.91 Suricata-Main
>
>
>
> (DATARING_SIZE = 8GB, DESCRING_SIZE=2GB – 8 rings/threads)
>
> 55652 user  20   0  170g 168g 160g S 226.8 66.9  30:39.41 Suricata-Main
>
>
>
> In addition, since Suricata is leveraging SNF through libpcap, I am
> wondering if it is known how the pcap.buffer-size parameter that is defined
> in the suricata.yaml relates to the DATARING_SIZE/DESCRING_SIZE parameters?

I went through the libpcap code a while ago and no, they don't seem to
be related. pcap buffer is just that - a library userspace buffer.

data/desc ring are buffer that the card sends packets to with on board
DMA processor. It is allocated in the userspace and mapped into each
processes. Both are managed not by libpcap but libsnf.

>
>
>
> pcap:
>
>   - interface: eth4
>
>     # On Linux, pcap will try to use mmaped capture and will use buffer-size
>
>     # as total of memory used by the ring. So set this to something bigger
>
>     # than 1% of your bandwidth.
>
>     buffer-size: 16777216
>
>
>
> I appreciate any guidance the community can provide on these items, in
> addition to any other tuning considerations when using Suricata with
> Sniffer10g hardware.
>
>

Shoot any questions you might have, I use between 10-20 Myricom
sensors with various NSM software, some of them with Suricata.

>
> Zach
>
>
>
>
>
> ________________________
>
> Zach Rasmor
>
> Senior Software Engineer
>
> Lockheed Martin CIRT
>
> 700 N Frederick Ave | Gaithersburg, MD 20879
>
> Email: zachary.r.rasmor at lmco.com
>
> Office: 301.240.6116
>
>
>
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Training now available: http://suricata-ids.org/training/