[Oisf-users] Suricata Myricom and 10Gbit

[Anoop Saldanha]

Michal,

What ruleset are you using?  Have you disabled decoder rules?

On Mon, Mar 31, 2014 at 8:22 PM, Erich Lerch <erich.lerch at gmail.com> wrote:
> Michał,
>
> We have a similar setup, also with the Myricom 10gb interface.
>
> The following values give us fairly good results, although our traffic
> does not exceed 2Gbit at the moment.
> Performance also depends on the ruleset.
> If you have profiling compiled in, try without.
>
> SNF settings:
> SNF_NUM_RINGS=16  SNF_DATARING_SIZE=34359738368 SNF_DESCRING_SIZE=2147483648
>
> Suricata settings:
>
> ...
> max-pending-packets: 2048
> ...
> # very important:
> detect-engine:
>   - profile: custom
>   - custom-values:
>       toclient-src-groups: 200
>       toclient-dst-groups: 200
>       toclient-sp-groups: 200
>       toclient-dp-groups: 300
>       toserver-src-groups: 200
>       toserver-dst-groups: 400
>       toserver-sp-groups: 200
>       toserver-dp-groups: 200
>   - sgh-mpm-context: single
>   - inspection-recursion-limit: 3000
>   - rule-reload: true
> ...
> threading:
>   set-cpu-affinity: yes
>   [definition of cpu sets]
> ...
> defrag:
>   memcap: 512mb
>   hash-size: 65536
>   trackers: 65535
>   max-frags: 65535
>   prealloc: yes
>   timeout: 10
> ...
> flow:
>   memcap: 256mb
>   hash-size: 262144
>   prealloc: 300000
>   emergency-recovery: 30
> ...
> flow-timeouts:
>   [a lot more aggressive than the default!!!]
> ...
> stream:
>   memcap: 12gb
>   checksum-validation: no
>   inline: no
>   prealloc-sessions: 10000000
>   reassembly:
>     memcap: 16gb
>     depth: 6mb                 # reassemble 1mb into a stream
>     toserver-chunk-size: 2560
>     toclient-chunk-size: 2560
>     randomize-chunk-size: yes
> ...
>
> pcap:
>   - interface: snf0
>     threads: 16
>     buffer-size: 512mb
>     checksum-checks: no
>     promisc: no
> ...
>
> Also set the app-layer values much higher than the defaults.
>
> cheers,
> erich
>
>
> 2014-03-31 13:27 GMT+02:00 Michał Purzyński <michalpurzynski1 at gmail.com>:
>> Hello.
>>
>> I'm trying to tune Suricata to handle up to 10Gbit/sec of traffic (that's a
>> peak, jumps like crazy from 2.5 - 4.5 - 6 and up). So far my results were
>> quite bad, so I'm seeking help - must be missing something obvious here
>> judging by the numbers of articles where everyone seems to use Suricata on
>> 10Gbit traffic.
>>
>> Server:
>>
>> 2 x Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (16 physical cores)
>> 64GB RAM
>>
>> NIC - Myricom 10Gb 10G-PCIE-8B-S with the Sniffer software loaded and
>> activated
>>
>> Software:
>>
>> This is Suricata version 2.0rc2 RELEASE
>>
>> Command line:
>>
>> SNF_NUM_RINGS=16 SNF_FLAGS=0x1 SNF_DESCRING_SIZE=1073741824
>> SNF_DATARING_SIZE=1073741824 SNF_DEBUG_MASK=0x3 suricata -c
>> /etc/nsm/nsm11-eth4/suricata.yaml -i eth4 --runmode=workers
>>
>> (16 threads, 1GB for each buffer)
>>
>> The Myricom debug output seems fine.
>>
>> Config file - pretty standard, most important things:
>>
>> max-pending-packets: 5000
>> runmode: workers
>>
>> detect-engine:
>>   - profile: medium
>>
>> Did not touch parameters here.
>>
>>   set-cpu-affinity: no
>>
>> Also default settings here.
>>
>>   detect-thread-ratio: 1.5
>>
>> (should not it be 1.0?)
>>
>> defrag:
>>   memcap: 512mb
>>   trackers: 65535 # number of defragmented flows to follow
>>   max-frags: 65535 # number of fragments to keep (higher than trackers)
>>   prealloc: yes
>>   timeout: 60
>>
>> flow:
>>   memcap: 32mb
>>   hash-size: 65536
>>   prealloc: 10000
>>   emergency-recovery: 30
>>
>> stream:
>>   memcap: 16gb
>>   max-sessions: 20000000
>>   prealloc-sessions: 10000000
>>   checksum-validation: yes      # reject wrong csums
>>   inline: no                    # no inline mode
>>   reassembly:
>>     memcap: 14gb
>>     depth: 6mb                  # reassemble 1mb into a stream
>>     toserver-chunk-size: 2560
>>     toclient-chunk-size: 2560
>>
>> pcap:
>>   - interface: eth4
>>     threads: 16
>>     buffer-size: 512kb
>>     checksum-checks: no
>>
>> The myricom tools show a high packet loss
>>
>>                      SNF recv pkts:            634485790
>>                 SNF drop ring full:            137774061
>>                         Interrupts:             12053363
>>            Net bad PHY/CRC32 drop:                32092
>>                  Net overflow drop:               219656
>>
>> Also note that it reports quite a few interrupts, which there should be
>> almost none.
>>
>> What is the direction I should go here? I know that tuning a high capacity
>> Suricata isn't exactly a single afternoon task, but I need to advise what to
>> do now, how to proceed, etc.
>>
>> Looking for clues.
>>
>> --
>> Michał Purzyński
>>
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> OISF: http://www.openinfosecfoundation.org/
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/



-- 
-------------------------------
Anoop Saldanha
http://www.poona.me
-------------------------------