[Oisf-users] Capture loss ~50% reported using Myricom with Suri v 5.0.2

[Edgmand, Craig]

Hi Fetema,

Not an expert, but have you tried increasing these


SNF_DATARING_SIZE=4096MB

SNF_DESCRING_SIZE=1024MB

If you have the memory, I would multiply these by a factor of 4.  On my servers these numbers are huge. Might also increase buffer size.

Thanks,

Craig

From: Oisf-users <oisf-users-bounces at lists.openinfosecfoundation.org> On Behalf Of fatema bannatwala
Sent: Friday, June 19, 2020 12:59 PM
To: Open Information Security Foundation <oisf-users at lists.openinfosecfoundation.org>
Subject: [Oisf-users] Capture loss ~50% reported using Myricom with Suri v 5.0.2

CAUTION: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe
Hello Experts,

Need some help tuning down our prod suricata box running Suricata v 5.0.2 with Myricom NIC: 10G-PCIE-8B-S myri_snf 3.0.20.50894

It is consistently reporting ~50% capture loss, calculated based off of the capture.kernel_packets and capture.kernel_dropped values reported in stats.log file.

I have followed the https://blog.inliniac.net/2012/07/10/suricata-on-myricom-capture-cards/<https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.inliniac.net%2F2012%2F07%2F10%2Fsuricata-on-myricom-capture-cards%2F&data=02%7C01%7Ccraig.edgmand%40okstate.edu%7Cfebd2ad7dac24eddbea508d8147a76d5%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C1%7C637281863494345977&sdata=XcrXhcZVFFiw280MZJxI6JVssD%2BdlYEyyYJJdsSIxLg%3D&reserved=0>
guide to pin the cpus to the worker nodes and use pcap.buffer_size to increase the SNF dataring size, but no effect..

We have one Myri card connected to p2p1 and two NUMA nodes, each with 8 cores (16 HT):
NUMA node0 CPU(s):     0-7,16-23
NUMA node1 CPU(s):     8-15,24-31
OS: Centos 7

Any help in the right direction would be appreciated! :)

Thanks!
Fatema

Following is settings from suricata.yml file


# Myricom support

pcap:

  - interface: p2p1

    threads: 14

    buffer-size: 2gb

    checksum-checks: no

pcap-file:

  checksum-checks: auto



threading:

  set-cpu-affinity: yes

  cpu-affinity:

    - management-cpu-set:

        cpu: [ "0" ]

        mode: "balanced"

        prio:

          default: "low"

    - worker-cpu-set:

        cpu: [ "1-7","9-15" ]

        mode: "exclusive"

        prio:

          default: "high"


Following is the currently recorded stats.log:

------------------------------------------------------------------------------------

Date: 6/19/2020 -- 10:55:36 (uptime: 0d, 04h 04m 10s)

------------------------------------------------------------------------------------

Counter                                       | TM Name                   | Value

------------------------------------------------------------------------------------

capture.kernel_packets                        | Total                     | 28447139411

capture.kernel_drops                          | Total                     | 27910518132

capture.kernel_ifdrops                        | Total                     | 6034

decoder.pkts                                  | Total                     | 536633135



SNF parameters:





SNF_APP_ID=32

SNF_DATARING_SIZE=4096MB

SNF_DESCRING_SIZE=1024MB

SNF_NUM_RINGS=14

SNF_FLAGS=0x1



LD_PRELOAD="/opt/snf/lib/libpcap.so.1"



OPTIONS="--user suricata --group suricata --pcap"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20200619/41b0aacf/attachment-0001.html>