[Oisf-users] Capture loss ~50% reported using Myricom with Suri v 5.0.2
Erich Lerch
erich.lerch at gmail.com
Fri Jun 19 20:17:14 UTC 2020
Hi
Craig is right, we also have huge numbers there, and no packet loss.
These settings really are important.
That's how we start suri:
SNF_NUM_RINGS=10 SNF_FLAGS=0x1 SNF_DATARING_SIZE=12884901888
SNF_DESCRING_SIZE=3221225472 LD_PRELOAD=... \
/opt/suricata/bin/suricata -v -c suricata.yaml -i snf0 -D
The "pcap" part from yuricata.yaml:
# ----------------------------
pcap:
- interface: snf0
threads: 10
buffer-size: 2gb
checksum-checks: no
promisc: no
snaplen: 1520
bpf-filter: "<your_filter>"
# ----------------------------
Cheers,
Erich
On 19.06.20 20:29, Edgmand, Craig wrote:
> Hi Fatema,
>
>
>
> That’s interesting because the way I read it from this article
>
>
>
> https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Myricom
>
>
>
> You could set them on the command line like this…
>
>
>
> SNF_NUM_RINGS=16 SNF_DATARING_SIZE=17179869184
> SNF_DESCRING_SIZE=4294967296 SNF_FLAGS=0x1 suricata -c suricata.yaml -i
> eth5 --runmode=workers
>
>
>
>
>
> I only use Myricom cards on my Zeek servers so I haven’t tested it and
> it has no such restrictions.
>
>
>
> Good luck,
>
>
>
> Craig
>
>
>
> *From:*fatema bannatwala <fatema.bannatwala at gmail.com>
> *Sent:* Friday, June 19, 2020 1:16 PM
> *To:* Edgmand, Craig <craig.edgmand at okstate.edu>
> *Cc:* Open Information Security Foundation
> <oisf-users at lists.openinfosecfoundation.org>
> *Subject:* Re: [Oisf-users] Capture loss ~50% reported using Myricom
> with Suri v 5.0.2
>
>
>
> *CAUTION:*This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe
>
> Thanks Craig, I tried increasing SNF_DATARING_SIZE, but that variable
> gets overwritten and controlled by pcap.buffer-size in suricata.yml file
> which allows a max of 2gb, can't set more than that.
>
> Hence setting SNF_DATARING_SIZE explicitly has no effect since.
>
>
>
> This has been done:
>
> The following pull request opened by Myricom in the libpcap project
> indicates that a future SNF software release could provide support for
> setting the SNF_DATARING_SIZE via the pcap.buffer-size yaml setting:
>
> Ref: https://github.com/the-tcpdump-group/libpcap/pull/435
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fthe-tcpdump-group%2Flibpcap%2Fpull%2F435&data=02%7C01%7Ccraig.edgmand%40okstate.edu%7Cd4444c14ab184963043408d8147cf441%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C0%7C637281874192318869&sdata=jhJDwiTawSglR4%2FOm33%2F685%2B2oXHYSnzu0ysXIj0%2BaE%3D&reserved=0>
>
>
>
>
>
>
>
> On Fri, Jun 19, 2020 at 11:09 AM Edgmand, Craig
> <craig.edgmand at okstate.edu <mailto:craig.edgmand at okstate.edu>> wrote:
>
> Hi Fetema,
>
>
>
> Not an expert, but have you tried increasing these
>
>
>
> SNF_DATARING_SIZE=4096MB
>
> SNF_DESCRING_SIZE=1024MB
>
>
>
> If you have the memory, I would multiply these by a factor of 4. On
> my servers these numbers are huge. Might also increase buffer size.
>
>
>
> Thanks,
>
>
>
> Craig
>
>
>
> *From:*Oisf-users
> <oisf-users-bounces at lists.openinfosecfoundation.org
> <mailto:oisf-users-bounces at lists.openinfosecfoundation.org>> *On
> Behalf Of *fatema bannatwala
> *Sent:* Friday, June 19, 2020 12:59 PM
> *To:* Open Information Security Foundation
> <oisf-users at lists.openinfosecfoundation.org
> <mailto:oisf-users at lists.openinfosecfoundation.org>>
> *Subject:* [Oisf-users] Capture loss ~50% reported using Myricom
> with Suri v 5.0.2
>
>
>
> *CAUTION:*This email originated from outside of the organization. Do
> not click links or open attachments unless you recognize the sender
> and know the content is safe
>
> Hello Experts,
>
>
>
> Need some help tuning down our prod suricata box running Suricata v
> 5.0.2 with Myricom NIC: 10G-PCIE-8B-S myri_snf 3.0.20.50894
>
>
>
> It is consistently reporting ~50% capture loss, calculated based off
> of the capture.kernel_packets and capture.kernel_dropped values
> reported in stats.log file.
>
>
>
> I have followed
> the https://blog.inliniac.net/2012/07/10/suricata-on-myricom-capture-cards/
> <https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.inliniac.net%2F2012%2F07%2F10%2Fsuricata-on-myricom-capture-cards%2F&data=02%7C01%7Ccraig.edgmand%40okstate.edu%7Cd4444c14ab184963043408d8147cf441%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C0%7C637281874192328864&sdata=5KgANusNnHK7cijv1B0rUDomBZaosOK8AuD9mTfd2Dg%3D&reserved=0>
>
> guide to pin the cpus to the worker nodes and use pcap.buffer_size
> to increase the SNF dataring size, but no effect..
>
>
>
> We have one Myri card connected to p2p1 and two NUMA nodes, each
> with 8 cores (16 HT):
>
> NUMA node0 CPU(s): 0-7,16-23
> NUMA node1 CPU(s): 8-15,24-31
>
> OS: Centos 7
>
>
>
> Any help in the right direction would be appreciated! :)
>
>
>
> Thanks!
>
> Fatema
>
>
>
> Following is settings from suricata.yml file
>
>
>
> # Myricom support
>
> pcap:
>
> - interface: p2p1
>
> threads: 14
>
> buffer-size: 2gb
>
> checksum-checks: no
>
> pcap-file:
>
> checksum-checks: auto
>
>
>
> threading:
>
> set-cpu-affinity: yes
>
> cpu-affinity:
>
> - management-cpu-set:
>
> cpu: [ "0" ]
>
> mode: "balanced"
>
> prio:
>
> default: "low"
>
> - worker-cpu-set:
>
> cpu: [ "1-7","9-15" ]
>
> mode: "exclusive"
>
> prio:
>
> default: "high"
>
>
>
> Following is the currently recorded stats.log:
>
> ------------------------------------------------------------------------------------
>
> Date: 6/19/2020 -- 10:55:36 (uptime: 0d, 04h 04m 10s)
>
> ------------------------------------------------------------------------------------
>
> Counter | TM Name
> | Value
>
> ------------------------------------------------------------------------------------
>
> capture.kernel_packets | Total
> | 28447139411
>
> capture.kernel_drops | Total
> | 27910518132
>
> capture.kernel_ifdrops | Total
> | 6034
>
> decoder.pkts | Total
> | 536633135
>
>
>
>
>
> SNF parameters:
>
>
>
>
>
> SNF_APP_ID=32
>
> SNF_DATARING_SIZE=4096MB
>
> SNF_DESCRING_SIZE=1024MB
>
> SNF_NUM_RINGS=14
>
> SNF_FLAGS=0x1
>
>
>
> LD_PRELOAD="/opt/snf/lib/libpcap.so.1"
>
>
>
> OPTIONS="--user suricata --group suricata --pcap"
>
>
> _______________________________________________
> NOTE: this list will soon be closed. New topics should be brought to: https://forum.suricata.io
>
More information about the Oisf-users
mailing list