[Oisf-users] Suricata with Myricom NIC using only one Worker Thread

[Fabian Franz]

Hi all, 


I am currently trying to get Suricata to work together with a Myricom card running a Sniffer10G driver. The problems I have seem to be somewhat similar to what Alexander Merck described on this list in Feb 2018 (https://lists.openinfosecfoundation.org/pipermail/oisf-users/2018-February/007790.html) but I could not find an answer to the problem in there and did not want to dig up such an old thread. 


I have installed the card and the driver on a Ubuntu 18.04 server with 64gigs of RAM and 16 cores (including HT). I followed the instructions here: https://blog.inliniac.net/2012/07/10/suricata-on-myricom-capture-cards/ and here: https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Myricomto build and run suricata. 

The card seems to be working and when viewing the debug output using one of the snf driver tools everything seems fine. However, no debug output is generated when setting the debug flag while running suricata. 

Now this wouldn't bother me too much if it wasn't for the stats.log file. This looks like the following:


capture.kernel_packets                        | W#01-ens5                 | 53827149
capture.kernel_packets                        | W#02-ens5                 | 10
capture.kernel_packets                        | W#03-ens5                 | 9
capture.kernel_packets                        | W#04-ens5                 | 0
capture.kernel_packets                        | W#05-ens5                 | 0
capture.kernel_packets                        | W#06-ens5                 | 10
capture.kernel_packets                        | W#07-ens5                 | 0
capture.kernel_packets                        | W#08-ens5                 | 18
capture.kernel_packets                        | W#09-ens5                 | 2
capture.kernel_packets                        | W#10-ens5                 | 2
capture.kernel_packets                        | W#11-ens5                 | 20
capture.kernel_packets                        | W#12-ens5                 | 4
capture.kernel_packets                        | W#13-ens5                 | 2
capture.kernel_packets                        | W#14-ens5                 | 3
capture.kernel_packets                        | W#15-ens5                 | 3
capture.kernel_packets                        | W#16-ens5                 | 4


Seemingly, only one worker thread is getting a considerable amount of packets while the others are more or less idle. This can also be confirmed when looking at the load of the single threads using htop. Surely this can't be right? Did I miss anything when setting up the driver and/or suricata? Is there a configuration flag or smiliar that I did not set?


The traffic I am currently seeing varies between 1 and 6Gbps. Especially when I am seeing more than 3 Gbps, the capture.kernel_drops counter also of W#01 rises pretty quickly to more than 10%.


I would be very grateful for any help or hints!

Best

FabFaeb 


P.S: 

My settings look like this:

myricom:

SNF_NUM_RINGS=16
SNF_FLAGS=0x1
SNF_DATARING_SIZE=34359738368
SNF_DESCRING_SIZE=8589934592


suricata.yaml:

pcap:
  - interface: ens5
    buffer-size: 2048mb
    checksum-checks: no
    threads: 16
  - interface: default

  cpu-affinity:
    - management-cpu-set:
        cpu: [ 0 ] 
    - receive-cpu-set:
        cpu: [ 0 ]  
    - worker-cpu-set:
        cpu: [ "1-15" ] 
        mode: "exclusive"
        prio:
          default: "high"


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190628/ce83a12f/attachment.html>