[Oisf-users] Packets "dissappear" with PF_RING.

Thu Oct 16 16:49:34 UTC 2014

I think there is some issue at least in the way Suricata reports capture.kernel_packets (if not in something deeper!).

My real traffic hitting eth2+eth3 is about 2Gbps. I know it for sure since it’s a SPAN from a core router which I have perfectly controlled!

In my case, If using PF_RING, the aggregation of kernel_packets for each interface (RxPFReth*) is about 90Mbps, no matter how many threads or which nic driver using.
If using libpcap, the aggregation with 1 thread for each iface is about 2Gbps (which is the real traffic so I guess it’s right) but if using 2 threads for each iface, it reports up to 5Gbps (??), more than twice the real traffic.
It’s not reporting significant kernel_drops neither kernel_ifdrops in any case.

Couldn’t test with af_packet because Suricata dies with error “Coudn't set fanout mode, error Invalid argument”. I’m using kernel 3.10 and Suricata 2.1beta1

Someone is getting some kind of report from stats file and can confirm than the bytes ratio reported there is right for him?

Best regards,
Xavier Romero

De: Giuseppe Longo [mailto:giuseppelng at gmail.com]
Enviado el: dilluns, 13 d'octubre de 2014 7:43
Para: Xavier Romero
Asunto: RE: [Oisf-users] Packets "dissappear" with PF_RING.

Hi Xavier,
Could you tell me which PF_RING version are you using?
Driver? OS?

Thanks
Il 12/ott/2014 22:00 "Xavier Romero" <XRomero at nexica.com<mailto:XRomero at nexica.com>> ha scritto:
Hi Giuseppe,

It certainly stopped “Cluster Fragmentend Discard”, which remains to 0 now, and the number of packets processed by Suricata got higher, but not real yet. Now Suricata reports 150.000pkt/s which is way better than 40.000 but still the half of the real packets at the interface (300.000). Suricata does not reports any packet drop.

I’ve tried with pfcount (tool that comes with pfring to count packets) and it sees the right traffic, all the 300.000 pkt/s.

Any other idea?

Thank you very much,
Xavier Romero.

[root at nex-ids-01 suricata]# cat /proc/net/pf_ring/info
PF_RING Version          : 6.0.2 ($Revision: exportado$)
Total rings              : 16

Standard (non DNA) Options
Ring slots               : 65534
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Transparent mode         : Yes [mode 0]
Total plugins            : 0
Cluster Fragment Queue   : 21
Cluster Fragment Discard : 0

De: Giuseppe Longo [mailto:giuseppelng at gmail.com<mailto:giuseppelng at gmail.com>]
Enviado el: diumenge, 12 d'octubre de 2014 12:04
Para: Xavier Romero
Asunto: Re: [Oisf-users] Packets "dissappear" with PF_RING.

Hi Xavier,
Try to increase the ring slot:

rmmod pf_ring
modprobe pf_ring transparent_mode=0 min_num_slots=65534

Then adjust the packet size in suricata.yaml:

default-packet-size: 65534

Regards
Il 12/ott/2014 11:53 "Xavier Romero" <XRomero at nexica.com<mailto:XRomero at nexica.com>> ha scritto:
Hello,

I’ve switched from PCAP mode to PF_RING and now I realize that suricata is getting much less pakets, it is, instead of our usual 300.000 pkt/s to just 40.000 pkt/s (according to stats file). In fact alerts generated dropped drastically, my Kibana visualization can confirm it!

Stats file does not report any drop, so it may be related to PF_RING? OR the way I’ve configured Suricata for PF_RING?

My config:
runmode: workers
pfring:
  - interface: eth2
    threads: 8
    cluster-id: 94
    cluster-type: cluster_flow
  - interface: eth3
    threads: 8
    cluster-id: 94
    cluster-type: cluster_flow

(I’ve got 2 x 8-core CPU, 32GB RAM)

My previous config was default runmode with default pcap settings, it worked perfect, just that cpu was near 100% most time and I wanted it to scale a bit more ☺

PF_RING info:

[root at nex-ids-01 suricata]# cat /proc/net/pf_ring/info
PF_RING Version          : 6.0.2 ($Revision: 8014$)
Total rings              : 16

Standard (non DNA) Options
Ring slots               : 4096
Slot version             : 16
Capture TX               : Yes [RX+TX]
IP Defragment            : No
Socket Mode              : Standard
Transparent mode         : Yes [mode 0]
Total plugins            : 0
Cluster Fragment Queue   : 6819
Cluster Fragment Discard : 825409065

Cluster Fragment Discard could have something to do?

From files /sys/class/net/eth2/statistics/rx_packets and /sys/class/net/eth3/statistics/rx_packets I can confirm that packets are arriving to the Linux box at a ratio of about 300.000 pkt/s however suricata reports a ratio of about 40.000 pkt/s in its stats file, without drops. I have no idea in which point the packets are dissappearing.

Any clue?

Best regards,
Xavier Romero

_______________________________________________
Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org<mailto:oisf-users at openinfosecfoundation.org>
Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
Training now available: http://suricata-ids.org/training/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20141016/55a08941/attachment-0002.html>