[Oisf-users] pfring, cento and suricata

Mon Feb 26 16:06:04 UTC 2018

On 26-02-18 12:26, Kerry Milestone wrote:
> 
> # modprobe i40e_zc RSS=1,1,1,1 (and some other bits)
> # cento-ids -Z -i zc:p3p2 --balanced-egress-queues 2
> 26/Feb/2018 10:47:01 [cento.cpp:1669] Created interface zc:p3p2
> 26/Feb/2018 10:47:01 [cento.cpp:1837] Forwarding interface zc:p3p2
> balanced traffic to zc:10 at 0
> 26/Feb/2018 10:47:01 [cento.cpp:1837] Forwarding interface zc:p3p2
> balanced traffic to zc:10 at 1
> 
> 26/Feb/2018 10:47:51 [NetworkInterface.cpp:1215] [zc:p3p2] [776'945
> pps/5.13 Gbps][472'124/219'066/0/512'000 act/exp/drop/max
> flows][3'743/36'508'924 RX/TX pkt drops][0 TX pps][472'124 active
> flows][0/0/0.00 fps/pps/Gbps offload]
> 
> Seems pretty happy so far (I can attach other applications, scale up and
> down queues)
> 
> pfring:
>   - interface: zc:10 at 0
>     threads: 2
>   - interface: zc:10 at 1
>     threads: 2
> 
> #suricata -vv -c /etc/suricata/suri.yaml --pfring
> --pidfile=/var/run/suricata.pid
> 
> 26/2/2018 -- 10:47:11 - <Info> - ZC interface detected, not setting
> cluster-id for PF_RING (iface zc:10 at 0)
> 26/2/2018 -- 10:47:11 - <Info> - ZC interface detected, not setting
> cluster type for PF_RING (iface zc:10 at 0)
> 26/2/2018 -- 10:47:11 - <Info> - Going to use 2 thread(s)
> 26/2/2018 -- 10:47:11 - <Perf> - Enabling zero-copy for zc:10 at 0
> 26/2/2018 -- 10:47:11 - <Error> - [ERRCODE: SC_ERR_PF_RING_OPEN(34)] -
> Failed to open zc:10 at 0: pfring_open error. Check if zc:10 at 0 exists and
> pf_ring module is loaded.
> 26/2/2018 -- 10:47:11 - <Perf> - Enabling zero-copy for zc:10 at 0
> 26/2/2018 -- 10:47:12 - <Error> - [ERRCODE: SC_ERR_PF_RING_OPEN(34)] -
> Failed to open zc:10 at 0: pfring_open error. Check if zc:10 at 0 exists and
> pf_ring module is loaded.
> 26/2/2018 -- 10:47:12 - <Info> - ZC interface detected, not setting
> cluster-id for PF_RING (iface zc:10 at 1)
> 26/2/2018 -- 10:47:12 - <Info> - ZC interface detected, not setting
> cluster type for PF_RING (iface zc:10 at 1)
> 26/2/2018 -- 10:47:12 - <Info> - Going to use 2 thread(s)
> 26/2/2018 -- 10:47:12 - <Perf> - Enabling zero-copy for zc:10 at 1
> 26/2/2018 -- 10:47:13 - <Error> - [ERRCODE: SC_ERR_PF_RING_OPEN(34)] -
> Failed to open zc:10 at 1: pfring_open error. Check if zc:10 at 1 exists and
> pf_ring module is loaded.
> 26/2/2018 -- 10:47:13 - <Perf> - Enabling zero-copy for zc:10 at 1
> 
> 26/2/2018 -- 10:47:14 - <Info> - Running in live mode, activating unix
> socket
> 26/2/2018 -- 10:47:14 - <Info> - Using unix socket file
> '/var/run/suricata/suricata-command.socket'
> 26/2/2018 -- 10:47:14 - <Error> - [ERRCODE: SC_ERR_THREAD_INIT(49)] -
> thread "W#01-zc:10 at 0" failed to initialize: flags 0145
> 26/2/2018 -- 10:47:14 - <Error> - [ERRCODE: SC_ERR_INITIALIZATION(45)] -
> Engine initialization failed, aborting...
> 
> If I try on the CLI (disabled in config) , same issue:
> 
> # suricata -vv -c /etc/suricata/suri.yaml --pfring-int=zc:10 at 0
> --pfring-int=zc:10 at 1 --pidfile=/var/run/suricata.pid
> It sure enough starts
> 
> 26/2/2018 -- 11:02:31 - <Info> - Unable to find pfring config for
> interface zc:10 at 0, using default value or 1.0 configuration system.
> 26/2/2018 -- 11:02:31 - <Info> - Going to use 1 thread(s)
> 26/2/2018 -- 11:02:31 - <Perf> - Enabling zero-copy for zc:10 at 0
> 26/2/2018 -- 11:02:31 - <Info> - ZC interface detected, not adding
> thread to cluster
> 26/2/2018 -- 11:02:31 - <Perf> - (W#01-zc:10 at 0) Using PF_RING v.7.1.0,
> interface zc:10 at 0, cluster-id 1, single-pfring-thread
> 26/2/2018 -- 11:02:31 - <Info> - Unable to find pfring config for
> interface zc:10 at 1, using default value or 1.0 configuration system.
> 26/2/2018 -- 11:02:31 - <Info> - Going to use 1 thread(s)
> 26/2/2018 -- 11:02:31 - <Perf> - Enabling zero-copy for zc:10 at 1
> 26/2/2018 -- 11:02:31 - <Info> - ZC interface detected, not adding
> thread to cluster
> 26/2/2018 -- 11:02:31 - <Perf> - (W#01-zc:10 at 1) Using PF_RING v.7.1.0,
> interface zc:10 at 1, cluster-id 1, single-pfring-thread
> 26/2/2018 -- 11:02:31 - <Info> - RunModeIdsPfringWorkers initialised
> 
> 
> 
> 
> If I try with the config:
> 
> pfring:
>   - interface: zc:p3p2
>     threads: 2
> 
> # suricata -vv -c /etc/suricata/suri.yaml --pfring-int=zc:p3p2
> --pidfile=/var/run/suricata.pid
> 
> 26/2/2018 -- 11:06:36 - <Info> - ZC interface detected, not setting
> cluster-id for PF_RING (iface zc:p3p2)
> 26/2/2018 -- 11:06:36 - <Info> - ZC interface detected, not setting
> cluster type for PF_RING (iface zc:p3p2)
> 26/2/2018 -- 11:06:36 - <Info> - Going to use 2 thread(s)
> 26/2/2018 -- 11:06:36 - <Perf> - Enabling zero-copy for zc:p3p2
> 26/2/2018 -- 11:06:37 - <Info> - ZC interface detected, not adding
> thread to cluster
> 26/2/2018 -- 11:06:37 - <Perf> - (W#01-zc:p3p2) Using PF_RING v.7.1.0,
> interface zc:p3p2, cluster-id 1
> 26/2/2018 -- 11:06:37 - <Perf> - Enabling zero-copy for zc:p3p2
> 26/2/2018 -- 11:06:38 - <Info> - ZC interface detected, not adding
> thread to cluster
> 26/2/2018 -- 11:06:38 - <Perf> - (W#02-zc:p3p2) Using PF_RING v.7.1.0,
> interface zc:p3p2, cluster-id 1
> 26/2/2018 -- 11:06:38 - <Info> - RunModeIdsPfringWorkers initialised
> 26/2/2018 -- 11:06:38 - <Info> - Running in live mode, activating unix
> socket
> 26/2/2018 -- 11:06:38 - <Info> - Using unix socket file
> '/var/run/suricata/suricata-command.socket'
> 26/2/2018 -- 11:06:38 - <Notice> - all 2 packet processing threads, 4
> management threads initialized, engine started.
> 26/2/2018 -- 11:06:38 - <Error> - [ERRCODE: SC_ERR_PF_RING_OPEN(34)] -
> pfring_enable_ring failed returned -1
> 26/2/2018 -- 11:06:38 - <Error> - [ERRCODE: SC_ERR_PF_RING_OPEN(34)] -
> pfring_enable_ring failed returned -1
> 26/2/2018 -- 11:06:38 - <Error> - [ERRCODE: SC_ERR_FATAL(171)] - thread
> W#01-zc:p3p2 failed
> 
> I call it on the interface directly, I get:
> 26/2/2018 -- 11:09:48 - <Info> - Going to use 1 thread(s)
> 26/2/2018 -- 11:09:48 - <Perf> - Enabling zero-copy for p3p2
> 26/2/2018 -- 11:09:48 - <Perf> - (W#01-p3p2) Using PF_RING v.7.1.0,
> interface p3p2, cluster-id 1, single-pfring-thread
> 26/2/2018 -- 11:09:48 - <Info> - RunModeIdsPfringWorkers initialised
> 26/2/2018 -- 11:09:48 - <Info> - Running in live mode, activating unix
> socket
> 26/2/2018 -- 11:09:48 - <Info> - Using unix socket file
> '/var/run/suricata/suricata-command.socket'
> 26/2/2018 -- 11:09:48 - <Notice> - all 1 packet processing threads, 4
> management threads initialized, engine started.
> 26/2/2018 -- 11:11:01 - <Notice> - Signal Received.  Stopping engine.
> 26/2/2018 -- 11:11:01 - <Perf> - 0 new flows, 0 established flows were
> timed out, 0 flows in closed state
> 26/2/2018 -- 11:11:01 - <Info> - time elapsed 73.626s
> 26/2/2018 -- 11:11:01 - <Perf> - 0 flows processed
> 26/2/2018 -- 11:11:01 - <Perf> - (W#01-p3p2) Kernel: Packets 0, dropped 0
> 26/2/2018 -- 11:11:01 - <Perf> - (W#01-p3p2) Packets 0, bytes 0
> 26/2/2018 -- 11:11:01 - <Info> - (W#01-p3p2) Files logged: 0
> 26/2/2018 -- 11:11:01 - <Info> - Alerts: 0
> 26/2/2018 -- 11:11:01 - <Perf> - ippair memory usage: 398144 bytes,
> maximum: 16777216
> 26/2/2018 -- 11:11:02 - <Perf> - host memory usage: 14640336 bytes,
> maximum: 67108864
> 26/2/2018 -- 11:11:02 - <Info> - cleaning up signature grouping
> structure... complete
> 26/2/2018 -- 11:11:02 - <Notice> - Stats for 'p3p2':  pkts: 0, drop: 0
> (-nan%), invalid chksum: 0
> 26/2/2018 -- 11:11:02 - <Perf> - Cleaning up Hyperscan global scratch
> 26/2/2018 -- 11:11:02 - <Perf> - Clearing Hyperscan database cache
> 
> I've tried several other ways too, but would like to hear some advice on
> how to run suricata, in pfring with zero copy and multiple threads on a
> cluster ring rather than an interface.  Box is fairly standard Centos7,
> (suricata compiled with associated libraries).

I have no experience with Cento, but as it does not appear to be open
source I would suggest asking the ntop folks for support.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------