<div dir="ltr">Hi Michal<div><br></div><div>Thanks for your prompt reply. I tried with rx-flow-hash with sd, mtu 9000 etc but there is no visible change in the number of pkt_on_wrong_threads. </div><div><br></div><div>I was wondering whether it is something to do with the card driver . I may try it later</div><div><br></div><div>I also realized that this list is deprecated so should I post this on <a href="http://forum.suricata.io">forum.suricata.io</a> and take it from there</div><div><br></div><div>Thanks</div><div><br></div><div>Kashif</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jul 6, 2020 at 7:02 PM Michał Purzyński <<a href="mailto:michalpurzynski1@gmail.com">michalpurzynski1@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>There might be several reasons for this - like fragmented packets or protocols your card does not recognize. The fact that enabling autofp fixes it, makes me think it's the former. Autofp will re-hash packets after defragmenting.</div><div><br></div><div>Fragmented packets do not have port numbers, but only an indicator "hey I'm part of flow X" and they will be hashed in a wrong way. You want to avoid hashing port numbers and only use src + dst IP and maybe the protocol number, AKA the 3-tuple. Same goes for any packet broker if you have one in front of your cluster.</div><div><br></div><div>Here's my current working setup, systemd configures interfaces here but you should be able to find what you need.</div><div><br></div><div>Look at lines like</div><div><br></div><div> /usr/bin/ethtool -N enp8s0f0 rx-flow-hash tcp4 sd</div><div><br></div><div><br></div><div>[Unit]<br>Description=Configure monitor interfaces<br>Before=bro.service<br>Wants=network.target<br><br>[Service]<br>Type=oneshot<br><br>ExecStart=/usr/bin/ip link set enp8s0f0 promisc on arp off up<br><br># While the interface is still down<br>ExecStartPost=/usr/bin/ip link set enp8s0f0 mtu 9000<br>ExecStartPost=-/usr/bin/ethtool -G enp8s0f0 rx 512<br><br>ExecStartPost=-/usr/bin/ethtool -L enp8s0f0 combined 6<br><br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 rx off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 tx off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 tso off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 ufo off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 gso off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 gro off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 lro off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 rxhash on<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 ntuple on<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 sg off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 txvlan off<br>ExecStartPost=-/usr/bin/ethtool -K enp8s0f0 rxvlan off<br>ExecStartPost=-/usr/bin/ethtool -X enp8s0f0 hkey 6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A equal 6<br>ExecStartPost=-/usr/bin/ethtool -N enp8s0f0 rx-flow-hash tcp4 sd<br>ExecStartPost=-/usr/bin/ethtool -N enp8s0f0 rx-flow-hash tcp6 sd<br>ExecStartPost=-/usr/bin/ethtool -N enp8s0f0 rx-flow-hash udp4 sd<br>ExecStartPost=-/usr/bin/ethtool -N enp8s0f0 rx-flow-hash udp6 sd<br><br>ExecStartPost=-/usr/bin/ethtool -C enp8s0f0 adaptive-rx on rx-usecs 64<br><br>ExecStartPost=-/usr/bin/ethtool -G enp8s0f0 rx 512<br><br>ExecStartPost=-/usr/bin/ethtool -A enp8s0f0 rx off tx off<br><br>ExecStartPost=-/usr/bin/sysctl -w sys.net.ipv6.conf.enp8s0f0.disable_ipv6=1<br><br>ExecStop=/usr/bin/ip link set enp8s0f0 promisc off arp off down<br><br>RemainAfterExit=yes<br><br>SuccessExitStatus=1<br><br>[Install]<br>WantedBy=multi-user.target</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jul 6, 2020 at 10:06 AM mohammad kashif <<a href="mailto:kashif.alig@gmail.com" target="_blank">kashif.alig@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi<div><br></div><div>I am aware of <a href="https://redmine.openinfosecfoundation.org/issues/2725" target="_blank">https://redmine.openinfosecfoundation.org/issues/2725</a> and there seems to be a conclusion that cluster_qm with symetric hashing solved packet_on_wrong thread issue. Unformutaly this is not the case for my setup. </div><div>I am using two X710 10G cards on two numa nodes with two Intel 5218 CPU HT enabled.</div><div>It's going to be a production suricata setup and I am getting around 3-5 Gbps on one interface. I have enabled only around 6000 rules for testing.</div><div>The only way I don't get any pkt_on_wrong_thread if I use autofp but cpu usage goes on the top so I don't think it is sustainable.</div><div><br></div><div>I am testing with cluster_qm and symmetric hashing</div><div><br></div><div>My setup is</div><div><br></div><div>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">ethtool -i ens3f0</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">driver: i40e</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">version: 2.3.2-k</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">firmware-version: 7.10 0x800075df 19.5.12</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">expansion-rom-version:<span> </span></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">bus-info: 0000:3b:00.0</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">supports-statistics: yes</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">supports-test: yes</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">supports-eeprom-access: yes</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">supports-register-dump: yes</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">suricata from debian repo</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">suricata -V</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">This is Suricata version 4.1.2 RELEASE</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">kernel : </span><span style="font-variant-ligatures:no-common-ligatures">uname -r</span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">4.19.0-9-amd64</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">cat /sys/devices/system/node/node0/cpulist</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">cat /sys/devices/system/node/node1/cpulist</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">ethtool -L ens3f0 combined 32</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">ethtool -K ens3f0 rxhash on</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">ethtool -K ens3f0 ntuple on</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">same with ens4f0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">./set_irq_affinity 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46,48,50,52,54,56,58,60,62 ens3f0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">./set_irq_affinity 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39,41,43,45,47,49,51,53,55,57,59,61,63 ens4f0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">ethtool -X ens3f0 hkey 6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A equal 32</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">same with ens4f0</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">for both interfaces</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">for i in rx tx tso gso gro <span> </span>rxhash ntuple sg txvlan rxvlan ; do ethtool -K ens4f0 $i off ; echo $i ; done</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">for proto in tcp4 udp4 tcp6 udp6; do /sbin/ethtool -N ens4f0 rx-flow-hash $proto sdfn ; done</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">suricata --dump-config | grep af-packet</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet = (null)</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0 = interface</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.interface = ens3f0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.threads = 32</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.cluster-id = 99</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.cluster-type = cluster_qm</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.defrag = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.use-mmap = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.mmap-locked = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.tpacket-v3 = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.ring-size = 200000</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.0.block-size = 1048576</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1 = interface</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.interface = ens4f0</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.threads = 32</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.cluster-id = 98</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.cluster-type = cluster_qm</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.defrag = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.tpacket-v3 = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.use-mmap = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.mmap-locked = yes</span></p><p style="margin:0px;font:11px Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.ring-size = 200000</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">af-packet.1.block-size = 1048576</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">When I start suricata, pkt_on_wrong_thread is around 20 percent of capture.kernel_packets but gradually in a few hours it comes down to 1-2% but keeps increasing.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">I can see from pidstat that only even numbered cpu being used on ens3f0 and odd numbered on ens4fo as expected due to numa node architecture.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">mpstat shows all cpu being used but the usage is really low, 2-3%.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">I haven't enabled cpu_affinity in config files as I can not see load an issue here.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Even if i use a single interface, it is still showing pkt_on_wrong_thread.</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures">Any suggestion would be really appreciated as no config change is removing </span><span style="font-variant-ligatures:no-common-ligatures">pkt_on_wrong_thread.</span><span style="font-variant-ligatures:no-common-ligatures"> </span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"> </span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-stretch:normal;font-size:11px;line-height:normal;font-family:Menlo;color:rgb(0,0,0)"><span style="font-variant-ligatures:no-common-ligatures"><br></span></p></div><div>Regards</div><div><br></div><div>Kashif</div><div><br></div></div>
_______________________________________________<br>
NOTE: this list will soon be closed. New topics should be brought to: <a href="https://forum.suricata.io" rel="noreferrer" target="_blank">https://forum.suricata.io</a><br>
</blockquote></div>
</blockquote></div>