[Oisf-devel] suricata & two-way traffic
Peter Manev
petermanev at gmail.com
Wed Jul 16 09:09:24 UTC 2014
> On 16 jul 2014, at 11:10, Mahnaz Talebi <mhnz.talebi at gmail.com> wrote:
>
> Hi & Thanks for your replays.
> I find that all tx interrupts handle by one cpu and smp_affinity only influence on receive packets by different cpus.
> I examine different cluster-type and runmode compositions but the results were not much different.
> I use this command for run suricata:
> suricata -c /etc/suricata/suricata.yaml --af-packet
>
So how does your af-packet section in suricata.yaml look like?
> I use igb driver, active RSS and use /proc/irq/irq#/smp_affinity for setting interrupt affinity.
>
> Each tx/rx peer has same interrupt number:
>
> cat /proc/interrupts | grep p115p3
> 116: 1 0 0 0 0 0 0 0 PCI-MSI-edge p115p3
> 117: 430880 0 0 0 0 0 0 0 PCI-MSI-edge p115p3-TxRx-0
> 118: 52 400648 0 0 0 0 0 0 PCI-MSI-edge p115p3-TxRx-1
> 119: 52 0 395915 0 0 0 0 0 PCI-MSI-edge p115p3-TxRx-2
> 120: 63 0 0 376400 0 0 0 0 PCI-MSI-edge p115p3-TxRx-3
> 121: 58 0 0 0 384137 0 0 0 PCI-MSI-edge p115p3-TxRx-4
> 122: 52 0 0 0 0 393522 0 0 PCI-MSI-edge p115p3-TxRx-5
> 123: 52 0 0 0 0 0 394861 0 PCI-MSI-edge p115p3-TxRx-6
> 124: 52 0 0 0 0 0 0 389069 PCI-MSI-edge p115p3-TxRx-7
>
> I set smp_affinity for two devices that peer together, but for sender device (p115p4) I have: (receiver device is p115p3)
> cat /proc/interrupts | grep p115p4
> 125: 1 0 0 0 0 0 0 0 PCI-MSI-edge p115p4
> 126: 1165 0 0 0 0 0 0 0 PCI-MSI-edge p115p4-TxRx-0
> 127: 51 1099 0 0 0 0 0 0 PCI-MSI-edge p115p4-TxRx-1
> 128: 51 0 1099 0 0 0 0 0 PCI-MSI-edge p115p4-TxRx-2
> 129: 53 0 0 1099 0 0 0 0 PCI-MSI-edge p115p4-TxRx-3
> 130: 57 0 0 0 1099 0 0 0 PCI-MSI-edge p115p4-TxRx-4
> 131: 51 0 0 0 0 1220633 0 0 PCI-MSI-edge p115p4-TxRx-5
> 132: 58 0 0 0 0 0 1099 0 PCI-MSI-edge p115p4-TxRx-6
> 133: 51 0 0 0 0 0 0 1099 PCI-MSI-edge p115p4-TxRx-7
>
>
> > Hello Mahnaz,
> >
> > have you tried changing the cluster_type to cluster_flow ?
> > I'm not sure this can help but maybe it worth to check
> >
> > best regards
> > vito
> >
> > On 07/15/2014 12:04 PM, Mahnaz Talebi wrote:
> >> Is there anyone who can help me to solve this problem?
>
> >What does your af-packet section in suricata.yaml look like?
> >Do you have affinity set up for both sniffing interfaces?
> >How do you start Suricata (command line)?
>
>
> >>
> >>
> >> On Tue, Jul 8, 2014 at 5:17 PM, Mahnaz Talebi <mhnz.talebi at gmail.com <mailto:mhnz.talebi at gmail.com>
> > wrote:
> >>
> >> Hi all,
> >>
> >>
> >> I am trying to evalute suricata's behavior, when sending traffic for two interface that peer together in af-packet mode. I use tcpreplay for sending traffic to these interfaces with rate 950Mbps.
> >> I use RSS & smp_affinity for distribute flows between cpus and use workers runmode and cluster-cpu as cluster-type in af-packet mode.
> >> when I send traffic for one of peered interfaces(p115p3), drop rate is 0%, and top -H report is :
> >>
> >> Cpu0 : 0.0%us, 20.1%sy, 12.2%ni, 54.6%id, 0.0%wa, 1.3%hi, 11.8%si, 0.0%st
> >> Cpu1 : 11.9%us, 18.0%sy, 0.0%ni, 52.2%id, 0.0%wa, 2.9%hi, 15.1%si, 0.0%st
> >> Cpu2 : 6.2%us, 16.7%sy, 0.0%ni, 20.7%id, 0.0%wa, 3.3%hi, 53.3%si, 0.0%st
> >> Cpu3 : 12.7%us, 18.0%sy, 0.0%ni, 57.6%id, 0.0%wa, 2.5%hi, 9.2%si, 0.0%st
> >> Cpu4 : 13.0%us, 20.6%sy, 0.0%ni, 51.3%id, 0.0%wa, 3.2%hi, 11.9%si, 0.0%st
> >> Cpu5 : 11.8%us, 19.3%sy, 0.0%ni, 51.4%id, 0.0%wa, 2.5%hi, 15.0%si, 0.0%st
> >> Cpu6 : 10.0%us, 15.3%sy, 0.0%ni, 57.7%id, 0.0%wa, 2.1%hi, 14.9%si, 0.0%st
> >> Cpu7 : 15.3%us, 27.8%sy, 0.0%ni, 40.9%id, 0.0%wa, 2.5%hi, 13.5%si, 0.0%st
> >> Mem: 20775960k total, 1003940k used, 19772020k free, 97688k buffers
> >> Swap: 5177340k total, 0k used, 5177340k free, 540524k cached
> >>
> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> >> 6783 root 18 -2 337m 61m 3376 R 43.2 0.3 0:09.74 AFPacketp115p38
> >> 6780 root 18 -2 337m 61m 3376 R 35.6 0.3 0:07.61 AFPacketp115p35
> >> 6779 root 18 -2 337m 61m 3376 R 32.2 0.3 0:06.96 AFPacketp115p34
> >> 6781 root 18 -2 337m 61m 3376 R 32.2 0.3 0:06.95 AFPacketp115p36
> >> 6777 root 20 0 337m 61m 3376 R 31.3 0.3 0:06.72 AFPacketp115p32
> >> 6776 root 22 2 337m 61m 3376 R 29.9 0.3 0:08.12 AFPacketp115p31
> >> 6782 root 18 -2 337m 61m 3376 R 26.6 0.3 0:05.67 AFPacketp115p37
> >> 6778 root 20 0 337m 61m 3376 R 24.3 0.3 0:05.21 AFPacketp115p33
> >> 6767 root 20 0 337m 61m 3376 S 0.7 0.3 0:00.07 Suricata-Main
> >> 6784 root 22 2 337m 61m 3376 S 0.7 0.3 0:00.12 FlowManagerThre
> >>
> >> but, when I send traffic to both interfaces, drop rate for each interface is almost 55% ! each interface has 8 threads.
> >> and top -H report is:
> >>
> >> Cpu0 : 1.0%us, 24.7%sy, 49.8%ni, 6.7%id, 0.0%wa, 2.0%hi, 15.7%si, 0.0%st
> >> Cpu1 : 50.7%us, 24.2%sy, 0.3%ni, 7.4%id, 0.0%wa, 2.0%hi, 15.4%si, 0.0%st
> >> Cpu2 : 43.0%us, 19.5%sy, 0.0%ni, 1.0%id, 0.0%wa, 1.7%hi, 34.9%si, 0.0%st
> >> Cpu3 : 59.4%us, 21.8%sy, 0.0%ni, 8.1%id, 0.0%wa, 1.7%hi, 9.1%si, 0.0%st
> >> Cpu4 : 56.3%us, 23.0%sy, 0.0%ni, 7.7%id, 0.0%wa, 1.7%hi, 11.3%si, 0.0%st
> >> Cpu5 : 53.7%us, 23.8%sy, 0.0%ni, 6.4%id, 0.0%wa, 1.7%hi, 14.4%si, 0.0%st
> >> Cpu6 : 52.3%us, 23.2%sy, 0.0%ni, 8.1%id, 0.0%wa, 2.0%hi, 14.4%si, 0.0%st
> >> Cpu7 : 54.5%us, 23.6%sy, 0.0%ni, 7.1%id, 0.0%wa, 2.0%hi, 12.8%si, 0.0%st
> >> Mem: 20775960k total, 1014884k used, 19761076k free, 97844k buffers
> >> Swap: 5177340k total, 0k used, 5177340k free, 541212k cached
> >>
> >> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> >> 6780 root 18 -2 337m 70m 3376 R 39.5 0.3 1:47.93 AFPacketp115p35
> >> 6771 root 18 -2 337m 70m 3376 R 39.2 0.3 0:09.97 AFPacketp115p44
> >> 6772 root 18 -2 337m 70m 3376 R 38.9 0.3 0:10.02 AFPacketp115p45
> >> 6783 root 18 -2 337m 70m 3376 R 38.9 0.3 2:11.36 AFPacketp115p38
> >> 6779 root 18 -2 337m 70m 3376 R 38.6 0.3 1:40.22 AFPacketp115p34
> >> 6773 root 18 -2 337m 70m 3376 R 38.2 0.3 0:09.65 AFPacketp115p46
> >> 6775 root 18 -2 337m 70m 3376 R 38.2 0.3 0:10.48 AFPacketp115p48
> >> 6781 root 18 -2 337m 70m 3376 R 38.2 0.3 1:39.22 AFPacketp115p36
> >> 6774 root 18 -2 337m 70m 3376 R 37.6 0.3 0:09.20 AFPacketp115p47
> >> 6782 root 18 -2 337m 70m 3376 R 37.2 0.3 1:22.00 AFPacketp115p37
> >> 6768 root 22 2 337m 70m 3376 R 36.2 0.3 0:09.99 AFPacketp115p41
> >> 6776 root 22 2 337m 70m 3376 R 36.2 0.3 1:33.93 AFPacketp115p31
> >> 6769 root 20 0 337m 70m 3376 R 35.9 0.3 0:09.28 AFPacketp115p42
> >> 6777 root 20 0 337m 70m 3376 R 35.9 0.3 1:36.01 AFPacketp115p32
> >> 6770 root 20 0 337m 70m 3376 R 30.9 0.3 0:07.85 AFPacketp115p43
> >> 6778 root 20 0 337m 70m 3376 R 30.6 0.3 1:17.34 AFPacketp115p33
> >>
> >> what is problem?!
> >>
> >>
> _______________________________________________
> Suricata IDS Devel mailing list: oisf-devel at openinfosecfoundation.org
> Site: http://suricata-ids.org | Participate: http://suricata-ids.org/participate/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
> Redmine: https://redmine.openinfosecfoundation.org/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20140716/ab883312/attachment-0002.html>
More information about the Oisf-devel
mailing list