<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div dir="ltr"></div><div dir="ltr">Can you compare the results with and without RPS?</div><div dir="ltr"><br>On Dec 7, 2018, at 4:49 AM, Nidhi V Singhai <<a href="mailto:nidhivar@gmail.com">nidhivar@gmail.com</a>> wrote:<br><br></div><blockquote type="cite"><div dir="ltr"><div dir="ltr"><div dir="ltr">Hello Peter<br><br><div class="gmail_quote"><div dir="ltr">On Fri, Dec 7, 2018 at 8:15 AM Peter Manev <<a href="mailto:petermanev@gmail.com">petermanev@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">On Thu, Dec 6, 2018 at 6:55 PM Nidhi V Singhai <<a href="mailto:nidhivar@gmail.com" target="_blank">nidhivar@gmail.com</a>> wrote:<br>
><br>
> Hello Victor<br>
><br>
> Apologies for the delay in response. I did not have access to my setup till now. Please find the requested data below<br>
><br>
> DUT -<br>
> Distro: CentOS<br>
> Kernel : 4.14.62-2.v7.x86_64<br>
> This is Intel I350 4-port Gigabit NIC and I am using 2 ports for my testing connected to two ports of the box running TRex.<br>
><br>
> This is a gen7 CPU and I have used RPS:<br>
> echo 4 > /sys/class/net/enp1s0f2/queues/rx-0/rps_cpus<br>
> echo 2 > /sys/class/net/enp1s0f3/queues/rx-0/rps_cpus<br>
><br>
<br>
Sorry it may be obvious but just wanted to confirm - Is this on the<br>
Suricata system or on the replay machine ?<br>
<br></blockquote><div>Yes, this is the DUT running suricata.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
> I have disabled NIC offloading as per the Septun doc, stopped irqbalance, and pinned interrupts to different cores. I can see that the right queues are being used for my packets in /proc/interrupts.<br>
><br>
> My AF-Packet config is below:<br>
> - interface: enp1s0f2<br>
> cluster-id: 99<br>
> cluster-type: cluster_flow<br>
> defrag: yes<br>
> use-mmap: yes<br>
> tpacket-v3: yes<br>
> ring-size: 400000<br>
> block-size: 393216<br>
><br>
> On Trex, currently I am using the configuration in the package (avl/sfr_delay_10_1g_asa_nat.yaml). I have modified this file to generate a traffic of 150Mbps by using only a couple of pcaps from this file and adjusting their cps. I have 255 clients and 10 servers.<br>
><br>
> Few stats from my last run. Please note that I do have other services running as well on the system which are using the CPU which would explain the kernel_drops.<br>
> capture.kernel_packets | Total | 75591077<br>
> capture.kernel_drops | Total | 4410661<br>
> tcp.pkt_on_wrong_thread | Total | 29509415<br>
><br>
<br>
Could you please upload a stats.log here please -<br>
<a href="https://redmine.openinfosecfoundation.org/issues/2725" rel="noreferrer" target="_blank">https://redmine.openinfosecfoundation.org/issues/2725</a> with your set<br>
up info?<br>
I am trying to trace down "wrong_thread" occurrences and to find the<br>
reason for it.<br>
<br></blockquote><div>I'll add the details to this link. I'll be able to do that next week once I have access to it again.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
> Kindly let me know if you need any more details.<br>
><br>
<br>
I think you may also consider increasing "max-pending-packets" in<br>
suricata.yaml if you haven't done it already.<br></blockquote><div><br></div><div>I haven't changed max-pending-packets. Will change this value and monitor the behavior.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<br>
> Many thanks<br>
> Nidhi.<br>
><br>
>><br>
>> ----------------------------------------------------------------------<br>
>><br>
>> Message: 1<br>
>> Date: Fri, 23 Nov 2018 16:14:40 +0100<br>
>> From: Victor Julien <<a href="mailto:lists@inliniac.net" target="_blank">lists@inliniac.net</a>><br>
>> To: <a href="mailto:oisf-users@lists.openinfosecfoundation.org" target="_blank">oisf-users@lists.openinfosecfoundation.org</a><br>
>> Subject: Re: [Oisf-users] Benchmark testing with Trex<br>
>> Message-ID: <<a href="mailto:e30943f3-1955-803f-1546-df9b3be31a05@inliniac.net" target="_blank">e30943f3-1955-803f-1546-df9b3be31a05@inliniac.net</a>><br>
>> Content-Type: text/plain; charset=utf-8<br>
>><br>
>> Hi Nidhi,<br>
>><br>
>> On 15-11-18 18:28, Nidhi V Singhai wrote:<br>
>> > I am collecting some suricata performance figures for my setup and am<br>
>> > using Trex for traffic replay. I am using standard trex profiles with<br>
>> > mostly http/https traffic currently. With suricata 4.1, I am observing<br>
>> > around 50% of the packets on wrong thread. I have a dual core system<br>
>> > with HT. I have disabled NIC offloading and set single rx/tx queues for<br>
>> > my interfaces. There are no capture drops registered. Has anyone else<br>
>> > tried suricata with trex? Can someone please suggest what might be the<br>
>> > cause of packets on wrong thread and how to avoid it?<br>
>><br>
>> At Suricon18 Joe's talk dealt with trex testing, however he did not test<br>
>> 4.1 yet. So the 'wrong thread' counters weren't yet available. He is<br>
>> planning to repeat the testing, so we maybe we will learn whether he<br>
>> will run into the same.<br>
>><br>
>> Can you give some more details on your setup? Distro, kernel, ethtool<br>
>> output, hardware, afpacket section of your yaml, etc?<br>
>><br>
>> Also things like trex setup would be useful.<br>
>><br>
>> Thanks!<br>
>> Victor<br>
><br>
</blockquote></div></div></div>
</div></blockquote><blockquote type="cite"><div dir="ltr"><span>_______________________________________________</span><br><span>Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a></span><br><span>Site: <a href="http://suricata-ids.org">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/">http://suricata-ids.org/support/</a></span><br><span>List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a></span><br><span></span><br><span>Conference: <a href="https://suricon.net">https://suricon.net</a></span><br><span>Trainings: <a href="https://suricata-ids.org/training/">https://suricata-ids.org/training/</a></span></div></blockquote></body></html>