[Oisf-users] unusual packet loss

[Peter Manev]

On Thu, Jan 7, 2016 at 2:10 PM, Yasha Zislin <coolyasha at hotmail.com> wrote:
> Peter,
>
> So I've found this article:
> https://home.regit.org/2013/11/using-linux-perf-tools-for-suricata-performance-analysis/
> Decided to give it a shot to see at which process exactly is the CPU
> saturation. I've recompiled suricata with that configure flag.
> After I've started the service on my problematic sensor, packet loss
> disappeared. Not sure what to think here. It is possible traffic might have

Interesting... can you reproduce that consistently? I mean if you do
the recompile switch with/without the flag the drops would
appear/disappear ?

Please have a look here for some  CFLAGS explanations -
https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/Optimize-Options.html


> changed but highly unlikely.
> BTW, perf top is still not showing suricata methods/functions even with that
> flag enabled.
>
> I dont recall but I think i did try suricata 3.0 with the same result.
>
> I will leave this sensor for now since it is working.
>
>> Subject: Re: [Oisf-users] unusual packet loss
>> From: petermanev at gmail.com
>> To: coolyasha at hotmail.com
>> CC: oisf-users at lists.openinfosecfoundation.org
>> Date: Sat, 2 Jan 2016 14:09:07 +0100
>
>>
>> On Thu, 2015-12-24 at 12:09 +0000, Yasha Zislin wrote:
>> > I have 4 threads running to monitor one interface. One of the threads
>> > is consuming 100% CPU and starts to have packet loss. Other 3 have
>> > zero packet loss.
>>
>> I was afraid that it is the "management" thread(s) that does this - but
>> it is not :)
>>
>> Which pf_ring version are you employing? (I had a similar case with an
>> older version - not sure if it is pf_ring though)
>>
>> Sometimes UDP load balancing on the NIC helps -
>> ethtool -N eth1 rx-flow-hash udp4 sdfn
>> ethtool -N eth1 rx-flow-hash udp6 sdfn
>>
>> Very curious if you experience the same issue with 3.0RC3 ?
>>
>> Thanks
>>
>> >
>> > > Date: Wed, 23 Dec 2015 22:36:44 +0100
>> > > Subject: Re: [Oisf-users] unusual packet loss
>> > > From: petermanev at gmail.com
>> > > To: coolyasha at hotmail.com
>> > > CC: oisf-users at lists.openinfosecfoundation.org
>> > >
>> > > On Wed, Dec 23, 2015 at 3:36 PM, Yasha Zislin
>> > <coolyasha at hotmail.com> wrote:
>> > > > I am running Suricata 2.1beta4 with PF_RING.
>> > > > I have 4 threads (4 logical CPUs) monitoring one interface. After
>> > a few
>> > > > minutes of running, I get 50% packet loss.
>> > > > I have tweaked all of the stream reassembly buffers to avoid
>> > packet loss.
>> > > > Only one of the threads gets kernel packet drops. I've noticed
>> > that one CPU
>> > > > is running at 100% and others are almost idle. Looking at
>> > stats.log, that
>> > > > one thread for some reason is digesting more packets than others.
>> > > > Throughput on this sensor is not that big. About 500k packets a
>> > minute. I
>> > > > use this image on other sensors without issues.
>> > > >
>> > > > Need help to figure out why only one thread is doing MOST of the
>> > work.
>> > >
>> > > Can you share "top -H" screenshot ?
>> > >
>> > > >
>> > > > Thank you.
>> > > >
>> > > > _______________________________________________
>> > > > Suricata IDS Users mailing list:
>> > oisf-users at openinfosecfoundation.org
>> > > > Site: http://suricata-ids.org | Support:
>> > http://suricata-ids.org/support/
>> > > > List:
>> > https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> > > > Suricata User Conference November 4 & 5 in Barcelona:
>> > http://oisfevents.net
>> > >
>> > >
>> > >
>> > > --
>> > > Regards,
>> > > Peter Manev
>> >
>>
>> --
>> Regards,
>> Peter Manev
>>



-- 
Regards,
Peter Manev