[Oisf-users] number of alerts versus performance

[Yasha Zislin]

More info. It seems my threads process different amount of packets. It is not evenly distributed. Is there a setting somewhere for that in Suricata or in PFRING? It seems that thread with 100% cpu utilization changes from one to another over time. At that time I notice from stats.log that new busy thread is processing more packets.


________________________________
From: Peter Manev <petermanev at gmail.com>
Sent: Thursday, June 30, 2016 4:27 PM
To: Yasha Zislin
Cc: oisf-users at lists.openinfosecfoundation.org
Subject: Re: [Oisf-users] number of alerts versus performance

On Thu, 2016-06-30 at 15:54 +0000, Yasha Zislin wrote:
> Peter,
>
>
> I found one alert that was causing high alert count. After I've
> disabled it, count went down but packet loss is still around 20%.
>
>
> my stats.log does not contain anything useful such as flow emergency
> mode, or ssn memcap drop. The only thing that is off is kernel drops,
> and tcp reassembly gaps.
> From my understanding kernel drops have nothing to do with Suricata
> and point to OS problems.
>
>
> I do see one of the CPUs peak at 100% when packet loss increases. One
> thing to note. Two other CPUs are working on capturing traffic with
> high IRQs. My guess would be flow manager or detection engine.
>


You can see if you get more info from:
top -H -p `pidof suricata`
and
perf top -c cpu_number_here
example: perf top -c 0

> I dunno.
>
>
> Thanks
>
>
>
>
> ______________________________________________________________________
> From: Peter Manev <petermanev at gmail.com>
> Sent: Thursday, June 30, 2016 3:00 PM
> To: Yasha Zislin
> Cc: oisf-users at lists.openinfosecfoundation.org
> Subject: Re: [Oisf-users] number of alerts versus performance
>
> On Thu, 2016-06-30 at 14:41 +0000, Yasha Zislin wrote:
> > I have been trying to figure out a packet loss on one of my sensors
> > and I am puzzled.
> >
> > It is has 16 gigs of RAM, one quad core AMD CPU, and nic sees about
> 3
> > million packets per minute. Nothing special in my mind. I am using
> > PFRING 6.5.0 and Suricata 3.1.
> >
> > I get about 20% to 40% packet loss.  I have another identical server
> > which sees the same amount of traffic and maybe some of the same
> > traffic as well.
> >
> > I've been messing around with NIC settings, IRQs, PFRING settings,
> > Suricata settings trying to figure out why such a high packet loss.
> >
> >
> > I have just realized one big difference in these two sensors.
> > Problematic one gets 2k to 4k of alerts per minute which sounds
> huge.
> >
>
> Any particular sig that is alerting in excess ?
>
> > Second one gets like 80 alerts per minute. Both have the same
> > rulesets.
> >
> >
> > The difference of course is the home_net variable.
> >
> >
> > Can the fact that Suricata processes more rules due to HOME_NET
> > definition cause high performance strain on the server?
> >
>
> Yes HOME_NET size has effect on performance as well (among other
> things). For example -
> HOME_NET: "any"
> EXTERNAL_NET: "any"
> will certainly degrade your performance.
>
> >
> > If the packet does not match per HOME_NET, it will be discarded
> before
> > being processed in rules. Correct?
> >
> > Versus if packet passes HOME_NET check, it would have to go through
> > all of the rules, hence cause higher CPU utilization.
> >
> >
> > Thank you for the clarification.
> >
> >
> > _______________________________________________
> > Suricata IDS Users mailing list:
> oisf-users at openinfosecfoundation.org
> > Site: http://suricata-ids.org | Support:
[https://secure.gravatar.com/blavatar/b35fe77e09a7541f738f500f4db6b857?s=200&ts=1467306804]<http://suricata-ids.org/>

Suricata<http://suricata-ids.org/>
suricata-ids.org
Open Source IDS / IPS / NSM engine



> http://suricata-ids.org/support/
>
>
> Suricata
> suricata-ids.org
> Open Source IDS / IPS / NSM engine
>
>
> > List:
> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> > Suricata User Conference November 9-11 in Washington, DC:
> http://oisfevents.net
>
>

--
Regards,
Peter Manev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160630/2af8f4ef/attachment-0002.html>