<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>I tried using top with "H". I have 4 threads running under suricata. One of them peaks at 100% while others are at 50% or so. On occasion different thread gets 100%. Packet loss occurs on the thread which has 100% utilization. I am not sure why these are
not spread out evenly.</p>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Oisf-users <oisf-users-bounces@lists.openinfosecfoundation.org> on behalf of Andreas Herz <andi@geekosphere.org><br>
<b>Sent:</b> Wednesday, July 6, 2016 8:12 PM<br>
<b>To:</b> oisf-users@lists.openinfosecfoundation.org<br>
<b>Subject:</b> Re: [Oisf-users] number of alerts versus performance</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">On 06/07/16 at 11:06, Yasha Zislin wrote:<br>
> Sure. Home nets are nothing special.<br>
> <br>
> 123.123.64.0/18,123.123.128.0/18<br>
<br>
Well that shouldn't be an issue unless there is a strange bug :)<br>
<br>
> <br>
> Is there a way to find out which process in Suricata is eating up the CPU? Maybe that's how I can trace it down to what is causing dramatic packet loss.<br>
<br>
You can use top, htop and perf top.<br>
In top use "H" for example<br>
<br>
> <br>
> Thank you all.<br>
> <br>
> <br>
> ________________________________<br>
> From: Oisf-users <oisf-users-bounces@lists.openinfosecfoundation.org> on behalf of Andreas Herz <andi@geekosphere.org><br>
> Sent: Tuesday, July 5, 2016 8:20 PM<br>
> To: oisf-users@lists.openinfosecfoundation.org<br>
> Subject: Re: [Oisf-users] number of alerts versus performance<br>
> <br>
> On 01/07/16 at 14:03, Yasha Zislin wrote:<br>
> > My guess would be that the packet loss is due to the traffic type and<br>
> > home_net definition. This sensor is in management zone so it has a lot<br>
> > of Active Directory, SSH, RDP, and other management traffic. I dont<br>
> > know if this can have a negative impact on performance.<br>
> <br>
> Can you share your HOME_NET definition (with example IPs)?<br>
> So far we can just guess :)<br>
> <br>
> ><br>
> > For example, I have another sensor which mostly processes HTTP/S and<br>
> > SQL/Oracle traffic with 50mil packets a minute and it only has couple<br>
> > of % of packet loss. Sam physical server with the same suricata config<br>
> > with the exception of HOME_NET<br>
> ><br>
> ><br>
> > ________________________________ From: Peter Manev<br>
> > <petermanev@gmail.com> Sent: Thursday, June 30, 2016 9:57 PM To: Yasha<br>
> > Zislin Cc: oisf-users@lists.openinfosecfoundation.org Subject: Re:<br>
> > [Oisf-users] number of alerts versus performance<br>
> ><br>
> > On Thu, 2016-06-30 at 17:14 +0000, Yasha Zislin wrote:<br>
> > > More info. It seems my threads process different amount of packets.<br>
> > > It is not evenly distributed. Is there a setting somewhere for that<br>
> > > in Suricata or in PFRING? It seems that thread with 100% cpu<br>
> > > utilization changes from one to another over time. At that time I<br>
> > > notice from stats.log that new busy thread is processing more<br>
> > > packets.<br>
> > ><br>
> ><br>
> > You mentioned earlier you were messing around with a number of diff<br>
> > settings - might be related. Did you use the irq affinity script (if<br>
> > you got an Intel nic)?<br>
> ><br>
> > ><br>
> > ><br>
> > ><br>
> > > ______________________________________________________________________<br>
> > > From: Peter Manev <petermanev@gmail.com> Sent: Thursday, June 30,<br>
> > > 2016 4:27 PM To: Yasha Zislin Cc:<br>
> > > oisf-users@lists.openinfosecfoundation.org Subject: Re: [Oisf-users]<br>
> > > number of alerts versus performance<br>
> > ><br>
> > > On Thu, 2016-06-30 at 15:54 +0000, Yasha Zislin wrote:<br>
> > > > Peter,<br>
> > > ><br>
> > > ><br>
> > > > I found one alert that was causing high alert count. After I've<br>
> > > > disabled it, count went down but packet loss is still around 20%.<br>
> > > ><br>
> > > ><br>
> > > > my stats.log does not contain anything useful such as flow<br>
> > > > emergency mode, or ssn memcap drop. The only thing that is off is<br>
> > > > kernel<br>
> > > drops,<br>
> > > > and tcp reassembly gaps. From my understanding kernel drops have<br>
> > > > nothing to do with Suricata and point to OS problems.<br>
> > > ><br>
> > > ><br>
> > > > I do see one of the CPUs peak at 100% when packet loss increases.<br>
> > > One<br>
> > > > thing to note. Two other CPUs are working on capturing traffic<br>
> > > > with high IRQs. My guess would be flow manager or detection<br>
> > > > engine.<br>
> > > ><br>
> > ><br>
> > ><br>
> > > You can see if you get more info from: top -H -p `pidof suricata`<br>
> > > and perf top -c cpu_number_here example: perf top -c 0<br>
> > ><br>
> > > > I dunno.<br>
> > > ><br>
> > > ><br>
> > > > Thanks<br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ><br>
> > > ______________________________________________________________________<br>
> > > > From: Peter Manev <petermanev@gmail.com> Sent: Thursday, June 30,<br>
> > > > 2016 3:00 PM To: Yasha Zislin Cc:<br>
> > > > oisf-users@lists.openinfosecfoundation.org Subject: Re:<br>
> > > > [Oisf-users] number of alerts versus performance<br>
> > > ><br>
> > > > On Thu, 2016-06-30 at 14:41 +0000, Yasha Zislin wrote:<br>
> > > > > I have been trying to figure out a packet loss on one of my<br>
> > > sensors<br>
> > > > > and I am puzzled.<br>
> > > > ><br>
> > > > > It is has 16 gigs of RAM, one quad core AMD CPU, and nic sees<br>
> > > about<br>
> > > > 3<br>
> > > > > million packets per minute. Nothing special in my mind. I am<br>
> > > > > using PFRING 6.5.0 and Suricata 3.1.<br>
> > > > ><br>
> > > > > I get about 20% to 40% packet loss. I have another identical<br>
> > > server<br>
> > > > > which sees the same amount of traffic and maybe some of the same<br>
> > > > > traffic as well.<br>
> > > > ><br>
> > > > > I've been messing around with NIC settings, IRQs, PFRING<br>
> > > > > settings, Suricata settings trying to figure out why such a high<br>
> > > > > packet<br>
> > > loss.<br>
> > > > ><br>
> > > > ><br>
> > > > > I have just realized one big difference in these two sensors.<br>
> > > > > Problematic one gets 2k to 4k of alerts per minute which sounds<br>
> > > > huge.<br>
> > > > ><br>
> > > ><br>
> > > > Any particular sig that is alerting in excess ?<br>
> > > ><br>
> > > > > Second one gets like 80 alerts per minute. Both have the same<br>
> > > > > rulesets.<br>
> > > > ><br>
> > > > ><br>
> > > > > The difference of course is the home_net variable.<br>
> > > > ><br>
> > > > ><br>
> > > > > Can the fact that Suricata processes more rules due to HOME_NET<br>
> > > > > definition cause high performance strain on the server?<br>
> > > > ><br>
> > > ><br>
> > > > Yes HOME_NET size has effect on performance as well (among other<br>
> > > > things). For example - HOME_NET: "any" EXTERNAL_NET: "any" will<br>
> > > > certainly degrade your performance.<br>
> > > ><br>
> > > > ><br>
> > > > > If the packet does not match per HOME_NET, it will be discarded<br>
> > > > before<br>
> > > > > being processed in rules. Correct?<br>
> > > > ><br>
> > > > > Versus if packet passes HOME_NET check, it would have to go<br>
> > > through<br>
> > > > > all of the rules, hence cause higher CPU utilization.<br>
> > > > ><br>
> > > > ><br>
> > > > > Thank you for the clarification.<br>
> > > > ><br>
> > > > ><br>
> > > > > _______________________________________________ Suricata IDS<br>
> > > > > Users mailing list:<br>
> > > > oisf-users@openinfosecfoundation.org<br>
> > > > > Site: <a href="http://suricata-ids.org" id="LPlnk361868">http://suricata-ids.org</a> | Support:
<div id="LPBorder_GT_14678535205840.5256487224056359" style="margin-bottom: 20px; overflow: auto; width: 100%; text-indent: 0px;">
<table id="LPContainer_14678535205770.7226143204980835" cellspacing="0" style="width: 90%; position: relative; overflow: auto; padding-top: 20px; padding-bottom: 20px; margin-top: 20px; border-top-width: 1px; border-top-style: dotted; border-top-color: rgb(200, 200, 200); border-bottom-width: 1px; border-bottom-style: dotted; border-bottom-color: rgb(200, 200, 200); background-color: rgb(255, 255, 255);">
<tbody>
<tr valign="top" style="border-spacing: 0px;">
<td id="ImageCell_14678535205790.5121617213128207" colspan="1" style="width: 250px; position: relative; display: table-cell; padding-right: 20px;">
<div id="LPImageContainer_14678535205790.9620894099956285" style="height: 200px; position: relative; margin: auto; display: table; width: 200px; background-color: rgb(255, 255, 255);">
<a id="LPImageAnchor_14678535205800.33377836625007107" href="http://suricata-ids.org/" target="_blank" style="display: table-cell; text-align: center;"><img aria-label="Preview image with link selected. Double-tap to open the link." id="LPThumbnailImageID_14678535205800.3832315164773279" width="200" height="200" style="display: inline-block; max-width: 250px; max-height: 250px; height: 200px; width: 200px; border-width: 0px; vertical-align: bottom;" src="https://secure.gravatar.com/blavatar/b35fe77e09a7541f738f500f4db6b857?s=200&ts=1467853530"></a></div>
</td>
<td id="TextCell_14678535205810.6703128389195803" colspan="2" style="vertical-align: top; position: relative; padding: 0px; display: table-cell;">
<div id="LPRemovePreviewContainer_14678535205810.11008761244463705"></div>
<div id="LPTitle_14678535205810.40301168489480643" style="top: 0px; color: rgb(0, 120, 215); font-weight: normal; font-size: 21px; font-family: wf_segoe-ui_light, "Segoe UI Light", "Segoe WP Light", "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; line-height: 21px;">
<a id="LPUrlAnchor_14678535205820.4732729558967077" href="http://suricata-ids.org/" target="_blank" style="text-decoration: none;">Suricata</a></div>
<div id="LPMetadata_14678535205820.8104446226479718" style="margin: 10px 0px 16px; color: rgb(102, 102, 102); font-weight: normal; font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; font-size: 14px; line-height: 14px;">
suricata-ids.org</div>
<div id="LPDescription_14678535205830.3424548089099788" style="display: block; color: rgb(102, 102, 102); font-weight: normal; font-family: wf_segoe-ui_normal, "Segoe UI", "Segoe WP", Tahoma, Arial, sans-serif; font-size: 14px; line-height: 20px; max-height: 100px; overflow: hidden;">
Open Source IDS / IPS / NSM engine</div>
</td>
</tr>
</tbody>
</table>
</div>
<br>
<br>
> [<a href="https://secure.gravatar.com/blavatar/b35fe77e09a7541f738f500f4db6b857?s=200&ts=1467803086]<http://suricata-ids.org/">https://secure.gravatar.com/blavatar/b35fe77e09a7541f738f500f4db6b857?s=200&ts=1467803086]<http://suricata-ids.org/</a>><br>
> <br>
> Suricata<<a href="http://suricata-ids.org/">http://suricata-ids.org/</a>><br>
> suricata-ids.org<br>
> Open Source IDS / IPS / NSM engine<br>
> <br>
> <br>
> <br>
> > [<a href="https://secure.gravatar.com/blavatar/b35fe77e09a7541f738f500f4db6b857?s=200&ts=1467381526]<http://suricata-ids.org/">https://secure.gravatar.com/blavatar/b35fe77e09a7541f738f500f4db6b857?s=200&ts=1467381526]<http://suricata-ids.org/</a>><br>
> ><br>
> > Suricata<<a href="http://suricata-ids.org/">http://suricata-ids.org/</a>> suricata-ids.org Open Source IDS /<br>
> > IPS / NSM engine<br>
> ><br>
> ><br>
> ><br>
> > ><br>
> > ><br>
> > > Suricata suricata-ids.org Open Source IDS / IPS / NSM engine<br>
> > ><br>
> > ><br>
> > > > <a href="http://suricata-ids.org/support/">http://suricata-ids.org/support/</a><br>
> > > ><br>
> > > ><br>
> > > > Suricata suricata-ids.org Open Source IDS / IPS / NSM engine<br>
> > > ><br>
> > > ><br>
> > > > > List:<br>
> > > > <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users">
https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
> > > > > Suricata User Conference November 9-11 in Washington, DC:<br>
> > > > <a href="http://oisfevents.net">http://oisfevents.net</a><br>
> > > ><br>
> > > ><br>
> > ><br>
> > ><br>
> ><br>
> > -- Regards, Peter Manev<br>
> ><br>
> <br>
> > _______________________________________________<br>
> > Suricata IDS Users mailing list: oisf-users@openinfosecfoundation.org<br>
> > Site: <a href="http://suricata-ids.org">http://suricata-ids.org</a> | Support:
<a href="http://suricata-ids.org/support/">http://suricata-ids.org/support/</a><br>
> > List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users">
https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
> > Suricata User Conference November 9-11 in Washington, DC: <a href="http://oisfevents.net">
http://oisfevents.net</a><br>
> <br>
> <br>
> --<br>
> Andreas Herz<br>
> _______________________________________________<br>
> Suricata IDS Users mailing list: oisf-users@openinfosecfoundation.org<br>
> Site: <a href="http://suricata-ids.org">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/">
http://suricata-ids.org/support/</a><br>
> List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users">
https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
> Suricata User Conference November 9-11 in Washington, DC: <a href="http://oisfevents.net">
http://oisfevents.net</a><br>
<br>
-- <br>
Andreas Herz<br>
_______________________________________________<br>
Suricata IDS Users mailing list: oisf-users@openinfosecfoundation.org<br>
Site: <a href="http://suricata-ids.org">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/">
http://suricata-ids.org/support/</a><br>
List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users">
https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
Suricata User Conference November 9-11 in Washington, DC: <a href="http://oisfevents.net">
http://oisfevents.net</a></div>
</span></font></div>
</div>
</body>
</html>