<html>
<head>
<style><!--
.hmmessage P
{
margin:0px;
padding:0px
}
body.hmmessage
{
font-size: 12pt;
font-family:Calibri
}
--></style></head>
<body class='hmmessage'><div dir='ltr'><div><br></div><div><div class="rmTopSpacer rmTop" style="line-height: 19.9935989379883px; position: relative; padding: 5px 0px 0px; z-index: 0; margin-right: 0px; opacity: 0; color: rgb(68, 68, 68); font-family: 'Segoe UI', 'Segoe UI Web Regular', 'Segoe UI Symbol', 'Helvetica Neue', 'BBAlpha Sans', 'S60 Sans', Arial, sans-serif; font-size: 14px; background-color: rgb(255, 255, 255);"><div class="rmButtons" style="line-height: 19.9935989379883px; float: right; white-space: nowrap; margin-top: 8px; margin-right: 0px;"><span class="rmPrev" style="line-height: 19.9935989379883px; display: inline-block; vertical-align: middle; padding: 0px 3px;"><img src="https://col127.mail.live.com/ol/clear.gif" class="i_rm_p_d" style="line-height: 19.9935989379883px; width: 16px; height: 16px; overflow: hidden; background-image: url(https://a.gfx.ms/strip_metro7.png); background-attachment: scroll; background-color: transparent; background-position: -86px -74px; background-repeat: no-repeat;"></span><span class="rmNext" style="line-height: 19.9935989379883px; display: inline-block; vertical-align: middle; padding: 0px 3px;"><img src="https://col127.mail.live.com/ol/clear.gif" class="i_rm_n" style="line-height: 19.9935989379883px; width: 16px; height: 16px; overflow: hidden; background-image: url(https://a.gfx.ms/strip_metro7.png); background-attachment: scroll; background-color: transparent; background-position: -79px -160px; background-repeat: no-repeat;"></span><span class="rmFull " style="line-height: 19.9935989379883px; display: inline-block; vertical-align: middle; padding: 0px 0px 0px 6px;"><img src="https://col127.mail.live.com/ol/clear.gif" class="i_r_fv" style="line-height: 19.9935989379883px; width: 16px; height: 16px; overflow: hidden; background-image: url(https://a.gfx.ms/strip_metro7.png); background-attachment: scroll; background-color: transparent; background-position: -86px -57px; background-repeat: no-repeat;"></span><span class="rmClose " style="line-height: 19.9935989379883px; display: inline-block; vertical-align: middle; padding: 0px 0px 0px 6px;"><img src="https://col127.mail.live.com/ol/clear.gif" class="g_close" style="line-height: 19.9935989379883px; width: 11px; height: 11px; overflow: hidden; background-image: url(https://a.gfx.ms/strip_metro7.png); background-attachment: scroll; background-color: transparent; background-position: -86px -1px; background-repeat: no-repeat;"></span></div><h2 class="rmSubject" style="line-height: 29.9904003143311px; font-size: 21px; font-family: 'Segoe UI Light', 'Segoe UI Web Light', 'Segoe UI Web Regular', 'Segoe UI', 'Segoe UI Symbol', HelveticaNeue-Light, 'Helvetica Neue', Arial, sans-serif; font-weight: normal; margin: 0px -20px 13px 0px; padding-top: 2px;">Performance issue</h2></div><div id="readMessagePartControl1118f" class="c-ReadMessagePart ReadMsgContainer HasLayout ClearBoth HideShadows FullPart NoHistory Read RmIc" style="line-height: 19.9935989379883px; clear: both; border-top-width: 1px; border-top-style: solid; border-top-color: rgb(204, 204, 204); margin-bottom: 20px; position: relative; color: rgb(68, 68, 68); font-family: 'Segoe UI', 'Segoe UI Web Regular', 'Segoe UI Symbol', 'Helvetica Neue', 'BBAlpha Sans', 'S60 Sans', Arial, sans-serif; font-size: 14px; background-color: rgb(255, 255, 255);"><div class="c-ReadMessagePartBody" style="line-height: 19.9935989379883px;"><div class="readMsgBody" style="line-height: 19.9935989379883px; overflow-y: hidden; overflow-x: auto; padding: 16px 0px 0px;"><div id="bodyreadMessagePartBodyControl1120f" class="ExternalClass MsgBodyContainer" style="line-height: 21.2999992370605px; font-size: 15px; font-family: Calibri, sans-serif;"><div dir="ltr" style="line-height: 21.2999992370605px;"><div style="line-height: 21.2999992370605px;"><span style="line-height: 21.2999992370605px;">Hi,</span></div><div style="line-height: 21.2999992370605px;"><span style="line-height: 21.2999992370605px;"><br></span></div><div style="line-height: 21.2999992370605px;"><span style="line-height: 21.2999992370605px;">I am having a weird performance issue with Suricata.</span></div><div style="line-height: 21.2999992370605px;">I have Suricata 2.0.1 running on a beefy server (132gb of RAM, 40 Logical CPUs). It is monitoring two Span ports with mostly HTTP(S) traffic.</div><div style="line-height: 21.2999992370605px;">Each interface approximately has 10 million packets per second throughput. I am using PF_RING to reduce packet loss.</div><div style="line-height: 21.2999992370605px;">Suricata has been running great. I've tweaked all of the buffers to reduce packet loss to 0%.</div><div style="line-height: 21.2999992370605px;">Recently, I've noticed that number of alerts is way down from normal even with no packet loss. So I've tried restarting Suricata, and alerts went back to normal baseline.</div><div style="line-height: 21.2999992370605px;">I need to find out what is going on. Not sure where to look.</div><div style="line-height: 21.2999992370605px;"><br style="line-height: 21.2999992370605px;"></div><div style="line-height: 21.2999992370605px;">Couple of things about my setup:</div><div style="line-height: 21.2999992370605px;">- When Suricata starts, it is using 60 gb of RAM. I've noticed when alert count goes down, memory usage is at 105gb.</div><div style="line-height: 21.2999992370605px;">- After Suricata service restart, it runs for about a day until alert count decreases.</div><div style="line-height: 21.2999992370605px;">- All CPUs are kicking and at no stage does any single CPU gets to 100%.</div><div style="line-height: 21.2999992370605px;">- I have 20 detection threads per interface.</div><div style="line-height: 21.2999992370605px;">- I have 26k ruleset. I know it's big but since I got RAM, I've figured I should be ok.</div><div style="line-height: 21.2999992370605px;">- Here is my stream section of the config:</div><div style="line-height: 21.2999992370605px;"><div style="line-height: 21.2999992370605px;">stream:</div><div style="line-height: 21.2999992370605px;"> memcap: 60gb</div><div style="line-height: 21.2999992370605px;"> checksum-validation: no # reject wrong csums</div><div style="line-height: 21.2999992370605px;"> inline: no # auto will use inline mode in IPS mode, yes or no set it statically</div><div style="line-height: 21.2999992370605px;"> prealloc-sessions: 2000000</div><div style="line-height: 21.2999992370605px;"> midstream: false</div><div style="line-height: 21.2999992370605px;"> asyn-oneside: false</div><div style="line-height: 21.2999992370605px;"> reassembly:</div><div style="line-height: 21.2999992370605px;"> memcap: 90gb</div><div style="line-height: 21.2999992370605px;"> depth: 4mb # reassemble 1mb into a stream</div><div style="line-height: 21.2999992370605px;"> toserver-chunk-size: 2560</div><div style="line-height: 21.2999992370605px;"> toclient-chunk-size: 2560</div><div style="line-height: 21.2999992370605px;"> randomize-chunk-size: yes</div><div style="line-height: 21.2999992370605px;"> #randomize-chunk-range: 10</div><div style="line-height: 21.2999992370605px;"> #raw: yes</div><div style="line-height: 21.2999992370605px;"> chunk-prealloc: 3000000</div><div style="line-height: 21.2999992370605px;"> segments:</div><div style="line-height: 21.2999992370605px;"> - size: 4</div><div style="line-height: 21.2999992370605px;"> prealloc: 15000</div><div style="line-height: 21.2999992370605px;"> - size: 16</div><div style="line-height: 21.2999992370605px;"> prealloc: 200000</div><div style="line-height: 21.2999992370605px;"> - size: 112</div><div style="line-height: 21.2999992370605px;"> prealloc: 400000</div><div style="line-height: 21.2999992370605px;"> - size: 248</div><div style="line-height: 21.2999992370605px;"> prealloc: 300000</div><div style="line-height: 21.2999992370605px;"> - size: 512</div><div style="line-height: 21.2999992370605px;"> prealloc: 200000</div><div style="line-height: 21.2999992370605px;"> - size: 768</div><div style="line-height: 21.2999992370605px;"> prealloc: 100000</div><div style="line-height: 21.2999992370605px;"> - size: 1448</div><div style="line-height: 21.2999992370605px;"> prealloc: 1000000</div><div style="line-height: 21.2999992370605px;"> - size: 65535</div><div style="line-height: 21.2999992370605px;"> prealloc: 400000</div></div><div style="line-height: 21.2999992370605px;"><br style="line-height: 21.2999992370605px;"></div><div style="line-height: 21.2999992370605px;">Thank you.</div></div></div></div></div></div></div> </div></body>
</html>