<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Lucida Console";
panose-1:2 11 6 9 4 5 4 2 2 4;}
@font-face
{font-family:"Franklin Gothic Book";
panose-1:2 11 5 3 2 1 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:228006896;
mso-list-type:hybrid;
mso-list-template-ids:1085574212 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal>Hello,<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>I am trying to make sense of the positive “pruned” values that I’m seeing in my stats log. My understanding is that positive prune values are undesirable; however my understanding was also that pruning occurs in emergency mode after shortened timeouts have not stabilized memory usage…<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Per <a href="https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml">https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayaml</a><span style='color:#1F497D'> :<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-left:.5in'><span style='font-size:9.0pt;font-family:"Verdana","sans-serif";color:#484848;background:white'>“At the point the memcap will still be reached, despite prealloc, the flow-engine goes into the emergency-mode. In this mode, the engine will make use of shorter time-outs. It lets flows expire in a more aggressive manner so there will be more space for new Flows.<span class=apple-converted-space> </span></span><span style='font-size:9.0pt;font-family:"Verdana","sans-serif";color:#484848'><br><span style='background:white'>There are two options: emergency_recovery and prune_flows. The emergency recovery is set on 30. This is the percentage of prealloc'd flows after which the flow-engine will be back to normal (when 30 percent of the 10000 flows is completed).</span><br><span style='background:white'>If during the emergency-mode, the aggressive time-outs do not have the desired result, this option is the final resort. It ends some flows even if they have not reached their time-outs yet. The prune-flows option shows how many flows there will be terminated at each time a new flow is set up.”<o:p></o:p></span></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Verdana","sans-serif";color:#484848;background:white'><o:p> </o:p></span></p><p class=MsoNormal><span style='background:white'>I have pasted the final stats entry of a 30 min test below. As you can see, I never entered emergency mode, however my positive prune statistics.<o:p></o:p></span></p><p class=MsoNormal><span style='background:white'>(One additional note about the passage above: the “prune-flows” setting seems to have been deprecated as of v1.3.1) <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Verdana","sans-serif";color:#484848;background:white'><o:p> </o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt;font-family:"Lucida Console"'>flow_mgr.closed_pruned | FlowManagerThread | 5016163<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt;font-family:"Lucida Console"'>flow_mgr.new_pruned | FlowManagerThread | 1121133<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt;font-family:"Lucida Console"'>flow_mgr.est_pruned | FlowManagerThread | 1885848<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt;font-family:"Lucida Console"'>flow.memuse | FlowManagerThread | 1433762256<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt;font-family:"Lucida Console"'>flow.spare | FlowManagerThread | 2097144<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt;font-family:"Lucida Console"'>flow.emerg_mode_entered | FlowManagerThread | 0<o:p></o:p></span></p><p class=MsoNormal style='text-autospace:none'><span style='font-size:12.0pt;font-family:"Lucida Console"'>flow.emerg_mode_over | FlowManagerThread | 0<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>This leads to a few questions:<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>1.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Are positive pruned values necessarily a bad thing? <o:p></o:p></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>2.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>Per the Suricata training class: “new_pruned means flow were discarded before they were established” – what is the meaning of est_pruned and closed_pruned?<o:p></o:p></p><p class=MsoListParagraph style='text-indent:-.25in;mso-list:l0 level1 lfo1'><![if !supportLists]><span style='mso-list:Ignore'>3.<span style='font:7.0pt "Times New Roman"'> </span></span><![endif]>How is pruning occurring if we never enter emergency mode?<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>Thanks,<o:p></o:p></p><p class=MsoNormal>Zach<o:p></o:p></p><p class=MsoNormal><b><span style='font-size:12.0pt;font-family:"Franklin Gothic Book","sans-serif"'>________________________<o:p></o:p></span></b></p><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Franklin Gothic Book","sans-serif"'>Zach Rasmor<o:p></o:p></span></b></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Franklin Gothic Book","sans-serif"'>Senior Software Engineer<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Franklin Gothic Book","sans-serif"'>Lockheed Martin CIRT<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Franklin Gothic Book","sans-serif"'>700 N Frederick Ave | Gaithersburg, MD 20879<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Franklin Gothic Book","sans-serif"'>Email: <a href="mailto:zachary.r.rasmor@lmco.com">zachary.r.rasmor@lmco.com</a><o:p></o:p></span></p><p class=MsoNormal><span style='font-size:9.0pt;font-family:"Franklin Gothic Book","sans-serif"'>Office: 301.240.6116<o:p></o:p></span></p></div></body></html>