[Oisf-users] Question Regarding Prune Statistics

[Rasmor, Zachary R]

Hello,

 

I am trying to make sense of the positive "pruned" values that I'm seeing in
my stats log. My understanding is that positive prune values are
undesirable; however my understanding was also that pruning occurs in
emergency mode after shortened timeouts have not stabilized memory usage.

 

Per
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricatayam
l :

 

"At the point the memcap will still be reached, despite prealloc, the
flow-engine goes into the emergency-mode. In this mode, the engine will make
use of shorter time-outs. It lets flows expire in a more aggressive manner
so there will be more space for new Flows. 
There are two options: emergency_recovery and prune_flows. The emergency
recovery is set on 30. This is the percentage of prealloc'd flows after
which the flow-engine will be back to normal (when 30 percent of the 10000
flows is completed).
If during the emergency-mode, the aggressive time-outs do not have the
desired result, this option is the final resort. It ends some flows even if
they have not reached their time-outs yet. The prune-flows option shows how
many flows there will be terminated at each time a new flow is set up."

 

I have pasted the final stats entry of a 30 min test below. As you can see,
I never entered emergency mode, however my positive prune statistics.

(One additional note about the passage above: the "prune-flows" setting
seems to have been deprecated as of v1.3.1) 

 

flow_mgr.closed_pruned    | FlowManagerThread         | 5016163

flow_mgr.new_pruned       | FlowManagerThread         | 1121133

flow_mgr.est_pruned       | FlowManagerThread         | 1885848

flow.memuse               | FlowManagerThread         | 1433762256

flow.spare                | FlowManagerThread         | 2097144

flow.emerg_mode_entered   | FlowManagerThread         | 0

flow.emerg_mode_over      | FlowManagerThread         | 0

 

This leads to a few questions:

 

1.       Are positive pruned values necessarily a bad thing? 

2.       Per the Suricata training class: "new_pruned means flow were
discarded before they were established" - what is the meaning of est_pruned
and closed_pruned?

3.       How is pruning occurring if we never enter emergency mode?

 

Thanks,

Zach

________________________

Zach Rasmor

Senior Software Engineer

Lockheed Martin CIRT

700 N Frederick Ave | Gaithersburg, MD 20879

Email: zachary.r.rasmor at lmco.com

Office: 301.240.6116

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20150409/7009b4ca/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 11767 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20150409/7009b4ca/attachment-0001.bin>