[Oisf-users] Recommended settings for flow manager/recycler for 1Gbps of traffic

[Peter Manev]

On Tue, May 12, 2015 at 11:37 PM, Brian Keefer <chort at effu.se> wrote:
> On May 12, 2015, at 1:44 PM, Peter Manev <petermanev at gmail.com> wrote:
>
>> On Tue, May 12, 2015 at 10:33 PM, Brian Keefer <chort at effu.se> wrote:
>>> I’m planning on doing some testing of EVE-flows logging, working up to implementing it on a box handling full 1Gbps of traffic. I’m currently using 14 worker threads. Are there rough recommendations for how many flow managers and how many flow recyclers to enable?
>>>
>>
>> Hi,
>>
>> There are also other considerations that should be taken into account
>> (not just the traffic) - you can have a look here:
>> http://pevma.blogspot.se/2014/08/suricata-flows-flow-managers-and-effect.html
>
> Hello Peter,
>
> Are the number of management threads increased dynamically, based on how many other threads are configured (i.e. if I configure 2 flow managers, does the number of total management threads increase to 4?), or does the number of (flow manager + flow recycler) threads have to be less than the number of management threads?

No they are not dynamically increased. Actually if you are using the
latest git and you set up for example -

flow:
  memcap: 64mb
  hash-size: 65536
  prealloc: 10000
  emergency-recovery: 30
  managers: 2 # default to one flow manager
  recyclers: 2 # default to one flow recycler thread

That will give you 6 management threads in total - 2 flow manager
threads, 2 flow recycler threads and you always have 2 additional by
default  - SCPerfWakeupThread thread and a SCPerfMgmtThread thread = 6
in total

example of the suricata.log output in verbose mode:
....
[11773] 12/5/2015 -- 22:22:46 - (flow-manager.c:726) <Info>
(FlowManagerThreadSpawn) -- using 2 flow manager threads
[11792] 12/5/2015 -- 22:22:46 - (tm-threads.c:1005) <Info>
(TmThreadSetupOptions) -- Setting prio 2 for "FlowManagerThread"
thread , thread id 11792
[11792] 12/5/2015 -- 22:22:46 - (tmqh-packetpool.c:322) <Info>
(PacketPoolInit) -- preallocated 65534 packets. Total memory 230417544
[11793] 12/5/2015 -- 22:22:46 - (tm-threads.c:1005) <Info>
(TmThreadSetupOptions) -- Setting prio 2 for "FlowManagerThread"
thread , thread id 11793
[11793] 12/5/2015 -- 22:22:47 - (tmqh-packetpool.c:322) <Info>
(PacketPoolInit) -- preallocated 65534 packets. Total memory 230417544
[11773] 12/5/2015 -- 22:22:47 - (flow-manager.c:884) <Info>
(FlowRecyclerThreadSpawn) -- using 2 flow recycler threads
[11794] 12/5/2015 -- 22:22:47 - (tm-threads.c:1005) <Info>
(TmThreadSetupOptions) -- Setting prio 2 for "FlowRecyclerThread"
thread , thread id 11794
[11795] 12/5/2015 -- 22:22:47 - (tm-threads.c:1005) <Info>
(TmThreadSetupOptions) -- Setting prio 2 for "FlowRecyclerThread"
thread , thread id 11795
[11796] 12/5/2015 -- 22:22:47 - (tm-threads.c:1005) <Info>
(TmThreadSetupOptions) -- Setting prio 2 for "SCPerfWakeupThread"
thread , thread id 11796
[11797] 12/5/2015 -- 22:22:47 - (tm-threads.c:1005) <Info>
(TmThreadSetupOptions) -- Setting prio 2 for "SCPerfMgmtThread" thread
, thread id 11797
[11773] 12/5/2015 -- 22:22:47 - (tm-threads.c:1997) <Notice>
(TmThreadWaitOnThreadInit) -- all 16 packet processing threads, 6
management threads initialized, engine started.
......


thanks


>
> --
> bk



-- 
Regards,
Peter Manev