[Oisf-devel] tcp.ssn_memcap_drop

[Anoop Saldanha]

You are running the lastest master?

Can you reset your git head to this commit and check how the engine
behaves with the same ruleset?

cc4e89fbe1477d47e50fd720127e7c28d0d512ba

On Mon, Sep 19, 2011 at 10:43 PM, Martin Holste <mcholste at gmail.com> wrote:
> Now I've reduced the ruleset to a single rule--my heartbeat sig, and
> it's still missing almost all the time.  It appears that there's a
> major bottleneck in the flow distributer somewhere if the system can't
> grep for a single stream on just 600 Mb/sec.  Anyone else running
> heartbeat sigs and seeing the same thing?
>
> On Mon, Sep 19, 2011 at 11:06 AM, Martin Holste <mcholste at gmail.com> wrote:
>> Ok, I'm giving that a shot, but so far that doesn't seem to have
>> improved things.  Right now, it looks like the system is missing a ton
>> of heartbeats, so it's definitely not detecting everything even though
>> all the drop counters are zero.  I'm running just 3k signatures on
>> about 600 Mb/sec of HTTP on 8 CPU/16 GB system.
>>
>> On Mon, Sep 19, 2011 at 10:48 AM, Victor Julien <victor at inliniac.net> wrote:
>>> On 09/19/2011 05:43 PM, Martin Holste wrote:
>>>> I've got memcap at 4GB and max_sessions is 256k by default.  I'm
>>>
>>> You may want to try setting it a bit lower than the 4GB max, like 3.5GB
>>> or so. I think I've seen at least one occasion where it didn't behave
>>> properly with the max setting. Something we need to look into still.
>>>
>>> Cheers,
>>> Victor
>>>
>>>> having better luck now with more drastic emergency flow pruning:
>>>>
>>>> flow:
>>>>   #memcap: 33554432
>>>>   memcap: 4294967295
>>>>   #hash_size: 65536
>>>>   hash_size: 268435456
>>>>   prealloc: 10000
>>>>   emergency_recovery: 40 #30
>>>>   prune_flows: 500 #5
>>>>
>>>> flow-timeouts:
>>>>   default:
>>>>     new: 1 # 30
>>>>     established: 10 #300
>>>>     closed: 0
>>>>     emergency_new: 1 #10
>>>>     emergency_established: 1 #100
>>>>     emergency_closed: 0
>>>>   tcp:
>>>>     new: 1 #60
>>>>     established: 10 #3600
>>>>     closed: 120
>>>>     emergency_new: 1 #10
>>>>     emergency_established: 1 #300
>>>>     emergency_closed: 20
>>>>   udp:
>>>>     new: 1 #30
>>>>     established: 1 #300
>>>>     emergency_new: 1 #10
>>>>     emergency_established: 1 #100
>>>>   icmp:
>>>>     new: 1 #30
>>>>     established: 1 #300
>>>>     emergency_new: 1 #10
>>>>     emergency_established: 1 #100
>>>>
>>>> I'm not yet sure how this will affect detection, but prior to this,
>>>> most new flows were being discarded.  This policy should favor new
>>>> flows at the expense of old flows, which for malware detection should
>>>> be desired.
>>>>
>>>> On Mon, Sep 19, 2011 at 10:30 AM, Anoop Saldanha <poonaatsoc at gmail.com> wrote:
>>>>> stream:
>>>>>  memcap: 33554432              # 32mb
>>>>>
>>>>> At the same time, you might also want to set max_sessions to something
>>>>> bigger.  We default to 256k.  You can try a bigger no and see how that
>>>>> works out
>>>>>
>>>>> On Mon, Sep 19, 2011 at 8:07 PM, Martin Holste <mcholste at gmail.com> wrote:
>>>>>> I'm seeing a ton of tcp.ssn_memcap_drop in my stats.log.  Which memcap
>>>>>> do I need to tweak to decrease these drops?  I've already set them all
>>>>>> to 4GB.
>>>>>> _______________________________________________
>>>>>> Oisf-devel mailing list
>>>>>> Oisf-devel at openinfosecfoundation.org
>>>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anoop Saldanha
>>>>>
>>>> _______________________________________________
>>>> Oisf-devel mailing list
>>>> Oisf-devel at openinfosecfoundation.org
>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>>
>>>
>>>
>>> --
>>> ---------------------------------------------
>>> Victor Julien
>>> http://www.inliniac.net/
>>> PGP: http://www.inliniac.net/victorjulien.asc
>>> ---------------------------------------------
>>>
>>> _______________________________________________
>>> Oisf-devel mailing list
>>> Oisf-devel at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>
>>
> _______________________________________________
> Oisf-devel mailing list
> Oisf-devel at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>



-- 
Anoop Saldanha