[Oisf-users] memcap drops etc

[Fernando Sclavo]

I was "fighting" a lot with memcap drops in the last days, and the trick
to solve them was lower and lower the flow timeouts (as suggested from
various members of the list). Now I'm stable in memory consumption.


On 12/06/2012 07:40 AM, Christophe Vandeplas wrote:
> On Thu, Dec 6, 2012 at 11:21 AM, Peter Manev <petermanev at gmail.com> wrote:
>> Hi,
>>
>> what (how much) traffic do you average?
> Hello Peter,
>
> That was written in my mail, one of the IDSses sees only 15Mbps during
> the day on average. Spikes up to 40Mbps (but very short spikes 4 times
> a day). That should certainly be feasible with such a system.
>
> Once I get that IDS working fine I'll finetune the settings of the
> others. (150 Mbps and 80 Mbps on average during the day)
>
>
>> On Thu, Dec 6, 2012 at 11:17 AM, Christophe Vandeplas
>> <christophe at vandeplas.com> wrote:
>>> Hello,
>>>
>>>
>>> Almost all my IDSses are having
>>> tcp.segment_memcap_drop
>>> tcp.reassembly_gap
>>>
>>> And some of them have
>>> tcp.ssn_memcap_drop
>>>
>>> I have been playing around with the memory settings in suricata, but I
>>> must admit it still looks very unclear to me, any help would really be
>>> appreciated.
>>>
>>> To attack this problem I'm now concentrating my efforts on the IDS
>>> dealing with the least traffic: during the day average of 15 Mbps.
>>> The IDS has 8 virtual-cores (4-core + ht = 8 ), and 8 GB of ram. And
>>> is sniffing using -i on a bond0 interface.
>>>
>>> The stats file is here: http://pastebin.com/kSVFDHRM
>>>
>>>
>>> Outputs that are on: fast, unified2, http, stats, syslog.
>>> I did not change anything in the threading section.
>>> Defrag is also default:
>>> defrag:
>>>   max-frags: 65535
>>>   prealloc: yes
>>>   timeout: 60
>>>
>>> Raised flow:
>>> flow:
>>>   memcap: 2gb
>>>   hash-size: 65536
>>>   prealloc: 10000
>>>   emergency-recovery: 30
>>>   prune-flows: 5
>>>
>>> Flow-timeouts are default, and I raised stream memcaps:
>>> stream:
>>>   memcap: 2gb
>>>   checksum-validation: yes      # reject wrong csums
>>>   inline: no                    # no inline mode
>>>   reassembly:
>>>     memcap: 1gb
>>>     depth: 8mb                  # reassemble 1mb into a stream
>>>     toserver-chunk-size: 2560
>>>     toclient-chunk-size: 2560
>>>
>>>
>>> Any advice to further finetune is welcome !
>>>
>>> Thanks a lot
>>> Christophe
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>> OISF: http://www.openinfosecfoundation.org/
>>
>>
>>
>> --
>> Regards,
>> Peter Manev
>>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/
>
>