[Oisf-users] Drops: From none to gigantic in the blink of an eye

Thu Mar 24 22:14:10 UTC 2016

On 24-03-16 18:33, Cloherty, Sean E wrote:
> When I start up - the last line is as below.
> 
> 24/3/2016 -- 13:32:30 - <Notice> - This is Suricata version 3.0 RELEASE
> 24/3/2016 -- 13:32:30 - <Info> - CPUs/cores online: 32
> 24/3/2016 -- 13:32:30 - <Info> - 'default' server has 'request-body-minimal-inspect-size' set to 33882 and 'request-body-inspect-window' set to 4053 after randomization.
> 24/3/2016 -- 13:32:30 - <Info> - 'default' server has 'response-body-minimal-inspect-size' set to 42119 and 'response-body-inspect-window' set to 16872 after randomization.
> 24/3/2016 -- 13:32:30 - <Info> - DNS request flood protection level: 500
> 24/3/2016 -- 13:32:30 - <Info> - DNS per flow memcap (state-memcap): 524288
> 24/3/2016 -- 13:32:30 - <Info> - DNS global memcap: 16777216
> 24/3/2016 -- 13:32:30 - <Info> - Protocol detection and parser disabled for modbus protocol.
> 24/3/2016 -- 13:32:30 - <Info> - Found an MTU of 1500 for 'ens1f1'
> 24/3/2016 -- 13:32:30 - <Info> - allocated 3670016 bytes of memory for the defrag hash... 65536 buckets of size 56
> 24/3/2016 -- 13:32:30 - <Info> - preallocated 65535 defrag trackers of size 168
> 24/3/2016 -- 13:32:30 - <Info> - defrag memory usage: 14679896 bytes, maximum: 2147483648
> 24/3/2016 -- 13:32:30 - <Info> - AutoFP mode using default "Active Packets" flow load balancer

We should probably suppress this one, or make it conditional on the
actual runmode. It's not an indication you are running autofp. Just that
a certain config setting from the yaml was loaded/parsed.

Cheers,
Victor

> 
> -----Original Message-----
> From: Cooper F. Nelson [mailto:cnelson at ucsd.edu] 
> Sent: Thursday, March 24, 2016 13:30 PM
> To: Cloherty, Sean E <scloherty at mitre.org>; Victor Julien <lists at inliniac.net>; oisf-users at lists.openinfosecfoundation.org
> Subject: Re: [Oisf-users] Drops: From none to gigantic in the blink of an eye
> 
> It would be in the suricata.log file, the log entries look like this:
> 
>> [12105] 24/3/2016 -- 17:07:04 - (flow-manager.c:667) <Info> 
>> (FlowManager) -- Flow emergency mode over, back to normal... unsetting 
>> FLOW_EMERGENCY bit (ts.tv_sec: 1458839205, ts.tv_usec:62) flow_spare_q 
>> status(): 123% flows at the queue
> 
> Are you following this article, configuration-wise?
> 
> https://home.regit.org/2012/07/suricata-to-10gbps-and-beyond/
> 
> What makes you think you are running in "autofp" mode?  If you want, you can send your suricata.log file to me and I'll see if I can figure out what the problem is.
> 
> -Coop
> 
> On 3/24/2016 10:22 AM, Cloherty, Sean E wrote:
>> Grepping the stats.log file for 'emergency' came up blank.
>>
>> -----Original Message-----
>> From: Oisf-users 
>> [mailto:oisf-users-bounces at lists.openinfosecfoundation.org] On Behalf 
>> Of Victor Julien
>> Sent: Thursday, March 24, 2016 11:52 AM
>> To: oisf-users at lists.openinfosecfoundation.org
>> Subject: Re: [Oisf-users] Drops: From none to gigantic in the blink of 
>> an eye
>>
>> On 23-03-16 20:33, Cooper F. Nelson wrote:
>>> The thing about SYN floods is that don't generate that much traffic.
>>> They are designed to DOS hosts, not networks.  And it only takes a 
>>> few mbits of SYN packets from spoofed source addresses/ports to DOS a 
>>> suricata sensor, due to flow hashing/tracking.
>>>
>>> If it's a memory leak you should be able to see that by running top 
>>> and monitoring memory usage, which looks ok in your case.
>>>
>>> If it's an issue with the flow memory-cap I would think you be seeing 
>>> lots of messages in the suricata log about flow-emergency mode.
>>
>> Good point. The counters didn't show this, while they should have I think.
>>
>> Sean, did you get flow 'emergency' mode messages in your log or to stdout?
>>
>> Cheers,
>> Victor
>>
>>>
>>> -Coop
>>>
>>> On 3/23/2016 12:26 PM, Cloherty, Sean E wrote:
>>>> Thank you Cooper,
>>>>
>>>> I will give this a try.  Though I would assume that the SYN flood 
>>>> would still show up as increased network traffic on the interface.
>>>> This is a test machine, but I do have it integrated into our 
>>>> production Zabbix monitor so I can keep an eye on it.
>>>>
>>>> Does anyone think it might be a symptom of a memory leak?  Would it 
>>>> be worthwhile testing Victor's suggestion before trying the new RC 
>>>> that was released?
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: 
>>> http://suricata-ids.org/support/
>>> List: 
>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>> Suricata User Conference November 9-11 in Washington, DC: 
>>> http://oisfevents.net
>>>
>>
>>
>> --
>> ---------------------------------------------
>> Victor Julien
>> http://www.inliniac.net/
>> PGP: http://www.inliniac.net/victorjulien.asc
>> ---------------------------------------------
>>
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: 
>> http://suricata-ids.org/support/
>> List: 
>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> Suricata User Conference November 9-11 in Washington, DC: 
>> http://oisfevents.net _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: 
>> http://suricata-ids.org/support/
>> List: 
>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> Suricata User Conference November 9-11 in Washington, DC: 
>> http://oisfevents.net
>>
> 
> 
> --
> Cooper Nelson
> Network Security Analyst
> UCSD ITS Security Team
> cnelson at ucsd.edu x41042
> 

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------