[Oisf-users] Suricata 2.0 dev, 100% cpu utilization in AF_PACKET + workers mode?

[Anoop Saldanha]

On Tue, Jul 9, 2013 at 9:33 AM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 7/8/2013 8:51 PM, Anoop Saldanha wrote:
>> Cooper,
>>
>> 1. Can you reproduce this with every run?
>
> Yes.
>
>> 2. Have you enabled the dns parser in the yaml?
>
> No.
>
>> 3. Are event rules present in your loaded ruleset?
>
> No.
>
>> 4. If (1) is true, can you locate the offending commit?
>
> Unfortunately, no.  I think it was during the week prior to June 21st,
> as I went on vacation that day and reverted back to the stable release
> to address the issue.
>
>> Possible to get a pcap(privately if you want) for this?
>
> That would be difficult.  It's a production 10Gb system and it happens
> regardless of traffic after a period of time.  I followed this guide
> almost exactly:
>
> https://home.regit.org/2012/07/suricata-to-10gbps-and-beyond/
>
> I think if you try the current 2.0 series dev branch in that
> configuration against live traffic you can reproduce the issue.
>
> It may be related to this issue, the symptoms are identical:
>
> http://comments.gmane.org/gmane.comp.security.ids.oisf.user/2438
>

Can you confirm this by sticking gdb and getting a bt for all the
threads, when the cpus seem to be stuck at 100%?

>> On Tue, Jul 9, 2013 at 5:49 AM, Leonard Jacobs <ljacobs at netsecuris.com> wrote:
>>> Ok then it is probably not the same problem.
>>>
>>> -----Original Message-----
>>> From: Cooper F. Nelson [mailto:cnelson at ucsd.edu]
>>> Sent: Monday, July 08, 2013 6:45 PM
>>> To: Leonard Jacobs
>>> Cc: oisf-users at openinfosecfoundation.org
>>> Subject: Re: [Oisf-users] Suricata 2.0 dev, 100% cpu utilization in AF_PACKET + workers mode?
>>>
>> I'm on the latest 3.8 series kernel.  The 3.9 series kernel did not work with the Intel ixgbe driver.
>>
>> -Coop
>>
>>>>> 3.8.12-gentoo #1 SMP Sat May 11 16:43:33 UTC 2013 x86_64 Intel(R)
>>>>> Xeon(R) CPU X5560 @ 2.80GHz GenuineIntel GNU/Linux
>>
>> On 7/8/2013 4:37 PM, Leonard Jacobs wrote:
>>>>> I believe that there is a known problem before kernel 3.5 in Linux.
>>>>>
>>>>> -----Original Message-----
>>>>> From: oisf-users-bounces at openinfosecfoundation.org
>>>>> [mailto:oisf-users-bounces at openinfosecfoundation.org] On Behalf Of
>>>>> Cooper F. Nelson
>>>>> Sent: Monday, July 08, 2013 5:08 PM
>>>>> To: oisf-users at openinfosecfoundation.org
>>>>> Subject: [Oisf-users] Suricata 2.0 dev, 100% cpu utilization in AF_PACKET + workers mode?
>>>>>
>>>>> See subject.  The current dev. release of suricata has an issue where threads will get stuck at 100% cpu utilization after running for some period of time and stop processing packets.  The process then needs to terminated via 'kill -9' and restarted to free up the cores.  This does not happen in the production release.
>>>>>
>>>>> Is this a known issue?  The dev. release otherwise performs much better in our environment.
>>>>>
>>>>> _______________________________________________
>>>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>>>> Site: http://suricata-ids.org | Support:
>>>>> http://suricata-ids.org/support/
>>>>> List:
>>>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>> OISF: http://www.openinfosecfoundation.org/
>>>>>
>>
>>>
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>> OISF: http://www.openinfosecfoundation.org/
>>
>>
>>
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR24uEAAoJEKIFRYQsa8FWEdsH/2HgAi2sLfU01qVv4z1R/LRK
> EtuvEjZmyM9NRVu6rYOu+7CNF8R5putjnomJ0tTY5MWpMDtwTNAwg2wMa4UhOy/J
> +Qz+EPsljV9OC90tG+rQvC3xKO61d5pXh33lB3QmClUZY26XVEP8nIPuG/NKhCqb
> mEdtT+3BEow712hbqmTlyJOwc3vBcFW2p3fwjgTBFCKFVuZKdwJaUwALHpqTQ26E
> RXZlqnvXzLRxQNgnAHAZa/164sXFgLkOevXdQ7+c+4NAZ4bkfNK0bsebm41vdXna
> ssCU8bfJJvW1QsplpetCZRYTAiv1/pIczeWWihI+lR4zOhyeXDwxXtg3vM/q/jc=
> =/cMz
> -----END PGP SIGNATURE-----



-- 
-------------------------------
Anoop Saldanha
http://www.poona.me
-------------------------------