[Oisf-users] Recommend version of 'file/libmagic' for suricata?

Fri Jul 19 18:24:49 UTC 2013

Cooper,

Probably the dns stuff.  If you are seeing a fair bit of drops and the
dns parser/logging isn't that important, I would suggest disabling it
for now.  If there's any way to still keep it enabled, it would be
nice as well.  Helps us with testing all the new stuff.

On Fri, Jul 19, 2013 at 11:37 PM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> 16 2.8ghz cpu (hyperthreaded).  48 gigs RAM.
>
> Setting the AF_PACKET buffer bigger seems to help a bit:
>
>  ring-size: 1048576
>
> This is what the load looks like with ~300k packets-per-second.
>
>> top - 18:06:27 up 2 days, 23:24,  5 users,  load average: 15.34, 15.21, 14.75
>> Tasks: 225 total,   1 running, 224 sleeping,   0 stopped,   0 zombie
>> %Cpu0  : 94.0 us,  0.3 sy,  1.7 ni,  0.3 id,  0.0 wa,  0.0 hi,  3.7 si,  0.0 st
>> %Cpu1  : 58.1 us,  2.3 sy, 15.6 ni, 19.9 id,  0.0 wa,  0.0 hi,  4.0 si,  0.0 st
>> %Cpu2  : 77.7 us,  0.7 sy,  3.7 ni, 12.3 id,  0.0 wa,  0.0 hi,  5.6 si,  0.0 st
>> %Cpu3  : 93.4 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  6.6 si,  0.0 st
>> %Cpu4  : 87.0 us,  0.7 sy,  0.0 ni,  0.7 id,  0.0 wa,  0.0 hi, 11.6 si,  0.0 st
>> %Cpu5  : 82.7 us,  0.7 sy,  0.7 ni,  6.6 id,  0.0 wa,  0.0 hi,  9.3 si,  0.0 st
>> %Cpu6  : 51.5 us,  1.7 sy, 11.6 ni, 30.9 id,  0.0 wa,  0.0 hi,  4.3 si,  0.0 st
>> %Cpu7  : 78.1 us,  0.3 sy,  0.7 ni, 12.0 id,  0.0 wa,  0.0 hi,  9.0 si,  0.0 st
>> %Cpu8  : 86.7 us,  0.7 sy,  0.3 ni,  6.0 id,  0.0 wa,  0.0 hi,  6.3 si,  0.0 st
>> %Cpu9  : 49.2 us,  3.0 sy, 16.3 ni, 27.6 id,  0.0 wa,  0.0 hi,  4.0 si,  0.0 st
>> %Cpu10 : 85.4 us,  0.0 sy,  1.0 ni,  7.0 id,  0.0 wa,  0.0 hi,  6.6 si,  0.0 st
>> %Cpu11 : 77.4 us,  0.3 sy,  3.7 ni, 11.3 id,  0.0 wa,  0.0 hi,  7.3 si,  0.0 st
>> %Cpu12 : 79.7 us,  0.3 sy,  3.3 ni, 10.0 id,  0.0 wa,  0.0 hi,  6.6 si,  0.0 st
>> %Cpu13 : 70.4 us,  0.7 sy,  3.0 ni, 17.3 id,  0.0 wa,  0.0 hi,  8.6 si,  0.0 st
>> %Cpu14 : 69.1 us,  1.3 sy,  3.7 ni, 18.9 id,  0.0 wa,  0.0 hi,  7.0 si,  0.0 st
>> %Cpu15 : 84.1 us,  0.7 sy,  0.0 ni,  7.3 id,  0.0 wa,  0.0 hi,  8.0 si,  0.0 st
>> KiB Mem:  49456860 total, 40197244 used,  9259616 free,   101824 buffers
>> KiB Swap:        0 total,        0 used,        0 free,  8821212 cached
>>
>>   PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
>> 17825 root      20   0 29.611g 0.027t 0.025t S  1351 59.46 569:19.95 /usr/bin/suricata +
>
>
> - -Coop
>
> On 7/18/2013 8:49 PM, Anoop Saldanha wrote:
>> Cooper,
>>
>> What's your current hardware spec?
>>
>> On Fri, Jul 19, 2013 at 2:05 AM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:
>> Hi Victor,
>>
>> I managed to run some tests again on my rebuilt sensor and against the
>> 2.0 suricata beta.
>>
>> In the process I went over all the suricata.yaml settings and ended up
>> setting much more aggressive flow timeouts.  I've run with this config
>> for a few hours with no segfaults or wedged processes.  The DNS parser
>> is enabled as well.  The only real issue is that my sensor peaks at
>> ~100% cpu utilization, so I need to look into getting some new hardware.
>>
>> -Coop
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>> OISF: http://www.openinfosecfoundation.org/
>>
>>
>>
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR6YBaAAoJEKIFRYQsa8FWF+gH/R01y1OfSikc1UDWExhXAZ9X
> sYEwAZgyF13MtdW1fmuqjDCjnMVCEEFEXL923irlTa4krJnNMFhBpFlQrOTjfm4I
> fv+OtJaZO7WH8H6DAWSaX80/OtNgGH2b0w8lYuMZQU9HvbOFEGaZkrmjRkLHLDMk
> /YteWSQ07B4qbZSpKrrqdr0c0jkL+eUemffCawWsMX8+vkFsJSghlwB1ALmpqDxV
> 3I+bNiYo+7GBIjShYsryDiMvFru35CX9+PrwiBp8NrSeIZjP5PzQ8L08tMXZRJEe
> a0oJYj/Wq3dTKXEu6g1a0NraIrGey+gGigGG6hUsjFF4ixlkalkTDKa7/1phjIA=
> =Jyc2
> -----END PGP SIGNATURE-----

-- 
-------------------------------
Anoop Saldanha
http://www.poona.me
-------------------------------