[Oisf-devel] 77b708=WIN?
Chris Wakelin
c.d.wakelin at reading.ac.uk
Tue Oct 4 17:04:59 UTC 2011
Just updated to latest git master. I've not seen this before:
> Cpu0 : 87.4%us, 7.6%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 4.6%si, 0.0%st
> Cpu1 : 8.8%us, 1.6%sy, 0.0%ni, 87.0%id, 0.0%wa, 0.0%hi, 2.6%si, 0.0%st
> Cpu2 : 10.5%us, 2.5%sy, 0.0%ni, 85.7%id, 0.0%wa, 0.0%hi, 1.3%si, 0.0%st
> Cpu3 : 12.2%us, 2.0%sy, 0.0%ni, 84.8%id, 0.0%wa, 0.0%hi, 1.0%si, 0.0%st
> Cpu4 : 96.7%us, 0.0%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 3.0%si, 0.0%st
> Cpu5 : 6.5%us, 2.6%sy, 0.0%ni, 90.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu6 : 17.7%us, 0.9%sy, 0.0%ni, 79.8%id, 0.0%wa, 0.0%hi, 1.6%si, 0.0%st
> Cpu7 : 1.9%us, 1.9%sy, 0.0%ni, 94.8%id, 1.0%wa, 0.0%hi, 0.3%si, 0.0%st
> Mem: 16465276k total, 10679036k used, 5786240k free, 25700k buffers
> Swap: 3906552k total, 25876k used, 3880676k free, 3440596k cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ P COMMAND
> 10407 mysql 20 0 343m 59m 3116 R 99.8 0.4 5:39.85 0 mysqld
> 18037 root 20 0 7400m 6.0g 49m R 99.8 38.3 46:35.11 4 Pfring4
> 18091 snort 20 0 263m 187m 3028 S 17.9 1.2 35:09.99 1 argus
> 18039 root 20 0 7400m 6.0g 49m S 15.2 38.3 14:17.66 6 Pfring6
> 18036 root 20 0 7400m 6.0g 49m R 13.3 38.3 10:03.05 3 Pfring3
> 18035 root 20 0 7400m 6.0g 49m S 11.3 38.3 12:38.72 2 Pfring2
> 18034 root 20 0 7400m 6.0g 49m S 7.3 38.3 8:31.14 1 Pfring1
> 18038 root 20 0 7400m 6.0g 49m S 6.0 38.3 6:58.98 5 Pfring5
> 18040 root 20 0 7400m 6.0g 49m S 4.0 38.3 2:04.13 2 FlowManagerThre
> 18090 snort 20 0 263m 187m 3028 S 0.7 1.2 0:13.65 5 argus
> 131 root 20 0 0 0 0 S 0.3 0.0 0:04.81 4 kondemand/4
> 10370 mysql 20 0 343m 59m 3116 S 0.3 0.4 2:37.97 3 mysqld
> 1585 operator 20 0 19460 1612 1040 R 0.3 0.0 1:20.70 7 top
One of the Pfring's is maxing out its CPU. I'm using
* Will's "single" pf_ring runmode, with CPU affinity set
* max-pending-packets : 2000
* detect-engine/profile: medium
* sgh-mpm-context: full
* mpm-algo: ac
* ~3500 rules
* ~50,000 packets per second
Killing suricata dumped core (7GB of it) :
> #0 SigMatchSignaturesBuildMatchArraySIMD (th_v=<value optimised out>, de_ctx=<value optimised out>, det_ctx=0xaf67cd20, p=0xbabbaa70) at detect.c:918
> 918 for (u = 0; u < det_ctx->sgh->sig_cnt; u += 64) {
> #0 SigMatchSignaturesBuildMatchArraySIMD (th_v=<value optimised out>, de_ctx=<value optimised out>, det_ctx=0xaf67cd20, p=0xbabbaa70) at detect.c:918
> #1 SigMatchSignaturesBuildMatchArray (th_v=<value optimised out>, de_ctx=<value optimised out>, det_ctx=0xaf67cd20, p=0xbabbaa70) at detect.c:1021
> #2 SigMatchSignatures (th_v=<value optimised out>, de_ctx=<value optimised out>, det_ctx=0xaf67cd20, p=0xbabbaa70) at detect.c:1462
> #3 0x0000000000428f2c in Detect (tv=0x0, p=<value optimised out>, data=0x1e4, pq=<value optimised out>, postpq=0x14) at detect.c:1857
> #4 0x000000000041f1fc in FlowForceReassemblyForQ (q=<value optimised out>) at flow-timeout.c:452
> #5 0x000000000041f313 in FlowForceReassembly () at flow-timeout.c:515
> #6 0x0000000000407b14 in main (argc=6, argv=<value optimised out>) at suricata.c:1565
but I got something similar at lunchtime (git master as of late Sunday I
think) when I tried increasing max-pending-packets to 5000.
It had been running fine yesterday though.
Best Wishes,
Chris
On 04/10/11 17:01, Martin Holste wrote:
> It grows at about 2-3 GB/day, and we are running some of the tagged
> signatures, though not that many. Recall again that I'm running about
> 7k rules with ac full turned on, so my ram baseline is 33 GB.
>
> On Tue, Oct 4, 2011 at 10:20 AM, Victor Julien <victor at inliniac.net> wrote:
>> On 10/04/2011 05:03 PM, Martin Holste wrote:
>>> Hasn't crashed yet, which is I think as long as it's ever gone. Ram
>>> is up to 40 GB, so I'm glad to hear you've found and fixed a memory
>>> leak. I've got 100 GB more ram before it dies, so I'll let it run for
>>> a few more days to see if the initial crashing issue is fixed.
>>
>> Thanks Martin. Whats your memory use growth rate?
>>
>> The memleak should only occur if sigs that use tags fire. There are
>> quite a few of those it seems from quickly grepping through
>> emerging-all.rules. We'll see :)
>>
>> Cheers,
>> Victor
>>
>> --
>> ---------------------------------------------
>> Victor Julien
>> http://www.inliniac.net/
>> PGP: http://www.inliniac.net/victorjulien.asc
>> ---------------------------------------------
>>
>>
> _______________________________________________
> Oisf-devel mailing list
> Oisf-devel at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
--
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin, c.d.wakelin at reading.ac.uk
IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
More information about the Oisf-devel
mailing list