[Oisf-devel] suricata testing
Gurvinder Singh
gurvindersinghdahiya at gmail.com
Fri May 14 15:01:54 UTC 2010
Thanks rmkml for the interesting numbers. I just wonder if there were
any ICMP packets with the UDP traffic or not. As there is a known issue
on todo list to fix the slowdown when ICMP and UDP are together in the
traffic. If possible can you also test the engine with TCP traffic or
just UDP traffic, you can have ICMP with TCP, as ICMP handling with TCP
traffic is fine.
Cheers,
Gurvinder
rmkml wrote:
> Thx Victor and Will for reply,
> Im reply for victor question: no, 1) test it's ~150Mbit udp, then 3) test it's ~1Gbit udp...
>
> and for Will question, I have created new test 4) with emerging-threat rules (thx all and matt) at 12.668/18.496.000octet/148MBit (15% sending possibility):
> {today downloaded+unzip http://www.emergingthreats.net/rules/emerging-all.rules.zip and use on suricata engine without modification}
> stats.log output:
> decoder.pkts | Decode1 | 4946191
> decoder.pkts_per_sec | Decode1 | 19003.333333
> decoder.bytes | Decode1 | 7201654096
> decoder.bytes_per_sec | Decode1 | 27668853.333333
> decoder.mbit_per_sec | Decode1 | 221.350827
> decoder.ipv4 | Decode1 | 4946191
> decoder.ethernet | Decode1 | 4946191
> decoder.udp | Decode1 | 4946191
> decoder.avg_pkt_size | Decode1 | 1456.000000
> decoder.max_pkt_size | Decode1 | 1456
> ...
> and top output:
> top - 16:21:44 up 1 day, 23:45, 5 users, load average: 9.90, 8.33, 6.15
> Tasks: 241 total, 1 running, 240 sleeping, 0 stopped, 0 zombie
> Cpu0 : 29.7%us, 29.7%sy, 23.8%ni, 16.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu1 : 48.5%us, 4.0%sy, 0.0%ni, 47.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu2 : 49.0%us, 2.9%sy, 0.0%ni, 48.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu3 : 49.0%us, 3.9%sy, 0.0%ni, 47.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu4 : 49.5%us, 3.0%sy, 0.0%ni, 47.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu5 : 48.0%us, 2.9%sy, 0.0%ni, 49.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu6 : 52.0%us, 2.0%sy, 0.0%ni, 46.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu7 : 52.5%us, 4.0%sy, 0.0%ni, 43.6%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu8 : 57.8%us, 2.0%sy, 0.0%ni, 40.2%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu9 : 47.5%us, 3.0%sy, 0.0%ni, 49.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu10 : 49.5%us, 4.0%sy, 0.0%ni, 46.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu11 : 49.5%us, 3.0%sy, 0.0%ni, 47.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu12 : 47.5%us, 3.0%sy, 0.0%ni, 49.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu13 : 49.5%us, 2.0%sy, 0.0%ni, 48.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu14 : 49.0%us, 3.0%sy, 0.0%ni, 48.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu15 : 37.6%us, 2.0%sy, 0.0%ni, 33.7%id, 0.0%wa, 0.0%hi, 26.7%si, 0.0%st
> Mem: 12464792k total, 12001688k used, 463104k free, 174304k buffers
> Swap: 10482404k total, 2144k used, 10480260k free, 9287904k cached
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 28321 root 15 0 380m 130m 1348 S 876.4 1.1 68:03.01 suricata
> Regards
> Rmkml
>
>
> On Fri, 14 May 2010, Victor Julien wrote:
>
>
>> rmkml wrote:
>>
>>> Hi SDT (Suricata Devel Team),
>>> Im start playing with 16 core server for suricata (v0.9.1pre git12may).
>>> Im test with sp*rent test center gig and udp only src_port=dst_port=1024, size 1460 (zero filled) at this time on IDS mode.
>>> system is rhelv5.5i386 without pfring, but in this test, it's not a pb for me.
>>> network card is internal Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v2.0.2 (Aug 21, 2009).
>>>
>>> 1) with all my personal signatures (+old community rules)
>>> -- 7565 signatures processed. 8 are IP-only rules, 6567 are inspecting packet payload, 1490 inspect application layer
>>> result: 1597% cpu (16core), udp frame rate sending by sp*rent 12.668/18.496.000octet/148MBit (15% sending possibility)
>>> suricata stats.log file:
>>> decoder.pkts | Decode1 | 4076896
>>> decoder.pkts_per_sec | Decode1 | 15044.714286
>>> decoder.bytes | Decode1 | 5935960576
>>> decoder.bytes_per_sec | Decode1 | 21905104.000000
>>> decoder.mbit_per_sec | Decode1 | 175.240832
>>> decoder.ipv4 | Decode1 | 4076896
>>> decoder.ethernet | Decode1 | 4076896
>>> decoder.udp | Decode1 | 4076896
>>> decoder.avg_pkt_size | Decode1 | 1456.000000
>>> decoder.max_pkt_size | Decode1 | 1456
>>> (removed all field contains 0)
>>> top output:
>>> top - 14:42:59 up 1 day, 22:06, 4 users, load average: 16.36, 13.77, 9.79
>>> Tasks: 236 total, 2 running, 234 sleeping, 0 stopped, 0 zombie
>>> Cpu0 : 6.9%us, 13.9%sy, 79.2%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu4 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu6 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu7 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu8 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu9 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu10 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu11 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu15 : 95.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 5.0%si, 0.0%st
>>> Mem: 12464792k total, 12057272k used, 407520k free, 170072k buffers
>>> Swap: 10482404k total, 2144k used, 10480260k free, 9272564k cached
>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>> 8059 root 15 0 504m 253m 1348 S 1595.9 2.1 119:10.11 suricata
>>>
>>> 2) same test without signature on suricata:
>>> top output:
>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>> 10660 root 15 0 280m 31m 1312 S 88.3 0.3 0:30.11 suricata
>>>
>>> 3) suricata without signature receiving 1Gbit rate:
>>> top output:
>>> top - 14:55:16 up 1 day, 22:18, 4 users, load average: 4.53, 5.69, 7.99
>>> Tasks: 236 total, 1 running, 235 sleeping, 0 stopped, 0 zombie
>>> Cpu0 : 29.4%us, 60.8%sy, 0.0%ni, 9.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu1 : 1.0%us, 12.9%sy, 0.0%ni, 86.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu2 : 0.0%us, 10.9%sy, 0.0%ni, 89.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu3 : 1.0%us, 14.7%sy, 0.0%ni, 84.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu4 : 0.0%us, 12.0%sy, 0.0%ni, 88.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu5 : 1.0%us, 12.0%sy, 0.0%ni, 87.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu6 : 0.0%us, 11.0%sy, 0.0%ni, 89.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu7 : 0.0%us, 13.9%sy, 0.0%ni, 86.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu8 : 0.0%us, 11.9%sy, 0.0%ni, 88.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu9 : 0.0%us, 13.1%sy, 0.0%ni, 86.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu10 : 1.0%us, 10.9%sy, 0.0%ni, 88.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu11 : 0.0%us, 13.0%sy, 0.0%ni, 87.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu12 : 0.0%us, 12.0%sy, 0.0%ni, 88.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu13 : 1.0%us, 13.9%sy, 0.0%ni, 85.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu14 : 1.0%us, 13.0%sy, 0.0%ni, 86.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>>> Cpu15 : 0.0%us, 13.1%sy, 0.0%ni, 72.7%id, 0.0%wa, 0.0%hi, 14.1%si, 0.0%st
>>> Mem: 12464792k total, 11836420k used, 628372k free, 170492k buffers
>>> Swap: 10482404k total, 2144k used, 10480260k free, 9273984k cached
>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>> 10660 root 15 0 280m 31m 1312 S 284.3 0.3 10:42.76 suricata
>>> suricata stats.log file:
>>> decoder.pkts | Decode1 | 27887010
>>> decoder.pkts_per_sec | Decode1 | 126469.500000
>>> decoder.bytes | Decode1 | 40603486560
>>> decoder.bytes_per_sec | Decode1 | 184139592.000000
>>> decoder.mbit_per_sec | Decode1 | 1473.116736
>>> decoder.ipv4 | Decode1 | 27887010
>>> decoder.ethernet | Decode1 | 27887010
>>> decoder.udp | Decode1 | 27887010
>>> decoder.avg_pkt_size | Decode1 | 1456.000000
>>> decoder.max_pkt_size | Decode1 | 1456
>>>
>> Interesting numbers rmkml, thanks. The pkts_per_sec, bytes_per_sec and
>> mbit_per_sec counters are completely unreliable at this point, fixing
>> them is still on our todo list.
>>
>> Did both test runs send the same amount of packets? I see that the sigs
>> run did 4M packets, the bare run 27M.
>>
>> Cheers,
>> Victor
>>
> _______________________________________________
> Oisf-devel mailing list
> Oisf-devel at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>
>
More information about the Oisf-devel
mailing list