[Oisf-devel] suricata testing
rmkml
rmkml at free.fr
Fri May 14 13:15:48 UTC 2010
Hi Gurvinder,
thx for comments,
sp$rent not sending icmp, only big udp packet (in this case).
sorry, tcp it's not possible at this time.
Regards
Rmkml
On Fri, 14 May 2010, Gurvinder Singh wrote:
> Thanks rmkml for the interesting numbers. I just wonder if there were any
> ICMP packets with the UDP traffic or not. As there is a known issue on todo
> list to fix the slowdown when ICMP and UDP are together in the traffic. If
> possible can you also test the engine with TCP traffic or just UDP traffic,
> you can have ICMP with TCP, as ICMP handling with TCP traffic is fine.
> Cheers,
> Gurvinder
>
> rmkml wrote:
>> Thx Victor and Will for reply,
>> Im reply for victor question: no, 1) test it's ~150Mbit udp, then 3) test
>> it's ~1Gbit udp...
>>
>> and for Will question, I have created new test 4) with emerging-threat
>> rules (thx all and matt) at 12.668/18.496.000octet/148MBit (15% sending
>> possibility):
>> {today downloaded+unzip
>> http://www.emergingthreats.net/rules/emerging-all.rules.zip and use on
>> suricata engine without modification}
>> stats.log output:
>> decoder.pkts | Decode1 | 4946191
>> decoder.pkts_per_sec | Decode1 | 19003.333333
>> decoder.bytes | Decode1 | 7201654096
>> decoder.bytes_per_sec | Decode1 | 27668853.333333
>> decoder.mbit_per_sec | Decode1 | 221.350827
>> decoder.ipv4 | Decode1 | 4946191
>> decoder.ethernet | Decode1 | 4946191
>> decoder.udp | Decode1 | 4946191
>> decoder.avg_pkt_size | Decode1 | 1456.000000
>> decoder.max_pkt_size | Decode1 | 1456
>> ...
>> and top output:
>> top - 16:21:44 up 1 day, 23:45, 5 users, load average: 9.90, 8.33, 6.15
>> Tasks: 241 total, 1 running, 240 sleeping, 0 stopped, 0 zombie
>> Cpu0 : 29.7%us, 29.7%sy, 23.8%ni, 16.8%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu1 : 48.5%us, 4.0%sy, 0.0%ni, 47.5%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu2 : 49.0%us, 2.9%sy, 0.0%ni, 48.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu3 : 49.0%us, 3.9%sy, 0.0%ni, 47.1%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu4 : 49.5%us, 3.0%sy, 0.0%ni, 47.5%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu5 : 48.0%us, 2.9%sy, 0.0%ni, 49.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu6 : 52.0%us, 2.0%sy, 0.0%ni, 46.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu7 : 52.5%us, 4.0%sy, 0.0%ni, 43.6%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu8 : 57.8%us, 2.0%sy, 0.0%ni, 40.2%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu9 : 47.5%us, 3.0%sy, 0.0%ni, 49.5%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu10 : 49.5%us, 4.0%sy, 0.0%ni, 46.5%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu11 : 49.5%us, 3.0%sy, 0.0%ni, 47.5%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu12 : 47.5%us, 3.0%sy, 0.0%ni, 49.5%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu13 : 49.5%us, 2.0%sy, 0.0%ni, 48.5%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu14 : 49.0%us, 3.0%sy, 0.0%ni, 48.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>> 0.0%st
>> Cpu15 : 37.6%us, 2.0%sy, 0.0%ni, 33.7%id, 0.0%wa, 0.0%hi, 26.7%si,
>> 0.0%st
>> Mem: 12464792k total, 12001688k used, 463104k free, 174304k buffers
>> Swap: 10482404k total, 2144k used, 10480260k free, 9287904k cached
>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>> 28321 root 15 0 380m 130m 1348 S 876.4 1.1 68:03.01 suricata
>> Regards
>> Rmkml
>>
>>
>> On Fri, 14 May 2010, Victor Julien wrote:
>>
>>
>>> rmkml wrote:
>>>
>>>> Hi SDT (Suricata Devel Team),
>>>> Im start playing with 16 core server for suricata (v0.9.1pre git12may).
>>>> Im test with sp*rent test center gig and udp only src_port=dst_port=1024,
>>>> size 1460 (zero filled) at this time on IDS mode.
>>>> system is rhelv5.5i386 without pfring, but in this test, it's not a pb
>>>> for me.
>>>> network card is internal Broadcom NetXtreme II Gigabit Ethernet Driver
>>>> bnx2 v2.0.2 (Aug 21, 2009).
>>>>
>>>> 1) with all my personal signatures (+old community rules)
>>>> -- 7565 signatures processed. 8 are IP-only rules, 6567 are inspecting
>>>> packet payload, 1490 inspect application layer
>>>> result: 1597% cpu (16core), udp frame rate sending by sp*rent
>>>> 12.668/18.496.000octet/148MBit (15% sending possibility)
>>>> suricata stats.log file:
>>>> decoder.pkts | Decode1 | 4076896
>>>> decoder.pkts_per_sec | Decode1 | 15044.714286
>>>> decoder.bytes | Decode1 | 5935960576
>>>> decoder.bytes_per_sec | Decode1 | 21905104.000000
>>>> decoder.mbit_per_sec | Decode1 | 175.240832
>>>> decoder.ipv4 | Decode1 | 4076896
>>>> decoder.ethernet | Decode1 | 4076896
>>>> decoder.udp | Decode1 | 4076896
>>>> decoder.avg_pkt_size | Decode1 | 1456.000000
>>>> decoder.max_pkt_size | Decode1 | 1456
>>>> (removed all field contains 0)
>>>> top output:
>>>> top - 14:42:59 up 1 day, 22:06, 4 users, load average: 16.36, 13.77,
>>>> 9.79
>>>> Tasks: 236 total, 2 running, 234 sleeping, 0 stopped, 0 zombie
>>>> Cpu0 : 6.9%us, 13.9%sy, 79.2%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu1 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu2 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu3 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu4 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu5 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu6 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu7 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu8 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu9 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu10 : 99.0%us, 1.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu11 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu12 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu13 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu14 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu15 : 95.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 5.0%si,
>>>> 0.0%st
>>>> Mem: 12464792k total, 12057272k used, 407520k free, 170072k
>>>> buffers
>>>> Swap: 10482404k total, 2144k used, 10480260k free, 9272564k cached
>>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>>> 8059 root 15 0 504m 253m 1348 S 1595.9 2.1 119:10.11 suricata
>>>>
>>>> 2) same test without signature on suricata:
>>>> top output:
>>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>>> 10660 root 15 0 280m 31m 1312 S 88.3 0.3 0:30.11 suricata
>>>>
>>>> 3) suricata without signature receiving 1Gbit rate:
>>>> top output:
>>>> top - 14:55:16 up 1 day, 22:18, 4 users, load average: 4.53, 5.69,
>>>> 7.99
>>>> Tasks: 236 total, 1 running, 235 sleeping, 0 stopped, 0 zombie
>>>> Cpu0 : 29.4%us, 60.8%sy, 0.0%ni, 9.8%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu1 : 1.0%us, 12.9%sy, 0.0%ni, 86.1%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu2 : 0.0%us, 10.9%sy, 0.0%ni, 89.1%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu3 : 1.0%us, 14.7%sy, 0.0%ni, 84.3%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu4 : 0.0%us, 12.0%sy, 0.0%ni, 88.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu5 : 1.0%us, 12.0%sy, 0.0%ni, 87.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu6 : 0.0%us, 11.0%sy, 0.0%ni, 89.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu7 : 0.0%us, 13.9%sy, 0.0%ni, 86.1%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu8 : 0.0%us, 11.9%sy, 0.0%ni, 88.1%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu9 : 0.0%us, 13.1%sy, 0.0%ni, 86.9%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu10 : 1.0%us, 10.9%sy, 0.0%ni, 88.1%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu11 : 0.0%us, 13.0%sy, 0.0%ni, 87.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu12 : 0.0%us, 12.0%sy, 0.0%ni, 88.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu13 : 1.0%us, 13.9%sy, 0.0%ni, 85.1%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu14 : 1.0%us, 13.0%sy, 0.0%ni, 86.0%id, 0.0%wa, 0.0%hi, 0.0%si,
>>>> 0.0%st
>>>> Cpu15 : 0.0%us, 13.1%sy, 0.0%ni, 72.7%id, 0.0%wa, 0.0%hi, 14.1%si,
>>>> 0.0%st
>>>> Mem: 12464792k total, 11836420k used, 628372k free, 170492k buffers
>>>> Swap: 10482404k total, 2144k used, 10480260k free, 9273984k cached
>>>> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>>>> 10660 root 15 0 280m 31m 1312 S 284.3 0.3 10:42.76 suricata
>>>> suricata stats.log file:
>>>> decoder.pkts | Decode1 | 27887010
>>>> decoder.pkts_per_sec | Decode1 | 126469.500000
>>>> decoder.bytes | Decode1 | 40603486560
>>>> decoder.bytes_per_sec | Decode1 | 184139592.000000
>>>> decoder.mbit_per_sec | Decode1 | 1473.116736
>>>> decoder.ipv4 | Decode1 | 27887010
>>>> decoder.ethernet | Decode1 | 27887010
>>>> decoder.udp | Decode1 | 27887010
>>>> decoder.avg_pkt_size | Decode1 | 1456.000000
>>>> decoder.max_pkt_size | Decode1 | 1456
>>>>
>>> Interesting numbers rmkml, thanks. The pkts_per_sec, bytes_per_sec and
>>> mbit_per_sec counters are completely unreliable at this point, fixing
>>> them is still on our todo list.
>>>
>>> Did both test runs send the same amount of packets? I see that the sigs
>>> run did 4M packets, the bare run 27M.
>>>
>>> Cheers,
>>> Victor
>>>
>> _______________________________________________
>> Oisf-devel mailing list
>> Oisf-devel at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>
>>
>
>
More information about the Oisf-devel
mailing list