[Oisf-users] Percentage of dropped packets
Chris Wakelin
c.d.wakelin at reading.ac.uk
Tue May 29 15:29:07 UTC 2012
Flow emergencies are probably a bad thing. It might be worth tweaking
the timeouts.
I've been battling with tcp.reassembly_gap, which is pretty close to
zero with PF_RING and DNA. Without DNA, Bro IDS reckoned 80% of streams
were missing packets.
PF_RING gives some sort of drop statistics, but I think they're not
always reliable (might depend on the network card driver). "ethtool -S"
will tell you what the interface thought was dropped.
I've got some rough-and-ready scripts I use to try and work out what's
going on:
1) ethtool_stats.sh: run (as root) with something like "ethtool_stats.sh
eth4".
There are variations in names of some of the fields between NICs, so it
may need tweaking. You might also need to update to a more recent
version of ethtool (http://ftp.kernel.org/pub/software/network/ethtool/).
It gives output like:
> 2012-05-29 16:21:27 - Pkts: 543763502, Lost: 0, 0/10 %, Pkts/s: 105244, Mb/s: 659, Lost/s: 0, FSize 820
> 2012-05-29 16:21:37 - Pkts: 544835542, Lost: 0, 0/10 %, Pkts/s: 107204, Mb/s: 676, Lost/s: 0, FSize 827
2) suricata_stats.sh: run (as anything that can read stats.log) with
something like
tail -n +1 -F /var/log/suricata/stats.log \
| suricata_stats.sh decoder.pkts
(or whatever you're interested in) and gives output like:
> Date: 5/29/2012 -- 16:25:39 (uptime: 0d, 01h 28m 04s) : 8 secs, 51557 pkts/s, 332 Mb/s, 23743 gaps, 0 gaps/s 5513 10046 8883 8838 5442 12834
> Date: 5/29/2012 -- 16:25:47 (uptime: 0d, 01h 28m 12s) : 8 secs, 54312 pkts/s, 359 Mb/s, 23751 gaps, 1 gaps/s 8407 8001 7608 7587 9817 12890
where the last six (in this case) fields are per interface/queue
statistics for the parameter specified.
Hope this helps!
Best Wishes,
Chris
On 29/05/12 16:03, Peter Bates wrote:
>
> Hello all
>
> Apologies for what are probably FAQs.
>
> Being reasonably used to the Snort perfmonitor output, I'm trying to
> understand which line in stats.log might refer to dropped packets.
>
> Suricata is (when foregrounded) saying things like:
>
> [5535] 29/5/2012 -- 15:56:04 - (flow-manager.c:510) <Info>
> (FlowManagerThread) -- Flow emergency mode over, back to normal...
> unsetting FLOW_EMERGENCY bit (ts.tv_sec: 1338303363,
> ts.tv_usec:940223) flow_spare_q status(): 1062% flows at the queue
>
> And in stats.log I'm seeing:
>
> Date: 5/29/2012 -- 15:56:33 (uptime: 0d, 01h 31m 23s)
> -------------------------------------------------------------------
> Counter | TM Name | Value
> -------------------------------------------------------------------
> flow_mgr.closed_pruned | FlowManagerThread | 10540043
> flow_mgr.new_pruned | FlowManagerThread | 4116068
> flow_mgr.est_pruned | FlowManagerThread | 200991
> flow.memuse | FlowManagerThread | 30501404
> flow.spare | FlowManagerThread | 10233
> flow.emerg_mode_entered | FlowManagerThread | 118
> flow.emerg_mode_over | FlowManagerThread | 118
> decoder.pkts | AFPacketeth61 | 179519552
> decoder.bytes | AFPacketeth61 | 142002380276
> decoder.ipv4 | AFPacketeth61 | 179554718
> decoder.ipv6 | AFPacketeth61 | 379469
> decoder.ethernet | AFPacketeth61 | 179519552
> decoder.raw | AFPacketeth61 | 0
> decoder.sll | AFPacketeth61 | 0
> decoder.tcp | AFPacketeth61 | 151975697
> decoder.udp | AFPacketeth61 | 26584288
> decoder.sctp | AFPacketeth61 | 0
> decoder.icmpv4 | AFPacketeth61 | 88177
> decoder.icmpv6 | AFPacketeth61 | 21233
> decoder.ppp | AFPacketeth61 | 406824
> decoder.pppoe | AFPacketeth61 | 0
> decoder.gre | AFPacketeth61 | 406843
> decoder.vlan | AFPacketeth61 | 0
> decoder.avg_pkt_size | AFPacketeth61 | 791
> decoder.max_pkt_size | AFPacketeth61 | 1514
> defrag.ipv4.fragments | AFPacketeth61 | 375839
> defrag.ipv4.reassembled | AFPacketeth61 | 35789
> defrag.ipv4.timeouts | AFPacketeth61 | 0
> defrag.ipv6.fragments | AFPacketeth61 | 35
> defrag.ipv6.reassembled | AFPacketeth61 | 0
> defrag.ipv6.timeouts | AFPacketeth61 | 0
> tcp.sessions | AFPacketeth61 | 1809706
> tcp.ssn_memcap_drop | AFPacketeth61 | 0
> tcp.pseudo | AFPacketeth61 | 216
> tcp.invalid_checksum | AFPacketeth61 | 42560
> tcp.no_flow | AFPacketeth61 | 0
> tcp.reused_ssn | AFPacketeth61 | 67
> tcp.memuse | AFPacketeth61 | 4325376
> tcp.syn | AFPacketeth61 | 1878430
> tcp.synack | AFPacketeth61 | 1295929
> tcp.rst | AFPacketeth61 | 405377
> tcp.segment_memcap_drop | AFPacketeth61 | 0
> tcp.stream_depth_reached | AFPacketeth61 | 1
> tcp.reassembly_memuse | AFPacketeth61 | 15422350
> tcp.reassembly_gap | AFPacketeth61 | 4894
> detect.alert | AFPacketeth61 | 630
>
> Both values with 'drop' in their name are 0, is there a reported value
> in this list that corresponds to 'packets dropped'?
>
> Thanks.
>
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
--
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin, c.d.wakelin at reading.ac.uk
IT Services Centre, The University of Reading, Tel: +44 (0)118 378 2908
Whiteknights, Reading, RG6 6AF, UK Fax: +44 (0)118 975 3094
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ethtool_stats.sh
Type: application/x-shellscript
Size: 940 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20120529/f830b7eb/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: suricata_stats.sh
Type: application/x-shellscript
Size: 686 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20120529/f830b7eb/attachment-0001.bin>
More information about the Oisf-users
mailing list