[Oisf-users] troubleshooting packet loss
Peter Manev
petermanev at gmail.com
Sun Mar 27 10:00:22 UTC 2016
On Thu, Mar 24, 2016 at 7:02 PM, Yasha Zislin <coolyasha at hotmail.com> wrote:
> I am trying to figure out where the packet loss is coming from on one of my
> Suricata 3.0 sensor.
> The only thing that I see weird from stats.log is that
> tpc.stream_depth_reached and tcp.reassembly_gap is somewhat high.
> I am using latest PF_RING and monitoring one interface with 4 threads.
> 4 logical CPUs with 16 gigs of RAM. 66% of RAM is used.
What traffic speeds are those on? How many rules do you load?
On the first interface there is 6.5% loss on the second 3.67% - over
what period of time was that?
>
> Here is stats.log info.
>
> Thank you
>
> capture.kernel_packets | RxPFRbond01 | 34118172
> capture.kernel_drops | RxPFRbond01 | 2240130
> decoder.pkts | RxPFRbond01 | 34125944
> decoder.bytes | RxPFRbond01 | 26624108366
> decoder.invalid | RxPFRbond01 | 0
> decoder.ipv4 | RxPFRbond01 | 34707873
> decoder.ipv6 | RxPFRbond01 | 570
> decoder.ethernet | RxPFRbond01 | 34125944
> decoder.raw | RxPFRbond01 | 0
> decoder.null | RxPFRbond01 | 0
> decoder.sll | RxPFRbond01 | 0
> decoder.tcp | RxPFRbond01 | 23715873
> decoder.udp | RxPFRbond01 | 9702569
> decoder.sctp | RxPFRbond01 | 0
> decoder.icmpv4 | RxPFRbond01 | 98456
> decoder.icmpv6 | RxPFRbond01 | 0
> decoder.ppp | RxPFRbond01 | 0
> decoder.pppoe | RxPFRbond01 | 0
> decoder.gre | RxPFRbond01 | 0
> decoder.vlan | RxPFRbond01 | 0
> decoder.vlan_qinq | RxPFRbond01 | 0
> decoder.teredo | RxPFRbond01 | 570
> decoder.ipv4_in_ipv6 | RxPFRbond01 | 0
> decoder.ipv6_in_ipv6 | RxPFRbond01 | 0
> decoder.mpls | RxPFRbond01 | 0
> decoder.avg_pkt_size | RxPFRbond01 | 780
> decoder.max_pkt_size | RxPFRbond01 | 1514
> decoder.erspan | RxPFRbond01 | 0
> flow.memcap | RxPFRbond01 | 0
> defrag.ipv4.fragments | RxPFRbond01 | 1190975
> defrag.ipv4.reassembled | RxPFRbond01 | 592903
> defrag.ipv4.timeouts | RxPFRbond01 | 0
> defrag.ipv6.fragments | RxPFRbond01 | 0
> defrag.ipv6.reassembled | RxPFRbond01 | 0
> defrag.ipv6.timeouts | RxPFRbond01 | 0
> defrag.max_frag_hits | RxPFRbond01 | 0
> tcp.sessions | RxPFRbond01 | 169101
> tcp.ssn_memcap_drop | RxPFRbond01 | 0
> tcp.pseudo | RxPFRbond01 | 77497
> tcp.pseudo_failed | RxPFRbond01 | 0
> tcp.invalid_checksum | RxPFRbond01 | 0
> tcp.no_flow | RxPFRbond01 | 0
> tcp.syn | RxPFRbond01 | 180407
> tcp.synack | RxPFRbond01 | 146913
> tcp.rst | RxPFRbond01 | 138896
> tcp.segment_memcap_drop | RxPFRbond01 | 0
> tcp.stream_depth_reached | RxPFRbond01 | 107
> tcp.reassembly_gap | RxPFRbond01 | 6765
> detect.alert | RxPFRbond01 | 3426
> capture.kernel_packets | RxPFRbond02 | 33927252
> capture.kernel_drops | RxPFRbond02 | 1246692
> decoder.pkts | RxPFRbond02 | 33932611
> decoder.bytes | RxPFRbond02 | 25571688366
> decoder.invalid | RxPFRbond02 | 0
> decoder.ipv4 | RxPFRbond02 | 34483004
> decoder.ipv6 | RxPFRbond02 | 506
> decoder.ethernet | RxPFRbond02 | 33932611
> decoder.raw | RxPFRbond02 | 0
> decoder.null | RxPFRbond02 | 0
> decoder.sll | RxPFRbond02 | 0
> decoder.tcp | RxPFRbond02 | 24665968
> decoder.udp | RxPFRbond02 | 8600129
> decoder.sctp | RxPFRbond02 | 0
> decoder.icmpv4 | RxPFRbond02 | 113797
> decoder.icmpv6 | RxPFRbond02 | 0
> decoder.ppp | RxPFRbond02 | 0
> decoder.pppoe | RxPFRbond02 | 0
> decoder.gre | RxPFRbond02 | 0
> decoder.vlan | RxPFRbond02 | 0
> decoder.vlan_qinq | RxPFRbond02 | 0
> decoder.teredo | RxPFRbond02 | 506
> decoder.ipv4_in_ipv6 | RxPFRbond02 | 0
> decoder.ipv6_in_ipv6 | RxPFRbond02 | 0
> decoder.mpls | RxPFRbond02 | 0
> decoder.avg_pkt_size | RxPFRbond02 | 753
> decoder.max_pkt_size | RxPFRbond02 | 1514
> decoder.erspan | RxPFRbond02 | 0
> flow.memcap | RxPFRbond02 | 0
> defrag.ipv4.fragments | RxPFRbond02 | 1103110
> defrag.ipv4.reassembled | RxPFRbond02 | 550393
> defrag.ipv4.timeouts | RxPFRbond02 | 0
> defrag.ipv6.fragments | RxPFRbond02 | 0
> defrag.ipv6.reassembled | RxPFRbond02 | 0
> defrag.ipv6.timeouts | RxPFRbond02 | 0
> defrag.max_frag_hits | RxPFRbond02 | 0
> tcp.sessions | RxPFRbond02 | 172432
> tcp.ssn_memcap_drop | RxPFRbond02 | 0
> tcp.pseudo | RxPFRbond02 | 79224
> tcp.pseudo_failed | RxPFRbond02 | 0
> tcp.invalid_checksum | RxPFRbond02 | 0
> tcp.no_flow | RxPFRbond02 | 0
> tcp.syn | RxPFRbond02 | 183912
> tcp.synack | RxPFRbond02 | 150219
> tcp.rst | RxPFRbond02 | 143693
> tcp.segment_memcap_drop | RxPFRbond02 | 0
> tcp.stream_depth_reached | RxPFRbond02 | 105
> tcp.reassembly_gap | RxPFRbond02 | 4710
> detect.alert | RxPFRbond02 | 3469
> capture.kernel_packets | RxPFRbond03 | 38750498
> capture.kernel_drops | RxPFRbond03 | 1511800
> decoder.pkts | RxPFRbond03 | 38762341
> decoder.bytes | RxPFRbond03 | 32714534213
> decoder.invalid | RxPFRbond03 | 0
> decoder.ipv4 | RxPFRbond03 | 39299710
> decoder.ipv6 | RxPFRbond03 | 512
> decoder.ethernet | RxPFRbond03 | 38762341
> decoder.raw | RxPFRbond03 | 0
> decoder.null | RxPFRbond03 | 0
> decoder.sll | RxPFRbond03 | 0
> decoder.tcp | RxPFRbond03 | 21943466
> decoder.udp | RxPFRbond03 | 15992492
> decoder.sctp | RxPFRbond03 | 0
> decoder.icmpv4 | RxPFRbond03 | 178089
> decoder.icmpv6 | RxPFRbond03 | 0
> decoder.ppp | RxPFRbond03 | 0
> decoder.pppoe | RxPFRbond03 | 0
> decoder.gre | RxPFRbond03 | 0
> decoder.vlan | RxPFRbond03 | 0
> decoder.vlan_qinq | RxPFRbond03 | 0
> decoder.teredo | RxPFRbond03 | 512
> decoder.ipv4_in_ipv6 | RxPFRbond03 | 0
> decoder.ipv6_in_ipv6 | RxPFRbond03 | 0
> decoder.mpls | RxPFRbond03 | 0
> decoder.avg_pkt_size | RxPFRbond03 | 843
> decoder.max_pkt_size | RxPFRbond03 | 1514
> decoder.erspan | RxPFRbond03 | 0
> flow.memcap | RxPFRbond03 | 0
> defrag.ipv4.fragments | RxPFRbond03 | 1078454
> defrag.ipv4.reassembled | RxPFRbond03 | 537369
> defrag.ipv4.timeouts | RxPFRbond03 | 0
> defrag.ipv6.fragments | RxPFRbond03 | 0
> defrag.ipv6.reassembled | RxPFRbond03 | 0
> defrag.ipv6.timeouts | RxPFRbond03 | 0
> defrag.max_frag_hits | RxPFRbond03 | 0
> tcp.sessions | RxPFRbond03 | 169832
> tcp.ssn_memcap_drop | RxPFRbond03 | 0
> tcp.pseudo | RxPFRbond03 | 78504
> tcp.pseudo_failed | RxPFRbond03 | 0
> tcp.invalid_checksum | RxPFRbond03 | 0
> tcp.no_flow | RxPFRbond03 | 0
> tcp.syn | RxPFRbond03 | 181453
> tcp.synack | RxPFRbond03 | 147649
> tcp.rst | RxPFRbond03 | 139792
> tcp.segment_memcap_drop | RxPFRbond03 | 0
> tcp.stream_depth_reached | RxPFRbond03 | 94
> tcp.reassembly_gap | RxPFRbond03 | 2567
> detect.alert | RxPFRbond03 | 3416
> capture.kernel_packets | RxPFRbond04 | 63727760
> capture.kernel_drops | RxPFRbond04 | 3046651
> decoder.pkts | RxPFRbond04 | 63747722
> decoder.bytes | RxPFRbond04 | 55373084583
> decoder.invalid | RxPFRbond04 | 0
> decoder.ipv4 | RxPFRbond04 | 64056225
> decoder.ipv6 | RxPFRbond04 | 487
> decoder.ethernet | RxPFRbond04 | 63747722
> decoder.raw | RxPFRbond04 | 0
> decoder.null | RxPFRbond04 | 0
> decoder.sll | RxPFRbond04 | 0
> decoder.tcp | RxPFRbond04 | 55855784
> decoder.udp | RxPFRbond04 | 7447497
> decoder.sctp | RxPFRbond04 | 0
> decoder.icmpv4 | RxPFRbond04 | 133539
> decoder.icmpv6 | RxPFRbond04 | 0
> decoder.ppp | RxPFRbond04 | 0
> decoder.pppoe | RxPFRbond04 | 0
> decoder.gre | RxPFRbond04 | 0
> decoder.vlan | RxPFRbond04 | 0
> decoder.vlan_qinq | RxPFRbond04 | 0
> decoder.teredo | RxPFRbond04 | 487
> decoder.ipv4_in_ipv6 | RxPFRbond04 | 0
> decoder.ipv6_in_ipv6 | RxPFRbond04 | 0
> decoder.mpls | RxPFRbond04 | 0
> decoder.avg_pkt_size | RxPFRbond04 | 868
> decoder.max_pkt_size | RxPFRbond04 | 1514
> decoder.erspan | RxPFRbond04 | 0
> flow.memcap | RxPFRbond04 | 0
> defrag.ipv4.fragments | RxPFRbond04 | 619405
> defrag.ipv4.reassembled | RxPFRbond04 | 308503
> defrag.ipv4.timeouts | RxPFRbond04 | 0
> defrag.ipv6.fragments | RxPFRbond04 | 0
> defrag.ipv6.reassembled | RxPFRbond04 | 0
> defrag.ipv6.timeouts | RxPFRbond04 | 0
> defrag.max_frag_hits | RxPFRbond04 | 0
> tcp.sessions | RxPFRbond04 | 171368
> tcp.ssn_memcap_drop | RxPFRbond04 | 0
> tcp.pseudo | RxPFRbond04 | 78609
> tcp.pseudo_failed | RxPFRbond04 | 0
> tcp.invalid_checksum | RxPFRbond04 | 0
> tcp.no_flow | RxPFRbond04 | 0
> tcp.syn | RxPFRbond04 | 182409
> tcp.synack | RxPFRbond04 | 149124
> tcp.rst | RxPFRbond04 | 143473
> tcp.segment_memcap_drop | RxPFRbond04 | 0
> tcp.stream_depth_reached | RxPFRbond04 | 82
> tcp.reassembly_gap | RxPFRbond04 | 35459
> detect.alert | RxPFRbond04 | 3770
> flow_mgr.closed_pruned | FlowManagerThread | 310602
> flow_mgr.new_pruned | FlowManagerThread | 549722
> flow_mgr.est_pruned | FlowManagerThread | 380334
> flow.spare | FlowManagerThread | 799999
> flow.emerg_mode_entered | FlowManagerThread | 0
> flow.emerg_mode_over | FlowManagerThread | 0
> flow.tcp_reuse | FlowManagerThread | 237
> flow_mgr.closed_pruned | FlowManagerThread | 308878
> flow_mgr.new_pruned | FlowManagerThread | 544586
> flow_mgr.est_pruned | FlowManagerThread | 379393
> flow.spare | FlowManagerThread | 799402
> flow.emerg_mode_entered | FlowManagerThread | 0
> flow.emerg_mode_over | FlowManagerThread | 0
> flow.tcp_reuse | FlowManagerThread | 252
> tcp.memuse | Global | 439248976
> tcp.reassembly_memuse | Global | 1717630000
> dns.memuse | Global | 476478
> dns.memcap_state | Global | 0
> dns.memcap_global | Global | 0
> http.memuse | Global | 536216
> http.memcap | Global | 0
> flow.memuse | Global | 237040288
>
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 9-11 in Washington, DC:
> http://oisfevents.net
--
Regards,
Peter Manev
More information about the Oisf-users
mailing list