[Oisf-users] troubleshooting packet loss

Yasha Zislin coolyasha at hotmail.com
Thu Mar 24 18:02:06 UTC 2016


I am trying to figure out where the packet loss is coming from on one of my Suricata 3.0 sensor.The only thing that I see weird from stats.log is that tpc.stream_depth_reached  and tcp.reassembly_gap is somewhat high.I am using latest PF_RING and monitoring one interface with 4 threads.4 logical CPUs with 16 gigs of RAM. 66% of RAM is used.
Here is stats.log info.
Thank you
capture.kernel_packets    | RxPFRbond01               | 34118172capture.kernel_drops      | RxPFRbond01               | 2240130decoder.pkts              | RxPFRbond01               | 34125944decoder.bytes             | RxPFRbond01               | 26624108366decoder.invalid           | RxPFRbond01               | 0decoder.ipv4              | RxPFRbond01               | 34707873decoder.ipv6              | RxPFRbond01               | 570decoder.ethernet          | RxPFRbond01               | 34125944decoder.raw               | RxPFRbond01               | 0decoder.null              | RxPFRbond01               | 0decoder.sll               | RxPFRbond01               | 0decoder.tcp               | RxPFRbond01               | 23715873decoder.udp               | RxPFRbond01               | 9702569decoder.sctp              | RxPFRbond01               | 0decoder.icmpv4            | RxPFRbond01               | 98456decoder.icmpv6            | RxPFRbond01               | 0decoder.ppp               | RxPFRbond01               | 0decoder.pppoe             | RxPFRbond01               | 0decoder.gre               | RxPFRbond01               | 0decoder.vlan              | RxPFRbond01               | 0decoder.vlan_qinq         | RxPFRbond01               | 0decoder.teredo            | RxPFRbond01               | 570decoder.ipv4_in_ipv6      | RxPFRbond01               | 0decoder.ipv6_in_ipv6      | RxPFRbond01               | 0decoder.mpls              | RxPFRbond01               | 0decoder.avg_pkt_size      | RxPFRbond01               | 780decoder.max_pkt_size      | RxPFRbond01               | 1514decoder.erspan            | RxPFRbond01               | 0flow.memcap               | RxPFRbond01               | 0defrag.ipv4.fragments     | RxPFRbond01               | 1190975defrag.ipv4.reassembled   | RxPFRbond01               | 592903defrag.ipv4.timeouts      | RxPFRbond01               | 0defrag.ipv6.fragments     | RxPFRbond01               | 0defrag.ipv6.reassembled   | RxPFRbond01               | 0defrag.ipv6.timeouts      | RxPFRbond01               | 0defrag.max_frag_hits      | RxPFRbond01               | 0tcp.sessions              | RxPFRbond01               | 169101tcp.ssn_memcap_drop       | RxPFRbond01               | 0tcp.pseudo                | RxPFRbond01               | 77497tcp.pseudo_failed         | RxPFRbond01               | 0tcp.invalid_checksum      | RxPFRbond01               | 0tcp.no_flow               | RxPFRbond01               | 0tcp.syn                   | RxPFRbond01               | 180407tcp.synack                | RxPFRbond01               | 146913tcp.rst                   | RxPFRbond01               | 138896tcp.segment_memcap_drop   | RxPFRbond01               | 0tcp.stream_depth_reached  | RxPFRbond01               | 107tcp.reassembly_gap        | RxPFRbond01               | 6765detect.alert              | RxPFRbond01               | 3426capture.kernel_packets    | RxPFRbond02               | 33927252capture.kernel_drops      | RxPFRbond02               | 1246692decoder.pkts              | RxPFRbond02               | 33932611decoder.bytes             | RxPFRbond02               | 25571688366decoder.invalid           | RxPFRbond02               | 0decoder.ipv4              | RxPFRbond02               | 34483004decoder.ipv6              | RxPFRbond02               | 506decoder.ethernet          | RxPFRbond02               | 33932611decoder.raw               | RxPFRbond02               | 0decoder.null              | RxPFRbond02               | 0decoder.sll               | RxPFRbond02               | 0decoder.tcp               | RxPFRbond02               | 24665968decoder.udp               | RxPFRbond02               | 8600129decoder.sctp              | RxPFRbond02               | 0decoder.icmpv4            | RxPFRbond02               | 113797decoder.icmpv6            | RxPFRbond02               | 0decoder.ppp               | RxPFRbond02               | 0decoder.pppoe             | RxPFRbond02               | 0decoder.gre               | RxPFRbond02               | 0decoder.vlan              | RxPFRbond02               | 0decoder.vlan_qinq         | RxPFRbond02               | 0decoder.teredo            | RxPFRbond02               | 506decoder.ipv4_in_ipv6      | RxPFRbond02               | 0decoder.ipv6_in_ipv6      | RxPFRbond02               | 0decoder.mpls              | RxPFRbond02               | 0decoder.avg_pkt_size      | RxPFRbond02               | 753decoder.max_pkt_size      | RxPFRbond02               | 1514decoder.erspan            | RxPFRbond02               | 0flow.memcap               | RxPFRbond02               | 0defrag.ipv4.fragments     | RxPFRbond02               | 1103110defrag.ipv4.reassembled   | RxPFRbond02               | 550393defrag.ipv4.timeouts      | RxPFRbond02               | 0defrag.ipv6.fragments     | RxPFRbond02               | 0defrag.ipv6.reassembled   | RxPFRbond02               | 0defrag.ipv6.timeouts      | RxPFRbond02               | 0defrag.max_frag_hits      | RxPFRbond02               | 0tcp.sessions              | RxPFRbond02               | 172432tcp.ssn_memcap_drop       | RxPFRbond02               | 0tcp.pseudo                | RxPFRbond02               | 79224tcp.pseudo_failed         | RxPFRbond02               | 0tcp.invalid_checksum      | RxPFRbond02               | 0tcp.no_flow               | RxPFRbond02               | 0tcp.syn                   | RxPFRbond02               | 183912tcp.synack                | RxPFRbond02               | 150219tcp.rst                   | RxPFRbond02               | 143693tcp.segment_memcap_drop   | RxPFRbond02               | 0tcp.stream_depth_reached  | RxPFRbond02               | 105tcp.reassembly_gap        | RxPFRbond02               | 4710detect.alert              | RxPFRbond02               | 3469capture.kernel_packets    | RxPFRbond03               | 38750498capture.kernel_drops      | RxPFRbond03               | 1511800decoder.pkts              | RxPFRbond03               | 38762341decoder.bytes             | RxPFRbond03               | 32714534213decoder.invalid           | RxPFRbond03               | 0decoder.ipv4              | RxPFRbond03               | 39299710decoder.ipv6              | RxPFRbond03               | 512decoder.ethernet          | RxPFRbond03               | 38762341decoder.raw               | RxPFRbond03               | 0decoder.null              | RxPFRbond03               | 0decoder.sll               | RxPFRbond03               | 0decoder.tcp               | RxPFRbond03               | 21943466decoder.udp               | RxPFRbond03               | 15992492decoder.sctp              | RxPFRbond03               | 0decoder.icmpv4            | RxPFRbond03               | 178089decoder.icmpv6            | RxPFRbond03               | 0decoder.ppp               | RxPFRbond03               | 0decoder.pppoe             | RxPFRbond03               | 0decoder.gre               | RxPFRbond03               | 0decoder.vlan              | RxPFRbond03               | 0decoder.vlan_qinq         | RxPFRbond03               | 0decoder.teredo            | RxPFRbond03               | 512decoder.ipv4_in_ipv6      | RxPFRbond03               | 0decoder.ipv6_in_ipv6      | RxPFRbond03               | 0decoder.mpls              | RxPFRbond03               | 0decoder.avg_pkt_size      | RxPFRbond03               | 843decoder.max_pkt_size      | RxPFRbond03               | 1514decoder.erspan            | RxPFRbond03               | 0flow.memcap               | RxPFRbond03               | 0defrag.ipv4.fragments     | RxPFRbond03               | 1078454defrag.ipv4.reassembled   | RxPFRbond03               | 537369defrag.ipv4.timeouts      | RxPFRbond03               | 0defrag.ipv6.fragments     | RxPFRbond03               | 0defrag.ipv6.reassembled   | RxPFRbond03               | 0defrag.ipv6.timeouts      | RxPFRbond03               | 0defrag.max_frag_hits      | RxPFRbond03               | 0tcp.sessions              | RxPFRbond03               | 169832tcp.ssn_memcap_drop       | RxPFRbond03               | 0tcp.pseudo                | RxPFRbond03               | 78504tcp.pseudo_failed         | RxPFRbond03               | 0tcp.invalid_checksum      | RxPFRbond03               | 0tcp.no_flow               | RxPFRbond03               | 0tcp.syn                   | RxPFRbond03               | 181453tcp.synack                | RxPFRbond03               | 147649tcp.rst                   | RxPFRbond03               | 139792tcp.segment_memcap_drop   | RxPFRbond03               | 0tcp.stream_depth_reached  | RxPFRbond03               | 94tcp.reassembly_gap        | RxPFRbond03               | 2567detect.alert              | RxPFRbond03               | 3416capture.kernel_packets    | RxPFRbond04               | 63727760capture.kernel_drops      | RxPFRbond04               | 3046651decoder.pkts              | RxPFRbond04               | 63747722decoder.bytes             | RxPFRbond04               | 55373084583decoder.invalid           | RxPFRbond04               | 0decoder.ipv4              | RxPFRbond04               | 64056225decoder.ipv6              | RxPFRbond04               | 487decoder.ethernet          | RxPFRbond04               | 63747722decoder.raw               | RxPFRbond04               | 0decoder.null              | RxPFRbond04               | 0decoder.sll               | RxPFRbond04               | 0decoder.tcp               | RxPFRbond04               | 55855784decoder.udp               | RxPFRbond04               | 7447497decoder.sctp              | RxPFRbond04               | 0decoder.icmpv4            | RxPFRbond04               | 133539decoder.icmpv6            | RxPFRbond04               | 0decoder.ppp               | RxPFRbond04               | 0decoder.pppoe             | RxPFRbond04               | 0decoder.gre               | RxPFRbond04               | 0decoder.vlan              | RxPFRbond04               | 0decoder.vlan_qinq         | RxPFRbond04               | 0decoder.teredo            | RxPFRbond04               | 487decoder.ipv4_in_ipv6      | RxPFRbond04               | 0decoder.ipv6_in_ipv6      | RxPFRbond04               | 0decoder.mpls              | RxPFRbond04               | 0decoder.avg_pkt_size      | RxPFRbond04               | 868decoder.max_pkt_size      | RxPFRbond04               | 1514decoder.erspan            | RxPFRbond04               | 0flow.memcap               | RxPFRbond04               | 0defrag.ipv4.fragments     | RxPFRbond04               | 619405defrag.ipv4.reassembled   | RxPFRbond04               | 308503defrag.ipv4.timeouts      | RxPFRbond04               | 0defrag.ipv6.fragments     | RxPFRbond04               | 0defrag.ipv6.reassembled   | RxPFRbond04               | 0defrag.ipv6.timeouts      | RxPFRbond04               | 0defrag.max_frag_hits      | RxPFRbond04               | 0tcp.sessions              | RxPFRbond04               | 171368tcp.ssn_memcap_drop       | RxPFRbond04               | 0tcp.pseudo                | RxPFRbond04               | 78609tcp.pseudo_failed         | RxPFRbond04               | 0tcp.invalid_checksum      | RxPFRbond04               | 0tcp.no_flow               | RxPFRbond04               | 0tcp.syn                   | RxPFRbond04               | 182409tcp.synack                | RxPFRbond04               | 149124tcp.rst                   | RxPFRbond04               | 143473tcp.segment_memcap_drop   | RxPFRbond04               | 0tcp.stream_depth_reached  | RxPFRbond04               | 82tcp.reassembly_gap        | RxPFRbond04               | 35459detect.alert              | RxPFRbond04               | 3770flow_mgr.closed_pruned    | FlowManagerThread         | 310602flow_mgr.new_pruned       | FlowManagerThread         | 549722flow_mgr.est_pruned       | FlowManagerThread         | 380334flow.spare                | FlowManagerThread         | 799999flow.emerg_mode_entered   | FlowManagerThread         | 0flow.emerg_mode_over      | FlowManagerThread         | 0flow.tcp_reuse            | FlowManagerThread         | 237flow_mgr.closed_pruned    | FlowManagerThread         | 308878flow_mgr.new_pruned       | FlowManagerThread         | 544586flow_mgr.est_pruned       | FlowManagerThread         | 379393flow.spare                | FlowManagerThread         | 799402flow.emerg_mode_entered   | FlowManagerThread         | 0flow.emerg_mode_over      | FlowManagerThread         | 0flow.tcp_reuse            | FlowManagerThread         | 252tcp.memuse                | Global                    | 439248976tcp.reassembly_memuse     | Global                    | 1717630000dns.memuse                | Global                    | 476478dns.memcap_state          | Global                    | 0dns.memcap_global         | Global                    | 0http.memuse               | Global                    | 536216http.memcap               | Global                    | 0flow.memuse               | Global                    | 237040288
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160324/905c4060/attachment-0001.html>


More information about the Oisf-users mailing list