[Oisf-users] where are my missing packets ?

[Victor Julien]

On 02/23/2012 09:33 AM, Travel Factory S.r.l. wrote:
> 
> 
> I was collecting some infos.... suricata has been running all the night
> long, and now, with a 130 mbit load a
> tail -f stats.log | grep tcp.segment_memcap_drop
> reports
> 
> tcp.segment_memcap_drop   | Decode & Stream           | 32054746
> tcp.segment_memcap_drop   | Decode & Stream           | 32067757
> tcp.segment_memcap_drop   | Decode & Stream           | 32086127
> tcp.segment_memcap_drop   | Decode & Stream           | 32103102
> tcp.segment_memcap_drop   | Decode & Stream           | 32124890
> tcp.segment_memcap_drop   | Decode & Stream           | 32148578
> tcp.segment_memcap_drop   | Decode & Stream           | 32171766
> tcp.segment_memcap_drop   | Decode & Stream           | 32189165
> tcp.segment_memcap_drop   | Decode & Stream           | 32211397
> tcp.segment_memcap_drop   | Decode & Stream           | 32233739
> tcp.segment_memcap_drop   | Decode & Stream           | 32262092
> tcp.segment_memcap_drop   | Decode & Stream           | 32277511
> tcp.segment_memcap_drop   | Decode & Stream           | 32295917
> tcp.segment_memcap_drop   | Decode & Stream           | 32319345
> tcp.segment_memcap_drop   | Decode & Stream           | 32338257
> tcp.segment_memcap_drop   | Decode & Stream           | 32357508

To make these go away, increase your stream.reassembly.memcap value. I
think you have it set to 512mb or so:

tcp.reassembly_memuse     | Decode & Stream           | 536870870

You could try doubling it.

> 
> A stat.log record log is the following:
> -------------------------------------------------------------------
> Date: 2/23/2012 -- 09:31:32 (uptime: 0d, 16h 10m 01s)
> -------------------------------------------------------------------
> Counter                   | TM Name                   | Value
> -------------------------------------------------------------------
> detect.alert              | Detect                    | 18
> decoder.pkts              | Decode & Stream           | 514484781
> decoder.bytes             | Decode & Stream           | 475566946964
> decoder.ipv4              | Decode & Stream           | 513403747
> decoder.ipv6              | Decode & Stream           | 1899
> decoder.ethernet          | Decode & Stream           | 514484781
> decoder.raw               | Decode & Stream           | 0
> decoder.sll               | Decode & Stream           | 0
> decoder.tcp               | Decode & Stream           | 196568162
> decoder.udp               | Decode & Stream           | 285486352
> decoder.sctp              | Decode & Stream           | 0
> decoder.icmpv4            | Decode & Stream           | 596837
> decoder.icmpv6            | Decode & Stream           | 209
> decoder.ppp               | Decode & Stream           | 0
> decoder.pppoe             | Decode & Stream           | 0
> decoder.gre               | Decode & Stream           | 0
> decoder.vlan              | Decode & Stream           | 0
> decoder.avg_pkt_size      | Decode & Stream           | 924.355714
> decoder.max_pkt_size      | Decode & Stream           | 1518
> defrag.ipv4.fragments     | Decode & Stream           | 1483782
> defrag.ipv4.reassembled   | Decode & Stream           | 34346
> defrag.ipv4.timeouts      | Decode & Stream           | 0
> defrag.ipv6.fragments     | Decode & Stream           | 0
> defrag.ipv6.reassembled   | Decode & Stream           | 0
> defrag.ipv6.timeouts      | Decode & Stream           | 0
> tcp.sessions              | Decode & Stream           | 2265299
> tcp.ssn_memcap_drop       | Decode & Stream           | 0
> tcp.pseudo                | Decode & Stream           | 176610
> tcp.invalid_checksum      | Decode & Stream           | 0
> tcp.no_flow               | Decode & Stream           | 0
> tcp.reused_ssn            | Decode & Stream           | 318
> tcp.memuse                | Decode & Stream           | 34023072.000000
> tcp.syn                   | Decode & Stream           | 2429325
> tcp.synack                | Decode & Stream           | 2213982
> tcp.rst                   | Decode & Stream           | 257041
> tcp.segment_memcap_drop   | Decode & Stream           | 32621673
> tcp.stream_depth_reached  | Decode & Stream           | 0
> tcp.reassembly_memuse     | Decode & Stream           | 536870870.000000
> tcp.reassembly_gap        | Decode & Stream           | 86271
> flow_mgr.closed_pruned    | FlowManagerThread         | 1983794
> flow_mgr.new_pruned       | FlowManagerThread         | 601775
> flow_mgr.est_pruned       | FlowManagerThread         | 843364
> flow.memuse               | FlowManagerThread         | 29185432.000000
> flow.emerg_mode_entered   | FlowManagerThread         | 0
> flow.emerg_mode_over      | FlowManagerThread         | 0

Nothing out of the ordinary here, although udp is high vs tcp.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------