[Oisf-users] TCP reassembly gaps

[Chris Wakelin]

On 19/04/12 06:33, Victor Julien wrote:
>> The tcp.reassembly_gap count (added over all 6 threads) is increasing at
>> about 40/sec.
>>
>> This is using an Intel 10GB (ixgbe) card, and PF_RING reckons there are
>> no lost packets. The network load is about 300-400Mb/s rising to nearly
>> 1GB when the students are all here. (One oddity about this port mirror
>> is that the packets are VLAN-tagged in only one direction. Extreme
>> Networks say this is by design :-$; I've modified the PF_RING packet
>> hash to ignore VLAN tags)
>>
>> On the main campus network, 1GB port mirror (VLAN-tagged properly) there
>> are no gaps, even though it's frequently losing packets (e.g. when the
>> traffic goes over 1GB).
> 
>> Any idea how do debug this? Could it be an ethernet driver issue?
> 
> This is strange indeed.
> 
> One way to debug is to enable this rule:
> 
> # Sequence gap: missing data in the reassembly engine. Usually due to
> packet loss. Will be very noisy on a overloaded link / sensor.
> alert tcp any any -> any any (msg:"SURICATA STREAM reassembly sequence
> GAP -- missing packet(s)"; stream-event:reassembly_seq_gap; sid:2210048;
> rev:1;)

Ah, I forgot we could sig these now!

> 
> You may want to threshold it some.
> 
> Then look at the streams that fire...
> 

I think it must be a PF_RING/ixgbe issue. I've got IRQ-pinning on and
RSS enabled so it might be worth trying with RSS turned off.

I created a pcap (~200MB/250K packets) with PF-RING-enabled tcpdump for
a minute or so, filtering on a Google/Youtube /24 network with "net
64.156.119.0/24 or (vlan and net 64.156.119.0/24)" to get varied sources
and destinations, and Wireshark agrees it's missing packets. PF_RING
stats suggest no dropped packets though.

The above sig hits 45 times and gave me some src/dst pairs to check in
Wireshark :)

Best Wishes,
Chris

-- 
--+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
Christopher Wakelin,                           c.d.wakelin at reading.ac.uk
IT Services Centre, The University of Reading,  Tel: +44 (0)118 378 2908
Whiteknights, Reading, RG6 6AF, UK              Fax: +44 (0)118 975 3094