[Oisf-users] [EXT] Re: Packet loss and increased resource consumption after upgrade to 4.1.2 with Rust support

[Michał Purzyński]

So here's what I did at one of out offices. Intel X710 are used there

ethtool -L enp17s0f0 combined 4
ethtool -L enp17s0f0 rxhash on
ethtool -K enp17s0f0 ntuple on

ethtool -X enp17s0f0 hkey
6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A
equal 4

ethtool -N enp17s0f0 rx-flow-hash tcp4 sd

The _qm flow has been used in both Zeek and Suricata.

So far I can see traffic hashed correctly. If you have some ideas how I
could test this further, please let me know.


On Wed, Jun 5, 2019 at 11:52 AM Nelson, Cooper <cnelson at ucsd.edu> wrote:

> I updated the ticket, so I'm throwing some info here as well in case any
> OISF readers have ideas.
>
> It appears that switching to the older 'autofp' runmode resolves this
> issue (while also greatly increasing packet drops on my system).  My theory
> being that  by running the stream tracker on the RSS cores, separate from
> the detect threads, fragmented TCP are properly reassembled before being
> forwarded.  In the 'workers' runmode the fragments end up on the 'wrong'
> threads and increase the counter.
>
> Digging deeper into the kernel and suricata source code, at this point I
> think the solution would be to update cluster_flow to properly handle
> fragmented TCP/UDP packets.  Since this is 'broken' in the hardware RSS
> implementation it needs to be fixed in user space, the AF_PACKET code in
> the kernel doesn't seem to be able to handle this edge case properly.
>
> There are a couple ways this could be handled; as mentioned the easiest
> thing is to just 'punt' and only hash on the IP header src->dst for all
> protocols.
>
> Another thing would be to check the 'fragflag' and offset fields in each
> TCP/UDP packet and then send them to a dedicated thread queue for
> reassembly, then computing the hash and forwarding the packet to the proper
> thread.  This should guarantee that the fragments are put together in the
> proper order.
>
> -Coop
>
> -----Original Message-----
> From: Peter Manev <petermanev at gmail.com>
> Sent: Wednesday, May 29, 2019 2:00 PM
> To: Michał Purzyński <michalpurzynski1 at gmail.com>
> Cc: Nelson, Cooper <cnelson at ucsd.edu>; Cloherty, Sean E <
> scloherty at mitre.org>; Eric Urban <eurban at umn.edu>; Open Information
> Security Foundation <oisf-users at lists.openinfosecfoundation.org>
> Subject: Re: [Oisf-users] [EXT] Re: Packet loss and increased resource
> consumption after upgrade to 4.1.2 with Rust support
>
> On Wed, May 29, 2019 at 9:52 PM Michał Purzyński <
> michalpurzynski1 at gmail.com> wrote:
> >
> > How about ignoring layers above 3 and just going with ip src + ip dst?
> I'm pretty sure I can do that on i40e.
> >
>
> Lets give it a spin ? :)
> Maybe  do runs with/without taking into consideration of vlanids?
> (just to see if related)
>
>
> --
> Regards,
> Peter Manev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190701/0ac054b4/attachment.html>