[Oisf-users] [EXT] Re: Packet loss and increased resource consumption after upgrade to 4.1.2 with Rust support

[Cloherty, Sean E]

That also seems to be the case with me regarding high counts on tcp.pkt_on_wrong_thread.  I've reverted to 4.0.6 using the same setup and YAML and the stats look much better with no packet loss.  I will forward the data.  

Thanks.

-----Original Message-----
From: Peter Manev <petermanev at gmail.com> 
Sent: Wednesday, February 13, 2019 3:52 PM
To: Eric Urban <eurban at umn.edu>
Cc: Cloherty, Sean E <scloherty at mitre.org>; Open Information Security Foundation <oisf-users at lists.openinfosecfoundation.org>
Subject: Re: [EXT] Re: [Oisf-users] Packet loss and increased resource consumption after upgrade to 4.1.2 with Rust support

On Fri, Feb 8, 2019 at 6:34 PM Eric Urban <eurban at umn.edu> wrote:
>
> Peter, I emailed our config to you directly.  I mentioned in my original email that we did test having Rust enabled in 4.1.2 where I explicitly disabled the Rust parsers and still experienced significant packet loss.  In that case I added the following config under app-layer.protocols but left the rest of the config the same:
>


Thank you for sharing all the requested information.
Please find below my observations and some suggestions.

The good news with 4.1.2:
tcp.pkt_on_wrong_thread                    | Total                     | 100

This is very low (lowest i have seen) for the "tcp.pkt_on_wrong_thread " counter especially with a big run like the shared stats -over 20 days.
Do you mind sharing a bit more info on your NIC (Myricom i think - if I am not mistaken) - driver/version/any specific set up - we are trying to keep a record for that here -
https://redmine.openinfosecfoundation.org/issues/2725#note-13


Observations:
with 4.1.2 this counters seem odd -
capture.kernel_packets                     | Total
| 16345348068
capture.kernel_drops                        | Total
 | 33492892572
aka you have more kernel_drops than kernel_packets - seems odd.
Which makes me think It maybe a "counter" bug of some sort. Are the NIC driver versions the same on both boxes / same NIC config etc ?


Suggestions for the 4.1.2 set up:
Try a run where you disable those (false) and run again to see if any difference (?) :
  midstream: true            # allow midstream session pickups
  async-oneside: true        # enable async stream handling

Thank you