[Oisf-users] [EXT] Re: Packet loss and increased resource consumption after upgrade to 4.1.2 with Rust support

[Eric Urban]

Peter, I emailed our config to you directly.  I mentioned in my original
email that we did test having Rust enabled in 4.1.2 where I explicitly
disabled the Rust parsers and still experienced significant packet loss.
In that case I added the following config under app-layer.protocols but
left the rest of the config the same:

    krb5:
      enabled: no
    ikev2:
      enabled: no
    nfs:
      enabled: no
    tftp:
      enabled: no
    ntp:
      enabled: no
    dhcp:
      enabled: no



On Thu, Feb 7, 2019 at 12:54 PM Cloherty, Sean E <scloherty at mitre.org>
wrote:

> Still seeing this myself so I will be grabbing configs, rebuilding with
> 4.0.6 and getting that data as well.
>
> -----Original Message-----
> From: Oisf-users <oisf-users-bounces at lists.openinfosecfoundation.org> On
> Behalf Of Peter Manev
> Sent: Wednesday, February 6, 2019 11:54 AM
> To: Eric Urban <eurban at umn.edu>
> Cc: Open Information Security Foundation <
> oisf-users at lists.openinfosecfoundation.org>
> Subject: [EXT] Re: [Oisf-users] Packet loss and increased resource
> consumption after upgrade to 4.1.2 with Rust support
>
> On Tue, Feb 5, 2019 at 11:14 PM Eric Urban <eurban at umn.edu> wrote:
> >
> > I have seen a few emails on this list about users either having packet
> loss or increased resource consumption after upgrading to 4.1.2.  We are
> seeing much higher rates of packet loss after upgrading to 4.1.2 (with
> Rust) from 4.0.6 (no Rust) so would appreciate any input on how to best
> move forward with troubleshooting.  Please let me know if it would it be
> better to open a ticket in Redmine.
> >
> > Here are some details:
> > - We have two sets of Suricata sensors that are each getting the same
> set of traffic, so one acts as a redundant set.  These have the same
> hardware.
> > - Once we upgraded to 4.1.2, cpu and memory usage went up and we have
> had regular bursts of heavy packet loss.  I sampled traffic from yesterday
> early morning through today and a few sensors have had 2.49, 9.36, and
> 11.130% packet loss over that time frame.  For our 4.0.6 sensor set over
> the same time for the same traffic the sensor with the highest loss has
> 0.011%.  We have also had one occasion where a sensor had possible memory
> exhaustion as the stats.tcp.ssn_memcap_drop_delta counter hit 199.
> > - We rolled back our primary sensor set to 4.0.6 and immediately stopped
> having drops.
> > - We did not explicitly enable or disable any of the Rust parsers in our
> config (krb5, nfs, tftp, ntp, dhcp, ikev2) but do have SMB enabled so I
> believe will have the SMB2/3 parser.  I was not sure the default behavior
> in this case (as --dump-config had no values for the new Rust based
> parsers), so I did test disabling krb5, nfs, tftp, ntp, dhcp, and ikev2.
> We still had high percentages of drops in this case.  I plan to look into
> whether or not there is a way to disable just SMB2/3 with Rust enabled to
> see if that makes a difference.
> > - We use pcap capture mode with Myricom cards.  The driver version if
> not at the latest, though is only one patch version away from the latest.
> We tested updating to the latest version on one of our sensors and it had
> no effect.
> > - Suricata was compiled with rustc 1.30.1.  I did try upgrading to use
> Rust 1.31 but did not seem to have any effect.
> > - I compiled Suricata 4.1.2 without Rust and that looks to have
> positively affected this.  We had very little packet loss in this case.
> >
>
> Is it possible to share full stats.log from the two different runs
> (4.1.2 and 4.0.6) and any changes made to suricata.yaml ?
>
>
> --
> Regards,
> Peter Manev
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
> Conference: https://suricon.net
> Trainings: https://suricata-ids.org/training/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190208/074f5f10/attachment.html>