[Oisf-users] Tuning Suricata (2.0beta1) -- no rules and lots of packet loss

[Tritium Cat]

Thanks, I will give that a try after I work on filtering out all the
regular high volume flows I can find.  That has to be the problem.. it
makes sense now and I feel exceedingly stupid for not addressing it first.
(trying to analyze all traffic...)  Whenever we meet you are owed food and
drink, or shoot me your paypal.  At least I have a good sense of failure
conditions now. :>  I wonder about current DoS attack methodology on
multi-core IDS systems and the associated flow hashing algorithms... what
is the minimum volume required and how many flows launched to affect x out
of y cores on an IDS to cause partial blindness ?  (It is relative to the
core speed)  Does anyone see such behavior ?  I've not heard or read about
IDS evasion topics for a long time.  I can sleep now...

--TC



On Wed, Aug 14, 2013 at 11:24 PM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> You don't have enough memory, you need at least two gigs per core.  On
> my sensor I have three, so I run with deep ring buffers.  So, you need
> at least 96 gigs of RAM on your system (I see you have 64 from the top
> printout).  Try setting your ring size small (like 50000) or getting
> more memory.
>
> Did you restart irqbalance after you updated the driver?  I've had a
> similar issue in the past that was fixed by restarting irqbalance.
>
> IF you really have a limit of 32 RX queues, there is an easy fix for
> this.  Just set the 'threads' section in af-packet mode to '32' and it
> will use the first 32 cores only.  This will also fix your memory
> issues, as only 32 ring buffers will be allocated.
>
> - -Coop
>
> On 8/14/2013 11:03 PM, Tritium Cat wrote:
> > I tried all of your recommendations and performance is worse.  Short of
> > having the exact same server and single card there's nothing else I can
> > match exactly.
> >
> > ring-size=300000 exhausts memory and swap.  Even ring-size=100000 is
> taxing
> > on the system and consumes 56GB ram.
> >
> > Also updated ixgbe to latest version as recommended on the blog.  Not
> sure
> > if this is connected but oddly it seems now I am reaching a limit of 32
> > CPUs/queues with unused cores at 100% idle.  I think there are some
> errors
> > on the blog too with the ixgbe parameters and expectations of queues but
> > that is besides the point.  (FdirPballoc is for hardware filters and not
> > relevant here?)
> >
> > I cannot explain the feeling of how old this is becoming especially when
> > all others seem to have success.  I guess maybe the answer must be
> > filtering out lots of traffic however the traffic profile is definitely
> > less than the 1M to 1.5M pps mentioned in the blog.  (The blog server is
> > less powerful too?)
> >
> > For now I will just be happy with dropping 10-40% and restarting the IDS
> > daily, a half working IDS is better than no IDS :p
> >
> > Thanks for the detail it is still helpful.  Better luck tomorrow !
> >
> > --TC
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJSDHQlAAoJEKIFRYQsa8FW52cH/Ra9hCCkaILihWKhpIozi7Fh
> phjTjJckrsPjOAbtHmUvQmA/JY+D4rY3POyLWOgTsoeXdYOXqPs4dLkpSZtGwND0
> NLLz5IBDzQk5GRzqryAKbb9q3SXm+c4bOdc6ML/7sTVhfSNDHCjVygzynubHMVbz
> 6asVSeXUtiEj6OlCtOMlB2B6eJfeYb+lFwpIonK2JZeBI5EgSkhUPj0JNsZLrSsP
> Nu+eEFi37kG0YwoqgYvVcTOWkHRNLgd88e0DsaqraM7sVfJ3GGd11IL6aHwk8riF
> xLRhiFoyD5sCIHgB7yzEwHJtgHkRGkJ+JAmQ6tWGMkvZmx/i7uZG0cQgM6+TDQE=
> =lg89
> -----END PGP SIGNATURE-----
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130815/87e857ec/attachment-0002.html>