[Oisf-users] Tuning Suricata (2.0beta1) -- no rules and lots of packet loss

[Tritium Cat]

I tried all of your recommendations and performance is worse.  Short of
having the exact same server and single card there's nothing else I can
match exactly.

ring-size=300000 exhausts memory and swap.  Even ring-size=100000 is taxing
on the system and consumes 56GB ram.

Also updated ixgbe to latest version as recommended on the blog.  Not sure
if this is connected but oddly it seems now I am reaching a limit of 32
CPUs/queues with unused cores at 100% idle.  I think there are some errors
on the blog too with the ixgbe parameters and expectations of queues but
that is besides the point.  (FdirPballoc is for hardware filters and not
relevant here?)

I cannot explain the feeling of how old this is becoming especially when
all others seem to have success.  I guess maybe the answer must be
filtering out lots of traffic however the traffic profile is definitely
less than the 1M to 1.5M pps mentioned in the blog.  (The blog server is
less powerful too?)

For now I will just be happy with dropping 10-40% and restarting the IDS
daily, a half working IDS is better than no IDS :p

Thanks for the detail it is still helpful.  Better luck tomorrow !

--TC



On Wed, Aug 14, 2013 at 1:47 PM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> If you are monitoring a 10gig interface and using 'worker' mode, its
> critical you follow this guide exactly:
>
> > https://home.regit.org/2012/07/suricata-to-10gbps-and-beyond/
>
> And specifically, if a feature isn't set, don't set it.  AF_PACKET
> doesn't set a buffer by default in this configuration as you can see by
> checking the suricata.log file:
>
> > 14/8/2013 -- 11:00:39 - <Info> - all 16 packet processing threads, 3
> management threads initialized, engine started.
> > 14/8/2013 -- 11:00:39 - <Info> - AF_PACKET RX Ring params:
> block_size=32768 block_nr=26215 frame_size=1584 frame_nr=524300
> > 14/8/2013 -- 11:00:39 - <Info> - Using interface 'eth2' via socket 7
>
> I think the issue is you don't want a buffer in mmap mode as it will
> interfere with moving the packets directly into the RX Ring.  Basically,
> forcing a buffer breaks the 'zero copy' model.
>
> But again, as mentioned in all run modes there is a limit on the number
> of packets per second any single thread can process.  This is especially
> an issue with 'auto flow pinned mode' as flows will be pinned to a
> single core, so a single large flow can peg a core and cause a backlog
> and ultimately packet drops.
>
> I'm not sure if there is a fix for this (or how big an issue it is as
> properly sized box will drop less than 1% of packets).
>
> Some ideas I've had (which may be way off base)
>
> 1.  I think the linux kernel is still sending packets for large flows to
> the AF_PACKET RX Ring even after the stream limit has been reached and
> streams are no longer tracked.  If flows could be truncated once the
> stream depth limit is reached it should mitigate this issue in most cases.
>
> 2.  Allow a 'Ring emergency mode' where packets can be spooled to a
> second buffer if any of the AF_PACKET queues fill up.  This could even
> be a spool file, to avoid a OOM condition.
>
> - -Coop
>
> On 8/14/2013 11:43 AM, rmkml wrote:
> > Hi Tritium and Cooper,
> >
> > Maybe it's related to recently speak on list for new dns preproc cause
> > drop, I don't known if it possible to disable actually.
> > Maybe back to v1.4.4 for comparing ?
> >
> > Regards
> > @Rmkml
> >
>
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJSC+zjAAoJEKIFRYQsa8FW+VoIALT+vbKujpH62oxw0uJ9YPM6
> ibUjt5Vw2homkD73GdqIP2G6MePU+STk1aMAijv2UdVTIGv/QsL1jpS3m4DuG3f+
> Iu6GUT+CeLIlUjrg9GE1tWtbDoLiuh1MX3BtvI7mvsW2sZHtx5OLdrAaXNbL6Ccy
> sValT5g/D1L6CHvT9lTZkccQO6dLojLOz+yIQFVqv0dKNpF3ij1WLGn2knisK9AT
> 0mR8WXBDVOY34chx6YtHrYoYPOtFRQnN1TbtZoSi6rGKnj+5I8W5ed8+vllhn6pk
> sAZvgBaQifV6WVbev/6qnsLFgBZoBScFVZ0jlM4l47QuJin/orVhgg+MU3bgW2I=
> =OR37
> -----END PGP SIGNATURE-----
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130814/7edef121/attachment-0002.html>