[Oisf-users] SEPTun and memory usage

[erik clark]

ethtool -C em1 rx-usecs 1 adaptive-rx off
ethtool -G em1 rx 4096 tx 4096
for x in tso gro lro gso tx tx sg; do ethtool -K em1 $x off; done
ethtool -N em1 rx-flow-hash tcp4 sdfn
ethtool -N em1 rx-flow-hash udp4 sdfn

This is on El Repo kernel 4.12.0-1.el7.elrepo.x86_64.

tpacket-v3 is set to yes. (obviously mmap is true).

Hope this helps.


On Sat, Jul 15, 2017 at 5:31 PM, Peter Manev <petermanev at gmail.com> wrote:

> On Fri, Jul 14, 2017 at 3:44 PM, erik clark <philosnef at gmail.com> wrote:
> > Sean, thanks for this. I think I had actually googled up your paper
> before.
> > :) This memory calculator is very nice.
> >
> > I noticed that when I squashed rss queue to 1 on a 4.12 kernel, I went
> from
> > less than .05% packet loss to ~9% packet loss. Any idea why that might
> have
>
> If the only change you did was to switch to 1 RSS then it will be
> interesting to understand why.
> But it will be good to understand your NIC settings first in terms of
> ring descriptor size, coalescence, what was the CPU cache misses
> before and after etc..
>
> > occured? I have read some conflicting things about rss queues depending
> on
> > kernel version, namely this bit:
> >
> > AF_PACKET: 1 RSS queue and stay on kernel <=4.2 or make sure you have
> >>=4.4.16, >=4.6.5 or >=4.7. Exception: if RSS is symmetric cluster-type
> > 'cluster_qm' can be used to bind Suricata to the RSS queues. Disable NIC
> > offloading except the rx/tx csum.
> >
> > from
> > https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Packet_
> Capture
> >
> > Thank you for all the help tuning out memory issues suri. My goal is to
> try
> > and get packet loss below .01%. Heres for trying!
> >
> >
> > On Fri, Jul 14, 2017 at 9:23 AM, Cloherty, Sean E <scloherty at mitre.org>
> > wrote:
> >>
> >> I’ve post this earlier and hope that this can be useful.
> >>
> >>
> >>
> >> If you are using AF-PACKET (and why wouldn't you), the attached
> >> spreadsheet may help.  It is derived from Peter Manev's highly detailed
> >> review of various configuration options and their affect on memory
> >> utilization.
> >> http://pevma.blogspot.com/2015/10/suricata-with-
> afpacket-memory-of-it-all.html
> >>
> >>
> >>
> >> I began creating this during a Suricata training class so I could save
> >> time when testing different configurations.  Peter has reviewed it for
> >> accuracy and correct nomenclature.  I hope that it will be of some use
> to
> >> the community.
> >>
> >>
> >>
> >> Sean Cloherty
> >>
> >>
> >>
> >>
> >>
> >> From: Oisf-users
> >> [mailto:oisf-users-bounces at lists.openinfosecfoundation.org] On Behalf
> Of
> >> erik clark
> >> Sent: Thursday, July 13, 2017 07:58 AM
> >> To: Open Information Security Foundation
> >> <oisf-users at lists.openinfosecfoundation.org>
> >> Subject: Re: [Oisf-users] SEPTun and memory usage
> >>
> >>
> >>
> >> All, trying to find out who has worked with the SEPTun document that can
> >> provide some insight into how much memory they are using to sniff
> traffic.
> >>
> >>
> >>
> >> We (were) using 8 threads with 200 gigs of ram on a 2.5 Gb/s link. Until
> >> earlier this week, our drop rate was ~2%. I just moved up to 16 threads,
> >> still at 200 gigs of ram, since our throughput moved up a bit to
> ~3.1Gb/s
> >> and saw a 12% drop rate.
> >>
> >>
> >>
> >> We have 72 cores to work with, and 200 gigs of ram, and just moved to a
> >> 4.4 kernel from a modified 3.10 kernel. What seems reasonable on this
> kind
> >> of hardware? We are limited to an 82598 ixgbe interface with a single
> link.
> >
> >
> >
> > _______________________________________________
> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> > Site: http://suricata-ids.org | Support: http://suricata-ids.org/
> support/
> > List: https://lists.openinfosecfoundation.org/
> mailman/listinfo/oisf-users
> >
> > Conference: https://suricon.net
> > Trainings: https://suricata-ids.org/training/
>
>
>
> --
> Regards,
> Peter Manev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20170717/07644b02/attachment-0002.html>