[Oisf-users] High Suricata capture.kernel_drops

[fatema bannatwala]

Thanks Peter, Yeah next thing I would try is to recompile Suri with HS.
will give it a shot.

On Thu, Jul 12, 2018 at 2:43 AM, Peter Manev <petermanev at gmail.com> wrote:

> On Thu, Jul 12, 2018 at 9:06 AM, Peter Manev <petermanev at gmail.com> wrote:
> >
> >
> > On 11 Jul 2018, at 22:02, fatema bannatwala <fatema.bannatwala at gmail.com
> >
> > wrote:
> >
> > Hi Sean.
> >
> > I have two NUMA nodes, and Node 0 is the NICs NUMA node:
> >
> > NUMA node0 CPU(s):
> > 0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38
> > NUMA node1 CPU(s):
> > 1,3,5,7,9,11,13,15,17,19,21,23,25,27,29,31,33,35,37,39
> >
> > $ cat /sys/class/net/em1/device/numa_node
> > 0
> >
> > So does that mean that I can assign only threads from NUMA node0 to the
> > management-cpu-set and worker-cpu-set, as it's the NICs NUMA node?
> >
> >
> >
> > There are two ways you can go by here (the way I see it) but I think the
> > easiest from administrative point (to at least try out fast) might be to
> > just use numactl (including membind if needed) to make sure Suri is using
> > the NICs local NUMA
> >
> > I am not able to figure out from Septun doc that what threads/cores
> would be
> > pinned to which set in cpu-affinity, as you suggested earlier, hence went
> > with "all" in worker and cpu sets by default.
> >
> > I will try to update the drivers for the NICs next.
> >
> >
> > That is always recommended !
> >
> > As for HS, I didn't know about it before, and now that I have already
> > compiled Suricata from source, and do $suricata --buil-info, if shows
> > "Hyperscan support: no".
> > Hence assuming that I have to recompile suricata again to get that
> enabled,
> > which I would not like to do as of now.
> >
> >
> > There is an example here of how to compile Hyperscan on Ubuntu from the
> > docs-
> > https://suricata.readthedocs.io/en/latest/performance/
> hyperscan.html?highlight=Hyperscan
> >
> > Thanks
> >
>
>
> Since we are on the subject - this example should get you the latest
> Suricata with hyperscan (you may want to update the boost version
> though ) on RedHat/CentOS-
> https://pastebin.com/iSKK53Dw
>
> Hope it helps!
>
> >
> > Thanks,
> > Fatema.
> >
> >
> >
> >
> > On Wed, Jul 11, 2018 at 2:19 PM, Cloherty, Sean E <scloherty at mitre.org>
> > wrote:
> >>
> >> First get the NUMA node for the CPUs – lscpu should provide that in the
> >> last two lines of the output.
> >>
> >>
> >>
> >> Find your NICs NUMA node 1st  and go from there for affinity settings
> cat
> >> /sys/class/net/em1/device/numa_node
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Update the drivers for the NIC -
> >> https://downloadcenter.intel.com/download/24411/Intel-
> Network-Adapter-Driver-for-PCIe-40-Gigabit-Ethernet-
> Network-Connections-Under-Linux-?product=82947
> >>
> >>
> >>
> >> (Just remember that you will need to repeat this after any kernel
> updates)
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> From: fatema bannatwala [mailto:fatema.bannatwala at gmail.com]
> >> Sent: Wednesday, July 11, 2018 13:55 PM
> >> To: Cloherty, Sean E <scloherty at mitre.org>
> >> Cc: oisf-users at lists.openinfosecfoundation.org
> >> Subject: Re: [Oisf-users] High Suricata capture.kernel_drops
> >>
> >>
> >>
> >> Hi Sean,
> >>
> >>
> >>
> >> Thanks for some quick points and recommendations.
> >>
> >> I will work through those, and see if it helps.
> >>
> >>
> >>
> >> The documentation refers the tuning assuming two NICs p1p1 and p1p3,
> which
> >> was getting me confused, as I only have single NIC with 20 cores and 40
> >> online threads, so was struggling to set the config options right in the
> >> yaml file for cpu_affinity. I will try the hard coded method instead of
> all
> >> and see if it helps.
> >>
> >>
> >>
> >> Fatema.
> >
> >
> > _______________________________________________
> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> > Site: http://suricata-ids.org | Support: http://suricata-ids.org/
> support/
> > List: https://lists.openinfosecfoundation.org/
> mailman/listinfo/oisf-users
> >
> > Conference: https://suricon.net
> > Trainings: https://suricata-ids.org/training/
>
>
>
> --
> Regards,
> Peter Manev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20180712/f64fc062/attachment.html>