[Oisf-users] High Suricata capture.kernel_drops

[fatema bannatwala]

Hi Sean,

Looks like it helped some. Modified the cpu-set settings as you mentioned,
and now loss is around 4-5% [capture.kernel_packets: 685173701,
capture.kernel_drops: 8692212 ]

I will see if I can recompile Suricata with Hyper-Scan and see if the
kernel_drops reduce to a lower number.

Thanks!
Fatema.


On Thu, Jul 12, 2018 at 9:12 AM, Cloherty, Sean E <scloherty at mitre.org>
wrote:

> So looking at the docs – for runmode workers these are the two affinity
> settings which you need to concern yourself with – and the worker-cpu set
> is the critical one.
>
>
>
> management-cpu-set - used *for* management (example - flow.managers, flow.
> recyclers)
>
> worker-cpu-set - used *for* receive,streamtcp,decode,
> detect,output(logging),respond/reject
>
>
>
> What you want to do is to use that list in node 0 as the ones to use for
> workers and then pick any two for the management CPU from node one –
>
>
>
> So
>
>
>
>
>
>   cpu-affinity:
>
>     - management-cpu-set:
>
>         cpu: [ 37,39 ]  # include only these cpus in affinity settings
>
>     - receive-cpu-set:
>
>         cpu: [ 0 ]  # include only these cpus in affinity settings
>
>     - worker-cpu-set:
>
>         cpu: [ 4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38 ]
>
>
>
>         mode: "exclusive"
>
>         # Use explicitly 3 threads and don't compute number by using
>
>         # detect-thread-ratio variable:
>
>          threads: 18
>
>
>
>
>
>
>
> *From:* fatema bannatwala [mailto:fatema.bannatwala at gmail.com]
> *Sent:* Wednesday, July 11, 2018 15:03 PM
>
> *To:* Cloherty, Sean E <scloherty at mitre.org>
> *Cc:* oisf-users at lists.openinfosecfoundation.org
> *Subject:* Re: [Oisf-users] High Suricata capture.kernel_drops
>
>
>
> Hi Sean.
>
>
>
> I have two NUMA nodes, and Node 0 is the NICs NUMA node:
>
>
>
> NUMA node0 CPU(s):     0,2,4,6,8,10,12,14,16,18,20,
> 22,24,26,28,30,32,34,36,38
>
> NUMA node1 CPU(s):     1,3,5,7,9,11,13,15,17,19,21,
> 23,25,27,29,31,33,35,37,39
>
>
>
> $ cat /sys/class/net/em1/device/numa_node
>
> 0
>
>
>
> So does that mean that I can assign only threads from NUMA node0 to
> the management-cpu-set and worker-cpu-set, as it's the NICs NUMA node?
>
>
>
> I am not able to figure out from Septun doc that what threads/cores would
> be pinned to which set in cpu-affinity, as you suggested earlier, hence
> went with "all" in worker and cpu sets by default.
>
>
>
> I will try to update the drivers for the NICs next.
>
>
>
> As for HS, I didn't know about it before, and now that I have already
> compiled Suricata from source, and do $suricata --buil-info, if shows
> "Hyperscan support: no".
>
> Hence assuming that I have to recompile suricata again to get that
> enabled, which I would not like to do as of now.
>
>
>
>
>
> Thanks,
>
> Fatema.
>
>
>
>
>
>
>
>
>
> On Wed, Jul 11, 2018 at 2:19 PM, Cloherty, Sean E <scloherty at mitre.org>
> wrote:
>
> First get the NUMA node for the CPUs – lscpu should provide that in the
> last two lines of the output.
>
>
>
> Find your NICs NUMA node 1st  and go from there for affinity settings  cat
> /sys/class/net/em1/device/numa_node
>
>
>
>
>
>
>
> Update the drivers for the NIC - https://downloadcenter.intel.
> com/download/24411/Intel-Network-Adapter-Driver-for-
> PCIe-40-Gigabit-Ethernet-Network-Connections-Under-Linux-?product=82947
>
>
>
> (Just remember that you will need to repeat this after any kernel updates)
>
>
>
>
>
>
>
>
>
>
>
> *From:* fatema bannatwala [mailto:fatema.bannatwala at gmail.com]
> *Sent:* Wednesday, July 11, 2018 13:55 PM
> *To:* Cloherty, Sean E <scloherty at mitre.org>
> *Cc:* oisf-users at lists.openinfosecfoundation.org
> *Subject:* Re: [Oisf-users] High Suricata capture.kernel_drops
>
>
>
> Hi Sean,
>
>
>
> Thanks for some quick points and recommendations.
>
> I will work through those, and see if it helps.
>
>
>
> The documentation refers the tuning assuming two NICs p1p1 and p1p3, which
> was getting me confused, as I only have single NIC with 20 cores and 40
> online threads, so was struggling to set the config options right in the
> yaml file for cpu_affinity. I will try the hard coded method instead of all
> and see if it helps.
>
>
>
> Fatema.
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20180712/702b53e4/attachment.html>