[Oisf-users] Out of band 10Gb Suricata

[Christophe Vandeplas]

Just to be sure, how many queues do your vnics have?

You can easily check this with :
$ cat /proc/interrupts  | fgrep eth

This is important as the multiple threads of af_packet will need to
grab the packets from each queue. If you only have one queue in the
NIC, then only one thread can take care of reading these queues and
will jump to 100%, while the other receiving threads of af_packet will
do nothing.
Depending on the af_packet configuration this might even be worse.
If you're only having one queue, make sure af_packet is set to
"cluster-type: cluster_flow". You should see a considerable
improvement.

I had this problem with cheap commodity hardware as explained in my
post: http://christophe.vandeplas.com/2013/11/suricata-capturekerneldrops-caused-by.html

However, trying to get 10 Gbps with visualized hardware is perhaps a
little bit optimistic.



On 15 October 2015 at 02:48, Brian Hennigar <bhennigar at gmail.com> wrote:
> I think having 8 cores really is my issue. With no rules enabled, I'm still
> getting drops with af-packet although it is better.
>
> capture.kernel_drops      | AFPacketeth71             | 19611
> capture.kernel_drops      | AFPacketeth72             | 23942
> capture.kernel_drops      | AFPacketeth73             | 964
> capture.kernel_drops      | AFPacketeth74             | 14720
> capture.kernel_drops      | AFPacketeth75             | 0
> capture.kernel_drops      | AFPacketeth76             | 0
> capture.kernel_drops      | AFPacketeth77             | 0
> capture.kernel_drops      | AFPacketeth78             | 19216
>
>
> Thanks again for all of the help!  There's still much I need to learn about
> tuning Suricata.
>
> On Wed, Oct 14, 2015 at 8:23 PM, Brian Hennigar <bhennigar at gmail.com> wrote:
>>
>> I've looked into pf_ring.  vmxnet3 isn't supported by pf_ring and the
>> E1000 interface choice by ESXi is only 1gb which wouldn't work for 10Gb.
>> vmxnet3 supports 10gb.   Passing the interface directly through to the VM
>> might be an option but not ideal.
>>
>> I'm just starting on configuring it to use workers and af-packet.
>>
>> Thanks,
>> Brian
>>
>> On Wed, Oct 14, 2015 at 8:19 PM, Cooper F. Nelson <cnelson at ucsd.edu>
>> wrote:
>>>
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>>
>>> I didn't notice that either.  All my deployments are bare metal, so I
>>> don't know well that will work.  If the NICs support recieve-side
>>> scaling everything should work well.
>>>
>>> - -Coop
>>>
>>> On 10/14/2015 2:38 PM, Chris Wakelin wrote:
>>> > Also it seems you're using virtual NICs ("vmxnet3")?
>>> >
>>> > Depending on which interface type you use and whether it supports
>>> > AFPacket, you might need something like PF_RING ZC
>>> >
>>> > (http://www.ntop.org/products/packet-capture/pf_ring/pf_ring-zc-zero-copy/).
>>> >
>>> > Best Wishes,
>>> > Chris
>>>
>>>
>>> - --
>>> Cooper Nelson
>>> Network Security Analyst
>>> UCSD ACT Security Team
>>> cnelson at ucsd.edu x41042
>>> -----BEGIN PGP SIGNATURE-----
>>> Version: GnuPG v2.0.17 (MingW32)
>>>
>>> iQEcBAEBAgAGBQJWHuLnAAoJEKIFRYQsa8FWrvsH+wRBuQfoKKRFamD2qLXzuVUX
>>> JR9IeY22XRfoCrMGjD0h7Yic0fkt6DPLng/z4rmn0brgCjkSxYukdnhvHUyZzPTi
>>> lkDdkEevXGcA1CDqw2+ZyQsqRao2GO6EfOJ7pvH1QIL4rG7Aa2Nl+PVL1La2hq8k
>>> 8OEiTZr4/nGs7cUOGyFLooKgPh5lOeEjhRdkO0QueYK46IgWClRg/haIQEBT/YUK
>>> QbedoaAViBbQti2sWYbNi0MIZtWoELNuJxG+79aKEQkWWUbztbej29guX+mafojA
>>> el9JK1BuEnHz/VdIp+e1XCc39mur5qJMS47vwlVDD9IMFFfi2o69+ZdD5SiiiuQ=
>>> =2PmI
>>> -----END PGP SIGNATURE-----
>>
>>
>
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net