[Oisf-users] Out of band 10Gb Suricata

[Peter Manev]

On Thu, Oct 15, 2015 at 6:50 AM, Christophe Vandeplas
<christophe at vandeplas.com> wrote:
> Just to be sure, how many queues do your vnics have?
>
> You can easily check this with :
> $ cat /proc/interrupts  | fgrep eth
>
> This is important as the multiple threads of af_packet will need to
> grab the packets from each queue. If you only have one queue in the
> NIC, then only one thread can take care of reading these queues and
> will jump to 100%, while the other receiving threads of af_packet will
> do nothing.
> Depending on the af_packet configuration this might even be worse.
> If you're only having one queue, make sure af_packet is set to
> "cluster-type: cluster_flow". You should see a considerable
> improvement.
>
> I had this problem with cheap commodity hardware as explained in my
> post: http://christophe.vandeplas.com/2013/11/suricata-capturekerneldrops-caused-by.html
>
> However, trying to get 10 Gbps with visualized hardware is perhaps a
> little bit optimistic.
>

My view as well in terms of accuracy.
I have not yet seen (in my experience anyway) a hypervisor capable
traffic mirroring without doing some sort of negative interference
impacting the IDS/IPS inspection in a virtual environment - aka
packets reordered, offloading enabled (sometimes hardcoded and not
configurable) etc.... that requires a lot of time to investigate and
fix (if possible).

I must agree it is much easier to fire up a virtual machine than to
procure HW within  corporate policy regulations and approved suppliers
- both in terms of time and process.

>
>
> On 15 October 2015 at 02:48, Brian Hennigar <bhennigar at gmail.com> wrote:
>> I think having 8 cores really is my issue. With no rules enabled, I'm still
>> getting drops with af-packet although it is better.
>>
>> capture.kernel_drops      | AFPacketeth71             | 19611
>> capture.kernel_drops      | AFPacketeth72             | 23942
>> capture.kernel_drops      | AFPacketeth73             | 964
>> capture.kernel_drops      | AFPacketeth74             | 14720
>> capture.kernel_drops      | AFPacketeth75             | 0
>> capture.kernel_drops      | AFPacketeth76             | 0
>> capture.kernel_drops      | AFPacketeth77             | 0
>> capture.kernel_drops      | AFPacketeth78             | 19216
>>
>>
>> Thanks again for all of the help!  There's still much I need to learn about
>> tuning Suricata.
>>
>> On Wed, Oct 14, 2015 at 8:23 PM, Brian Hennigar <bhennigar at gmail.com> wrote:
>>>
>>> I've looked into pf_ring.  vmxnet3 isn't supported by pf_ring and the
>>> E1000 interface choice by ESXi is only 1gb which wouldn't work for 10Gb.
>>> vmxnet3 supports 10gb.   Passing the interface directly through to the VM
>>> might be an option but not ideal.
>>>
>>> I'm just starting on configuring it to use workers and af-packet.
>>>
>>> Thanks,
>>> Brian
>>>
>>> On Wed, Oct 14, 2015 at 8:19 PM, Cooper F. Nelson <cnelson at ucsd.edu>
>>> wrote:
>>>>
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>>
>>>> I didn't notice that either.  All my deployments are bare metal, so I
>>>> don't know well that will work.  If the NICs support recieve-side
>>>> scaling everything should work well.
>>>>
>>>> - -Coop
>>>>
>>>> On 10/14/2015 2:38 PM, Chris Wakelin wrote:
>>>> > Also it seems you're using virtual NICs ("vmxnet3")?
>>>> >
>>>> > Depending on which interface type you use and whether it supports
>>>> > AFPacket, you might need something like PF_RING ZC
>>>> >
>>>> > (http://www.ntop.org/products/packet-capture/pf_ring/pf_ring-zc-zero-copy/).
>>>> >
>>>> > Best Wishes,
>>>> > Chris
>>>>
>>>>
>>>> - --
>>>> Cooper Nelson
>>>> Network Security Analyst
>>>> UCSD ACT Security Team
>>>> cnelson at ucsd.edu x41042
>>>> -----BEGIN PGP SIGNATURE-----
>>>> Version: GnuPG v2.0.17 (MingW32)
>>>>
>>>> iQEcBAEBAgAGBQJWHuLnAAoJEKIFRYQsa8FWrvsH+wRBuQfoKKRFamD2qLXzuVUX
>>>> JR9IeY22XRfoCrMGjD0h7Yic0fkt6DPLng/z4rmn0brgCjkSxYukdnhvHUyZzPTi
>>>> lkDdkEevXGcA1CDqw2+ZyQsqRao2GO6EfOJ7pvH1QIL4rG7Aa2Nl+PVL1La2hq8k
>>>> 8OEiTZr4/nGs7cUOGyFLooKgPh5lOeEjhRdkO0QueYK46IgWClRg/haIQEBT/YUK
>>>> QbedoaAViBbQti2sWYbNi0MIZtWoELNuJxG+79aKEQkWWUbztbej29guX+mafojA
>>>> el9JK1BuEnHz/VdIp+e1XCc39mur5qJMS47vwlVDD9IMFFfi2o69+ZdD5SiiiuQ=
>>>> =2PmI
>>>> -----END PGP SIGNATURE-----
>>>
>>>
>>
>>
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net



-- 
Regards,
Peter Manev