[Oisf-users] af_packet and rss queue count

[Peter Manev]

On Wed, May 31, 2017 at 6:30 PM, erik clark <philosnef at gmail.com> wrote:
> Sadly, we have migrated to el repo 4.4.69 lt kernels. It was too difficult
> to maintain patches for the stock kernel modules on rhel7. :/

We were just discussing about that with Victor in the irc the other day...

So it seems to be really kernel version and patches specific  - as you
would need to double check if everything works specifically for that
set up(key/patch/kernel/nic driver version) every kernel/nic  upgrade
- seems a lot of overhead?

>
>
> On Wed, May 31, 2017 at 11:15 AM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:
>>
>> It looks like this doesn't work anymore on current kernel revisions.
>>
>> Does anyone have specific "known good" kernel/suricata revision that
>> this patch works with (so I can revert to it?)
>>
>> Thanks,
>> -Coop
>>
>> On 12/29/2016 3:20 AM, erik clark wrote:
>> > Yes, thats the patch at marc.info in the previous email. Please note the
>> > submitter of that patch. :)
>> >
>> > On Thu, Dec 29, 2016 at 4:10 AM, Peter Manev <petermanev at gmail.com>
>> > wrote:
>> >
>> >>
>> >>
>> >> On 28 Dec 2016, at 15:00, erik clark <philosnef at gmail.com> wrote:
>> >>
>> >> (patch for ixgbe)
>> >> http://marc.info/?l=linux-netdev&m=148181173415107&w=2
>> >>
>> >> (patch to ixgbe/src/kompat.h since it wont compile on rhel7.3 due to
>> >> kernel version issues. None of this is pf_ring specific)
>> >> https://raw.githubusercontent.com/ntop/PF_RING/41b6eb733e295
>> >> c91375b135d2816dbdd09b4b548/drivers/intel/ixgbe/ixgbe-4.1.
>> >> 5-zc/src/kcompat.h
>> >>
>> >>
>> >> In a different mail you mentioned that there is also a patch from Red
>> >> Hat
>> >> with regards to at-packet not working on this kernel version - is this
>> >> fixed/working now?
>> >>
>> >>
>> >> (ethtool arguments and irq setting)
>> >>
>> >> ethtool -C em1 rx-usecs 1 adaptive-rx off
>> >> ethtool -G em1 rx 4096 tx 4096
>> >> for x in tso gro lro gso rx tx sg; do ethtool -K em1 $x off; done
>> >> /opt/src/ixgbe-4.4.6/scripts/set_irq_affinity em1
>> >> ethtool -X em1 hkey 6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:
>> >> 6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:
>> >> 6d:5a:6d:5a:6d:5a:6d:5a:6d:5a
>> >> ethtool -N em1 rx-flow-hash tcp4 sdfn
>> >> ethtool -N em1 rx-flow-hash udp4 sdfn
>> >>
>> >>
>> >> Can you confirm all flows are good for tcp/udp/4/6?
>> >> (And how do you confirm it?)
>> >>
>> >>
>> >>
>> >>
>> >> I have 64 cores, and set_irq_affinity pulls a full set of 63 rss
>> >> queues.
>> >> Thanks!
>> >>
>> >> On Sat, Dec 24, 2016 at 1:28 PM, erik clark <philosnef at gmail.com>
>> >> wrote:
>> >>
>> >>> I will post that on Wednesday when I get back to work. Its 6? ethtool
>> >>> statements, and 2 ixgbe patches (for rhel7 at least). Anything else
>> >>> should
>> >>> be just 1 patch. This is running the 4.4.6 ixgbe released from intel
>> >>> directly. This has worked for the most recent 3 kernels under RHEL7.
>> >>>
>> >>> On Sat, Dec 24, 2016 at 12:51 PM, Peter Manev <petermanev at gmail.com>
>> >>> wrote:
>> >>>
>> >>>>
>> >>>>
>> >>>>> On 24 Dec 2016, at 18:22, erik clark <philosnef at gmail.com> wrote:
>> >>>>>
>> >>>>> I have seen several places commenting that you should set the RSS
>> >>>> queue to 1. However, when examining af_packet with Bro, a patch
>> >>>> released
>> >>>> from Redhat for the ixgbe kernel module, and some ethtool tweaking,
>> >>>> we have
>> >>>> found that (for Bro at least) running a full 63 (we have 54 cores)
>> >>>> RSS
>> >>>> queues vastly improves performance, and keeps state intact across
>> >>>> sessions.
>> >>>>>
>> >>>>> Based on this update, which fixes the broken implementation of
>> >>>>> setting
>> >>>> a symmetric hash in the hardware of the card
>> >>>>
>> >>>> Can you please share a bit in a bit more detail-
>> >>>> Which ixgbe/kernel version that is ?
>> >>>> Which patch is it ?
>> >>>> What is the ethtool tweaking procedure?
>> >>>>
>> >>>> Thanks
>> >>>>
>> >>>>> (again ixgbe, not tested with i40e), is it still necessary to run
>> >>>>> one
>> >>>> queue? If so, you can't run Bro and Suri on the same box with
>> >>>> af_packet and
>> >>>> get equivalent performance out of both tools. Having run Suri with 63
>> >>>> queues for a week now, it seems performance is considerably better
>> >>>> than
>> >>>> with pf_ring, and I can not find any unusual behavior in my alerts...
>> >>>>>
>> >>>>> _______________________________________________
>> >>>>> Suricata IDS Users mailing list:
>> >>>>> oisf-users at openinfosecfoundation.org
>> >>>>> Site: http://suricata-ids.org | Support:
>> >>>> http://suricata-ids.org/support/
>> >>>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/ois
>> >>>> f-users
>> >>>>
>> >>>
>> >>>
>> >>
>> >
>> >
>> >
>> > _______________________________________________
>> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> > Site: http://suricata-ids.org | Support:
>> > http://suricata-ids.org/support/
>> > List:
>> > https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> >
>>
>>
>> --
>> Cooper Nelson
>> IT Security - Information Technology Services
>> University of California San Diego
>> (858) 534-6487 - cnelson at ucsd.edu
>> https://cybersecurity.ucsd.edu
>>
>



-- 
Regards,
Peter Manev