[Oisf-users] af_packet and rss queue count

[erik clark]

(patch for ixgbe)
http://marc.info/?l=linux-netdev&m=148181173415107&w=2

(patch to ixgbe/src/kompat.h since it wont compile on rhel7.3 due to kernel
version issues. None of this is pf_ring specific)
https://raw.githubusercontent.com/ntop/PF_RING/
41b6eb733e295c91375b135d2816dbdd09b4b548/drivers/intel/
ixgbe/ixgbe-4.1.5-zc/src/kcompat.h


(ethtool arguments and irq setting)

ethtool -C em1 rx-usecs 1 adaptive-rx off
ethtool -G em1 rx 4096 tx 4096
for x in tso gro lro gso rx tx sg; do ethtool -K em1 $x off; done
/opt/src/ixgbe-4.4.6/scripts/set_irq_affinity em1
ethtool -X em1 hkey 6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:
6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:6d:5a:
6d:5a:6d:5a:6d:5a:6d:5a:6d:5a
ethtool -N em1 rx-flow-hash tcp4 sdfn
ethtool -N em1 rx-flow-hash udp4 sdfn


I have 64 cores, and set_irq_affinity pulls a full set of 63 rss queues.
Thanks!

On Sat, Dec 24, 2016 at 1:28 PM, erik clark <philosnef at gmail.com> wrote:

> I will post that on Wednesday when I get back to work. Its 6? ethtool
> statements, and 2 ixgbe patches (for rhel7 at least). Anything else should
> be just 1 patch. This is running the 4.4.6 ixgbe released from intel
> directly. This has worked for the most recent 3 kernels under RHEL7.
>
> On Sat, Dec 24, 2016 at 12:51 PM, Peter Manev <petermanev at gmail.com>
> wrote:
>
>>
>>
>> > On 24 Dec 2016, at 18:22, erik clark <philosnef at gmail.com> wrote:
>> >
>> > I have seen several places commenting that you should set the RSS queue
>> to 1. However, when examining af_packet with Bro, a patch released from
>> Redhat for the ixgbe kernel module, and some ethtool tweaking, we have
>> found that (for Bro at least) running a full 63 (we have 54 cores) RSS
>> queues vastly improves performance, and keeps state intact across sessions.
>> >
>> > Based on this update, which fixes the broken implementation of setting
>> a symmetric hash in the hardware of the card
>>
>> Can you please share a bit in a bit more detail-
>> Which ixgbe/kernel version that is ?
>> Which patch is it ?
>> What is the ethtool tweaking procedure?
>>
>> Thanks
>>
>> > (again ixgbe, not tested with i40e), is it still necessary to run one
>> queue? If so, you can't run Bro and Suri on the same box with af_packet and
>> get equivalent performance out of both tools. Having run Suri with 63
>> queues for a week now, it seems performance is considerably better than
>> with pf_ring, and I can not find any unusual behavior in my alerts...
>> >
>> > _______________________________________________
>> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> > Site: http://suricata-ids.org | Support: http://suricata-ids.org/suppor
>> t/
>> > List: https://lists.openinfosecfoundation.org/mailman/listinfo/
>> oisf-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20161228/5941c2e3/attachment-0002.html>