[Oisf-users] af_packet and rss queue count

[Peter Manev]

On Thu, Jan 26, 2017 at 8:32 PM, Michał Purzyński
<michalpurzynski1 at gmail.com> wrote:
> GRO is not hardware. LRO would be.
>
> GRO kicks in later in the path and results with pretty significant savings of the amount of work kernel needs to do per packet, since packets are merged.
>
> Hence the word Generic there ;) it's basically merging packets before taking them out of the DMA area. And it is smarter, much smarter, about which packets should be merged. LRO merges everything in sight - GRO does not.
>
> LRO is gone from newest cards too.
>
>> On Jan 26, 2017, at 11:23 AM, Cooper F. Nelson <cnelson at ucsd.edu> wrote:
>>
>> I can confirm this behavior as well.  We are successfully extracting
>> files off the wire and confirming their integrity via cryptographic hash.

Is that for IPv4 and IPv6 alike? (when using multiple queues with the
ixgbe patch from RH - allowing for passing/enforcing the key with the
ethtool)


>>
>> I'm also using hardware offloading for reassembly via GRO.  This
>> reassembles TCP and UDP flows in kernel space and passes packets in
>> unified blocks of up to 64kb to user space.  So it's possible the packet
>> reordering issue is only a problem if you disable the hardware
>> offloading (which is on by default in all distros that I know of.)
>>
>> I'll note that using hardware offloading for checksum and stream
>> reassembly results in some pretty significant performance improvements
>> for suricata, particularly with hyperscan.  The hs matcher in seems to
>> benefit from working with bigger blocks of packet data (I suspect due to
>> streaming issues with the suricata implementation).
>>
>> I'm seeing about 10X (wow!) performance improvements over suricata 3.0
>> and 5X over 3.l without hardware offloading.
>>
>> -Coop
>>
>>> On 1/26/2017 7:05 AM, erik clark wrote:
>>> If you set a symmetric queue (posted elsewhere in some thread), I believe
>>> you do not see issues with reordering. We have seen output from pf_ring and
>>> af_packet as being near identical in logged connections, events, so unless
>>> pf_ring is also mangling connectings the same way as if_packet/AF_PACKET,
>>> we can assume this is a non-issue. Packet loss is actually substantually
>>> lower for both suricata and bro using a symmetric key, proper udp/tcp
>>> hashing, and af_packet compared to pf_ring on identical hardware. It comes
>>> at cost of cpu over memory however.
>>
>>
>> --
>> Cooper Nelson
>> Network Security Analyst
>> UCSD ITS Security Team
>> cnelson at ucsd.edu x41042
>>
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users



-- 
Regards,
Peter Manev