[Oisf-users] Question about cpu-affinity

[Cooper F. Nelson]

On 3/2/2018 3:03 AM, Eric Leblond wrote:
>> Or maybe allow defining named cpu sets and allow assigning those to
>> af-packet interface configs:
>>
>> - cpu-set
>>   name: af-packet-eth0
>>   cpu: [ 0, 2, 4, 6, 8, 10, 12, 14]
>>   mode: "exclusive"
>> - cpu-set
>>   name: af-packet-eth1
>>   cpu: [1, 3, 5, 7, 9, 11, 13, 15 ]
>>   mode: "exclusive"
>>
>>
>>
>> af-packet:
>>   - interface: eth0
>>     cluster-id: 99
>>     cpu-set: "af-packet-eth0"
>>   - interface: eth1
>>     cluster-id: 98
>>     cpu-set: "af-packet-eth1"
> I like this second proposal better. From what I've seen a few packet
> capture APIs are using the numa node in the capture params, maybe we
> could combined both approach.

I'll vote for this as approach as well.

For some context, I've just got done deploying a 64 core AMD Piledriver
suricata system.  Dual 10 gig Intel NICs (ixgbe driver).

I based my build on Peter Manev's SEPTUN guide, however since AMD
doesn't support the same caching architecture that Intel does
(specifically DCA and DDIO) the performance wasn't as expected.  Using a
single RSS queue simply doesn't work, the core is pegged @100% with
significant packet loss.

What I ended up doing was creating a hybrid deployment that used my
standard HPC server build, 4 RSS queues/cores per NIC/NUMA node and
cluster_flow to have suri distribute flows to the remaining 56 cores in
software.  The reason I wanted to interleave the detect threads was to
leverage the AMD Hypertransport bus to evenly distribute the load from
both NICs over the whole system.

-- 
Cooper Nelson
Network Security Analyst
UCSD ITS Security Team
cnelson at ucsd.edu x41042

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20180304/d1b17932/attachment-0002.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20180304/d1b17932/attachment-0002.sig>