[Oisf-users] Question about cpu-affinity

[Victor Julien]

On 02-03-18 11:48, Victor Julien wrote:
> On 02-03-18 11:35, Eric Leblond wrote:
>> Hi Cooper,
>>
>> On Thu, 2018-03-01 at 08:12 -0800, Cooper F. Nelson wrote:
>>> Hi Eric,
>>>
>>> I hope you don't mind me copying you directly, as I saw you were the
>>> author of the threading/cpu-affinity module.
>>
>> That was long ago ;)
>>
>>> I'm wondering if it is currently possible when using multiple
>>> interfaces
>>> to 'interleave' the threads so that they are evenly distributed
>>> across
>>> all cores.  For example, consider a system with 16 cores and two
>>> interfaces, configured with 8 threads each.
>>>
>>> Can you do something like this?
>>>
>>>   - detect-cpu-set
>>>
>>>         cpu: [ 0, 2, 4, 6, 8, 10, 12, 14, 1, 3, 5, 7, 9, 11, 13, 15 ]
>>>
>>>         mode: "exclusive"
>>>
>>> The idea is we want the first 8 threads allocated to 0-14, even cores
>>> only.  The next 8 threads being allocated to 1-16, odd cores only. 
>>> Lets
>>> assume we are using RSS in this example.
>>>
>>> I had a look at the code and while it appears that this how you have
>>> implemented it, I'm not 100% certain, so I wanted to verify.
>>
>> I don't think this will work. The CPU set is an unordered. Suricata
>> build the CPU set as a map then it start by 0, get next CPU in the set
>> (testing if 0 is in then switching to 1 and checking if it is in the
>> set).
>>
>> So with current code, the only way to do what you want is to have a
>> ["all"] CPU set in exclusive mode in the affinity and an ugly af-packet 
>> configuration like:
>>
>>   af-packet:
>>     - interface: eth0
>>       threads: 1
>>       cluster-id: 99
>>     - interface: eth1
>>       threads: 1
>>       cluster-id: 98
>>     - interface: eth0
>>       threads: 1
>>       cluster-id: 99
>>     - interface: eth1
>>       threads: 1
>>       cluster-id: 98
>>     ...
>>     -
>> default:
>>       #set every global variable here
>>
> 
> Wonder how hard it would be to hide this kind of detail. E.g. something
> like:
> 
> af-packet:
>   - interface: eth0
>     cluster-id: 99
>     numa-node: 0
>   - interface: eth1
>     cluster-id: 98
>     numa-node: 1
> 
> It would immediately bring up the issue of how to exclude cores, etc. So
> might not be trivial.
> 

Or maybe allow defining named cpu sets and allow assigning those to
af-packet interface configs:

- cpu-set
  name: af-packet-eth0
  cpu: [ 0, 2, 4, 6, 8, 10, 12, 14]
  mode: "exclusive"
- cpu-set
  name: af-packet-eth1
  cpu: [1, 3, 5, 7, 9, 11, 13, 15 ]
  mode: "exclusive"



af-packet:
  - interface: eth0
    cluster-id: 99
    cpu-set: "af-packet-eth0"
  - interface: eth1
    cluster-id: 98
    cpu-set: "af-packet-eth1"

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------