[Oisf-devel] Linux af-packet::mmap enhancement

[Victor Julien]

On 06/23/2011 05:35 PM, chetan loke wrote:
> On Thu, Jun 23, 2011 at 10:58 AM, Victor Julien <victor at inliniac.net> wrote:
>> On 06/23/2011 04:41 PM, chetan loke wrote:
>>> On Thu, Jun 23, 2011 at 3:03 AM, Victor Julien <victor at inliniac.net> wrote:
>>>> On 06/22/2011 08:30 PM, chetan loke wrote:
>>>
>>> Correct me if I'm wrong. But what this support means is, for exploiting FANOUT,
>>> one needs to open multiple af-packet sockets for a single monitoring interface?
>>
>> Doesn't need to, but can. This way we have several CPU/core pinned
>> threads read packets from the same interface. We support this with
>> pfring's clusters and it's working very well for performance.
>>
> 
> I just read all the emails related to this on netdev to make sure I
> don't waste your time.
> Oki, so, if there is only 1 af-packet::socket listener for a
> monitoring interface then
> FANOUT is moot.

Yeah then you can just revert to regular AF_PACKET I assume.

> To exploit FANOUT one will need to have '1+' af-packet listeners for a
> single monitoring
> interface. That way:
> a) as you mentioned above, one can also pin the threads to CPU::cores.
> b) you can also maintain flow-context with a thread-parser.
> 
> With PACKET_FANOUT_HASH I see one problem :
> Since traffic profile is not known in advance, there could be cases
> where some threads
> could be lightly loaded with flows and others could be heavily loaded.
> example - When particular VLAN[s] is[are] heavily loaded then that
> thread will get bogged down.

Yup, this is a disadvantage the pfring cluster approach has as well iirc.

> With PACKET_FANOUT_LB I see another problem:
> We cannot maintain flow-context as mentioned in b) above.

Right.

> But let's say if we don't implement FANOUT then user-space can infact
> achieve both HASH+LB.
> If it's not present already, I can export the pre-computed 'rx_hash'
> in the libpcap-descriptor/pkt-header.
> Once the packet hits suricata, we can steer it an old-thread(who
> already is working on this flow)
> or round-robin it to the next available thread to achieve load-balance.

Suricata can do this. Currently it's internal (and optional) load
balancing is just as "blind" the rxhash approach (probably less even
balanced even :). Actual load of the threads is not taken into account.
But it would be possible it Suricata's architecture to improve this
without too much effort I think.

The disadvantage is that multiple threads/cpu's/cores will suddenly be
handling the same packet, instead of just the one dealing with the
FANOUT socket.

> Again, my understanding could be limited because I haven't looked at
> the code yet. So please ignore
> if flow-load distribution is already taken care off in some other way
> for these cases with pf_ring.

As far as I know the pfring approach is just as "dumb". I know some hw
accl suppliers have smarter ways though. Not sure how these work exactly.

Cheers,
Victor

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------