[Oisf-devel] Linux af-packet::mmap enhancement

[chetan loke]

On Sat, Jun 25, 2011 at 4:25 AM, Victor Julien <victor at inliniac.net> wrote:
> On 06/24/2011 09:17 PM, chetan loke wrote:
>> On Fri, Jun 24, 2011 at 11:25 AM, Victor Julien <victor at inliniac.net> wrote:
>>> On 06/23/2011 06:30 PM, chetan loke wrote:
>>>> On Thu, Jun 23, 2011 at 11:59 AM, Victor Julien <victor at inliniac.net> wrote:
>>>>> The disadvantage is that multiple threads/cpu's/cores will suddenly be
>>>>> handling the same packet, instead of just the one dealing with the
>>>>> FANOUT socket.
>>>>>
>>>>
>>>> My description wasn't so clear. Actually what I meant was:
>>>>
>>>> lookup_table : <rx_hash> <thread_id>;
>>>>
>>>> thread_id = get_worker_th_id (pkt_hdr->rx_hash);
>>>>
>>>> if (thread_id) {
>>>>   /* This flow is HOT. steer this flow to the same thread */
>>>>   enqueue_pkt_to_thread(thread_id);
>>>> } else {
>>>>  /* This flow is COLD.No thread is handling this flow yet.*/
>>>>  /* So queue it to the next thread */
>>>>     enqueue_pkt_to_thread(GET_NEXT_THREAD(ROUND_ROBIN_POLICY));
>>>>     update lookup_table;
>>>> }
>>>>
>>>> So no two threads will work on the same flow.
>>>
>>> To be clear, this functionality would live in the kernel, right?
>>
>> Nope, it would reside in user-space because kernel doesn't know which
>> fanout-listener
>> is under heavy load. Once the 'rx_hash' is exported in
>> tpacket_headers, we can spread
>> the load in user space.
>
> So how would this work? Would Suricata talk to a userspace daemon
> instead of the kernel directly?

Not really. I quickly looked at the code and my vague understanding is
as follows:

1) .func(pkt_rcv_function) of a receiver would be running in a thread.
2) .func  dumps packets in 'some-queue', correct?
3) decoder pulls packets from the above 'some-queue' and normalizes
them, correct?

We should introduce a new function called 'steer_packets' which will
implement the
above 'hash+load_balance' steering logic. rx_hash field could be null
for receiver's who don't yet
export rx_hash in the packet descriptor. So if the rx_hash field is
null then it should
operate in current mode. This also implies struct Packet should
introduce a new field
called rx_hash.

steer_packet() can be called as part of decoder logic but before the
pkt is picked-up by the next thread. But that's just implementation details.


>
> Regards,
> Victor

Regards
Chetan Loke