[Discussion] Cooperative event loops (e.g., libevent) to support asynch I/O

[Victor Julien]

Martin Fong wrote:
> Matt Jonkman wrote:
> 
>> Cooperative event loops (e.g., libevent) to support asynch I/O
>>
>> Do you mean something like parallel processing of different tools on the
>> same stream or event? For instance one stream with http would go through
>> regular matching but also be copied out to a http interpreter to pull
>> environment variables and the like onto a separate thread/processor?
> 
> I'm actually thinking about unix select () loops.  Because the current
> implementation simply invokes pcap_loop (), there's no way for a
> module to independently perform I/O or respond to a select ()-based
> timer.  There have been cases when I've needed to write data to a
> socket or read from a pipe, but my preprocessor only got the
> opportunity when it "processed" a packet (-- and clearly I don't want
> the I/O to block).  (And, yes, I know about the danger of having a
> module consume too many cycles doing I/O or computing, but there's
> nothing to prevents such abuse now -- and cooperative event processing
> _does_ work.)
> 
>      As far as your suggestion, that too sounds interesting...!

I was thinking about having a way for module to register functions that
will be executed at given times and intervals, probably by having a
separate thread for that. That would introduce a lot of locking issues
of course, so maybe another/additional way is needed.

What about having a module get called at a fixed interval even if there
are no packets? Pcap/ip_queue/nf_queue etc can all work with timeouts or
non-blocking io...

Maybe the module registration functions should be able to determine the
interval, so that if no module requires it the overhead is not imposed,
and if your module does it's set and used.

Interesting...

Cheers,
Victor

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------