[Discussion] Cooperative event loops (e.g., libevent) to support asynch I/O

[Martin Holste]

What kinds of tasks were you imagining would be executed at intervals?
Garbage collection, statistics?  What did you have in mind?

On Tue, Feb 10, 2009 at 3:32 AM, Victor Julien <lists at inliniac.net> wrote:

> Martin Fong wrote:
> > Matt Jonkman wrote:
> >
> >> Cooperative event loops (e.g., libevent) to support asynch I/O
> >>
> >> Do you mean something like parallel processing of different tools on the
> >> same stream or event? For instance one stream with http would go through
> >> regular matching but also be copied out to a http interpreter to pull
> >> environment variables and the like onto a separate thread/processor?
> >
> > I'm actually thinking about unix select () loops.  Because the current
> > implementation simply invokes pcap_loop (), there's no way for a
> > module to independently perform I/O or respond to a select ()-based
> > timer.  There have been cases when I've needed to write data to a
> > socket or read from a pipe, but my preprocessor only got the
> > opportunity when it "processed" a packet (-- and clearly I don't want
> > the I/O to block).  (And, yes, I know about the danger of having a
> > module consume too many cycles doing I/O or computing, but there's
> > nothing to prevents such abuse now -- and cooperative event processing
> > _does_ work.)
> >
> >      As far as your suggestion, that too sounds interesting...!
>
> I was thinking about having a way for module to register functions that
> will be executed at given times and intervals, probably by having a
> separate thread for that. That would introduce a lot of locking issues
> of course, so maybe another/additional way is needed.
>
> What about having a module get called at a fixed interval even if there
> are no packets? Pcap/ip_queue/nf_queue etc can all work with timeouts or
> non-blocking io...
>
> Maybe the module registration functions should be able to determine the
> interval, so that if no module requires it the overhead is not imposed,
> and if your module does it's set and used.
>
> Interesting...
>
> Cheers,
> Victor
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Discussion mailing list
> Discussion at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090210/52346cee/attachment-0002.html>