Ok, well other than the garbage collection, it sounds like all of those
are good candidates for going in the "application layer" detached
processes. As long as the main detection process allows itself to be
reconfigured on the fly, then there's no reason to bog it down with
administrative tasks. So, I think the question then becomes one of how
do we best allow for a running configuration to be updated? Certainly
a SIGHUP would be the simplest, but do we need something less
disruptive?<br><br><div class="gmail_quote">On Wed, Feb 11, 2009 at 3:10 AM, Victor Julien <span dir="ltr"><<a href="mailto:lists@inliniac.net">lists@inliniac.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="Ih2E3d">Martin Holste wrote:<br>
> What kinds of tasks were you imagining would be executed at intervals?<br>
> Garbage collection, statistics? What did you have in mind?<br>
<br>
</div>In Snort_inline I first ran into this when creating the clamav plugin<br>
(not the best of ideas ;-)). I wanted it to check for updated virus defs<br>
on an interval. Like Martin Fong wrote that was done using the packets<br>
timestamp but the code was never activated if there were no packets. Not<br>
a big problem in this case.<br>
<br>
Another example is a contract gig I did for a company, where they wanted<br>
a preprocessor to report some stats to a http server on a fixed<br>
interval, no matter if there had been packets or not.<br>
<br>
Garbage collection is a good point too. For example cleaning up expired<br>
state... (flow/stream tables come to mind)...<br>
<br>
Regards,<br>
Victor<br>
<div class="Ih2E3d"><br>
><br>
> On Tue, Feb 10, 2009 at 3:32 AM, Victor Julien <<a href="mailto:lists@inliniac.net">lists@inliniac.net</a><br>
</div><div><div></div><div class="Wj3C7c">> <mailto:<a href="mailto:lists@inliniac.net">lists@inliniac.net</a>>> wrote:<br>
><br>
> Martin Fong wrote:<br>
> > Matt Jonkman wrote:<br>
> ><br>
> >> Cooperative event loops (e.g., libevent) to support asynch I/O<br>
> >><br>
> >> Do you mean something like parallel processing of different tools<br>
> on the<br>
> >> same stream or event? For instance one stream with http would go<br>
> through<br>
> >> regular matching but also be copied out to a http interpreter to pull<br>
> >> environment variables and the like onto a separate thread/processor?<br>
> ><br>
> > I'm actually thinking about unix select () loops. Because the current<br>
> > implementation simply invokes pcap_loop (), there's no way for a<br>
> > module to independently perform I/O or respond to a select ()-based<br>
> > timer. There have been cases when I've needed to write data to a<br>
> > socket or read from a pipe, but my preprocessor only got the<br>
> > opportunity when it "processed" a packet (-- and clearly I don't want<br>
> > the I/O to block). (And, yes, I know about the danger of having a<br>
> > module consume too many cycles doing I/O or computing, but there's<br>
> > nothing to prevents such abuse now -- and cooperative event processing<br>
> > _does_ work.)<br>
> ><br>
> > As far as your suggestion, that too sounds interesting...!<br>
><br>
> I was thinking about having a way for module to register functions that<br>
> will be executed at given times and intervals, probably by having a<br>
> separate thread for that. That would introduce a lot of locking issues<br>
> of course, so maybe another/additional way is needed.<br>
><br>
> What about having a module get called at a fixed interval even if there<br>
> are no packets? Pcap/ip_queue/nf_queue etc can all work with timeouts or<br>
> non-blocking io...<br>
><br>
> Maybe the module registration functions should be able to determine the<br>
> interval, so that if no module requires it the overhead is not imposed,<br>
> and if your module does it's set and used.<br>
><br>
> Interesting...<br>
><br>
> Cheers,<br>
> Victor<br>
><br>
> --<br>
> ---------------------------------------------<br>
> Victor Julien<br>
> <a href="http://www.inliniac.net/" target="_blank">http://www.inliniac.net/</a><br>
> PGP: <a href="http://www.inliniac.net/victorjulien.asc" target="_blank">http://www.inliniac.net/victorjulien.asc</a><br>
> ---------------------------------------------<br>
><br>
> _______________________________________________<br>
> Discussion mailing list<br>
> <a href="mailto:Discussion@openinfosecfoundation.org">Discussion@openinfosecfoundation.org</a><br>
</div></div>> <mailto:<a href="mailto:Discussion@openinfosecfoundation.org">Discussion@openinfosecfoundation.org</a>><br>
<div class="Ih2E3d">> <a href="http://lists.openinfosecfoundation.org/mailman/listinfo/discussion" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/discussion</a><br>
><br>
><br>
<br>
<br>
</div>--<br>
<div><div></div><div class="Wj3C7c">---------------------------------------------<br>
Victor Julien<br>
<a href="http://www.inliniac.net/" target="_blank">http://www.inliniac.net/</a><br>
PGP: <a href="http://www.inliniac.net/victorjulien.asc" target="_blank">http://www.inliniac.net/victorjulien.asc</a><br>
---------------------------------------------<br>
<br>
</div></div></blockquote></div><br>