I've done a little bit of work with rule profiling enough to realize that I need help from someone who understands Aho-Corasick better so that I can more accurately figure out what the load would be on the detection engine. (I posted something to this effect on the Emerging Threats list last week.) One real performance factor, as far as I can tell, would be to see if the content matching already appears in another rule. If that's true, then the effective load of adding that rule could be negligible, since the detection engine's effective load doesn't actually increase. So, a tool which takes the entire ruleset into account would be very helpful. I know that Sourcefire kind of already does this in Snort in a few places, but I think the number of people who understand the information from the state transition table reports could be counted on one hand (judging from the lack of comments on the ET list). That information needs to be wrapped into a larger profiler.<br>
<br>Regarding libpcap stats, to put it simply, they lie. I'm speaking from Snort experince here, but when I've used router byte counters to audit how much traffic is going through an interface, then asked Snort how many MB/sec it processed, the numbers are very, very different until I reduce the load on the box via subnet-based BPF. The other problem is that libpcap drop numbers are completely useless if you're using an Endace DAG card or (correct me if this is not true) running through iptables. Undetected drops have been bad enough in my environment where I've resorted to creating specific heartbeat signatures and test for the absence of a signature to detect when a sensor is failing. That's still far from perfect, though, as there's plenty of room for drops in the middle. In any case, that tells a very different story than asking libpcap how many packets it's dropping.<br>
<br>So here's an idea from left field: what if there were multiple, overlapping detection engines running which were capable of auditing each other? It would be tough to get perfect, but one engine should have about the same TCP state info as another engine, for instance. Then a periodic comparison of those states could shed light on how bad things are. If the two engines agree on 99.9 percent of the traffic, it's a safe bet that they are able to read everything. Because if they were both overloaded, there's little chance that they would randomly agree on specific TCP states. That's just a brainstorm, and I realize that doubling the load on a box for audit purposes may be a bit much, but it's that kind of style that I would be looking for, as opposed to hoping that libpcap is truthful in its reports.<br>
<br><div class="gmail_quote">On Wed, Feb 11, 2009 at 3:27 AM, Victor Julien <span dir="ltr"><<a href="mailto:lists@inliniac.net">lists@inliniac.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="Ih2E3d">-----BEGIN PGP SIGNED MESSAGE-----<br>
Hash: SHA1<br>
<br>
</div><div class="Ih2E3d">Martin Holste wrote:<br>
> I think a fair amount of auto-configuration for the super-techy details<br>
> would really help. To complement that, I'd also really like to see a<br>
> focus on performance metrics. Too often we are in situations where we<br>
> have to try to infer something based on rules that _didn't_ fire. When<br>
> you're not confident in a sensor, that's basically impossible. Some<br>
> sort of real-time non-libpcap-based drop statistic or load-shedding<br>
> would be a huge leap forward. For bonus points, a system for providing<br>
> a 100% objective performance baseline of a given signature or module<br>
> would also really help. I know that each rule performs differently<br>
> depending on the traffic at hand, but a metric detailing<br>
> worst-case/best-case scenario performance would provide a really nice<br>
> guideline to aid in making decisions about which rules should make the<br>
> cut into the ruleset. This could be crudely calculated by, say, the<br>
> number of PCRE's used, length of content searches, etc.<br>
<br>
</div>Great suggestion. Matt and I have been talking about doing something<br>
like this for ET sigs for a while already, just never got to actually<br>
building something.<br>
<br>
You mentioned that you would like non-libpcap stats. Whats wrong with<br>
them and what is it you want instead?<br>
<br>
Regards,<br>
Victor<br>
<div class="Ih2E3d"><br>
<br>
> On Tue, Feb 10, 2009 at 10:12 AM, Matt Jonkman <<a href="mailto:jonkman@jonkmans.com">jonkman@jonkmans.com</a><br>
</div><div><div></div><div class="Wj3C7c">> <mailto:<a href="mailto:jonkman@jonkmans.com">jonkman@jonkmans.com</a>>> wrote:<br>
><br>
> I agree, I'm not enamored with the snort-style config. I'd much rather<br>
> it be more dynamic, and possibly even real-time adjustable by the engine<br>
> to suit it's resources.<br>
><br>
> Or even better, one that would build a baseline of the box's<br>
> capabilities and then config itself to suit. Such as choosing search<br>
> methods that fit the ram available, # of threads based on cpu's<br>
> available, etc. Take more of this out of black magic guesswork and into<br>
> a more scientific method...<br>
><br>
> Matt<br>
><br>
> Victor Julien wrote:<br>
> > Martin Fong wrote:<br>
> >> Matt Jonkman wrote:<br>
> ><br>
> >>> Non-tokenized preprocessor parameter lines<br>
> >> Let me rephrase this into what I'd like (versus definition by<br>
> >> negation): It would be great if processor arguments could<br>
> (optionally)<br>
> >> _include_ newlines to permit line-oriented parameter definition. For<br>
> >> example, this would allow<br>
> ><br>
> >> allow newlines<br>
> ><br>
> >> preprocessor myPreprocessor: \<br>
> >> threshold = 1.0 # a description \<br>
> >> max_count = 10 # another description<br>
> ><br>
> >> disallow newlines<br>
> ><br>
> >> where "[dis]allow newlines" would dictate the parameter token scanner<br>
> >> behavior.<br>
> ><br>
> >> As a side issue, I'd also like more functionality in the mSplit<br>
> >> () replacement. Specifically, it would be nice if it accepted 0<br>
> >> (zero) for max_strs and then dynamically allocate the requisite<br>
> >> members, particularly when the input is user-specified and thus<br>
> >> causing the maximum to be relatively unpredictable (e.g., IP<br>
> >> blacklists).<br>
> ><br>
> > I think we need to work out a rules syntax and configuration scheme<br>
> > first. I'm not convinced we should have a snort compatible<br>
> configuration<br>
> > scheme... I haven't thought of alternatives though.<br>
> ><br>
> > Regards,<br>
> > Victor<br>
> ><br>
><br>
> --<br>
> --------------------------------------------<br>
> Matthew Jonkman<br>
> Emerging Threats<br>
> Phone 765-429-0398<br>
> Fax 312-264-0205<br>
> <a href="http://www.emergingthreats.net" target="_blank">http://www.emergingthreats.net</a><br>
> --------------------------------------------<br>
><br>
> PGP: <a href="http://www.jonkmans.com/mattjonkman.asc" target="_blank">http://www.jonkmans.com/mattjonkman.asc</a><br>
><br>
><br>
> _______________________________________________<br>
> Discussion mailing list<br>
> <a href="mailto:Discussion@openinfosecfoundation.org">Discussion@openinfosecfoundation.org</a><br>
</div></div>> <mailto:<a href="mailto:Discussion@openinfosecfoundation.org">Discussion@openinfosecfoundation.org</a>><br>
<div class="Ih2E3d">> <a href="http://lists.openinfosecfoundation.org/mailman/listinfo/discussion" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/discussion</a><br>
><br>
><br>
<br>
<br>
</div><div class="Ih2E3d">- --<br>
- ---------------------------------------------<br>
Victor Julien<br>
<a href="http://www.inliniac.net/" target="_blank">http://www.inliniac.net/</a><br>
PGP: <a href="http://www.inliniac.net/victorjulien.asc" target="_blank">http://www.inliniac.net/victorjulien.asc</a><br>
- ---------------------------------------------<br>
<br>
-----BEGIN PGP SIGNATURE-----<br>
Version: GnuPG v1.4.9 (GNU/Linux)<br>
Comment: Using GnuPG with Mozilla - <a href="http://enigmail.mozdev.org" target="_blank">http://enigmail.mozdev.org</a><br>
<br>
</div>iEYEARECAAYFAkmSmeYACgkQiSMBBAuniMeuhwCfdnSPZxC5UG1ITzhhGzfdlhRo<br>
uBEAnRMcybFmg336SyNnQjKm3Ac6EDml<br>
=tl4o<br>
-----END PGP SIGNATURE-----<br>
</blockquote></div><br>