[Oisf-users] Testing

[Al MailingList]

Will,

Sounds great. Like I said, I hope when I stop moving around I can help
out. Keep up the great work!!

Al


On Mon, Jan 18, 2010 at 12:56 AM, Will Metcalf
<william.metcalf at gmail.com> wrote:
>
>
> On Thu, Jan 14, 2010 at 11:14 AM, Al MailingList
> <alpal.mailinglist at gmail.com> wrote:
>>
>> Hey guys,
>>
>> I notice some talk about people looking at replacing their production
>> SNORT installs with Suricata installs?
>
>
>  The current release is beta quality code.  I wouldn't go replacing any
> production sensors until we have a production quality release.  I'm not
> saying that the code isn't fairly stable, just that we have some things to
> do performance tuning wise, the addition of some keywords etc. With all of
> that said we are looking for willing beta testers to provide us feedback :D.
>
>> Just curious about the level of testing that's been done to ensure
>> 100% true pos rates on detection etc?
>
>
> If you mean are we a 100% drop-in replacement for snort at this point? No we
> are not, but we are getting closer ;-).  For the functionality we have
> implemented, we have nightly QA runs  with real traffic etc to test out
> various aspects of the engine.  We also have unit tests built into the code
> to ensure functionality and to detect regressions.  We run fuzzers around
> the clock to find all kinds of interesting corner cases that may make the
> engine explode etc.  Additionally we will never support the snort .so rules
> as these rely on the snort engine itself.
>
>>
>> I still haven't had a chance to actually look at the code, but in
>> particular interested in performance of the stream reassembler etc in
>> real world scenarios? Has anyone done a bake off against SNORT with
>> the same rules on realistic data rates?
>
>
> Like I mentioned previously we still have some performance tuning to do.
> But in the interest of full-disclosure we have gotten some reports that the
> engine performs much better than snort and some reports that it performs
> much worse. I have a feeling at this point this greatly depends on traffic
> mixture and rules.  For example if you are reading from pcap the current run
> mode for reading pcap files really doesn't take full advantage of threading
> framework if you have more than two cores (someday soon the number of
> threads for stream handling, detection, etc will be user configurable, and
> or auto-tuned). Additionally currently we look at the entire http response
> body whereas by default snort only looks at the first 300 bytes, this will
> add additional overhead to suricata.  Also testing IDP accurately is hard
> especially at high speeds, and it is also hard to perform an apples to
> apples comparison.  Probably about the closest thing that we could get is
> multiple flow-pinned snort processes but then certain things begin to break
> down like thresholding as the snort processes no longer share this data
> between them, so detection rates will differ here.
>
> If anybody is setup to do testing like this with Breaking Point, Spirent, or
> similar types of test boxes and would be willing to share their results with
> the team and or with the public we would be ecstatic to have this sort of
> feedback.
>
> Regards,
>
> Will
>>
>> _______________________________________________
>> Oisf-users mailing list
>> Oisf-users at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
>