[Discussion] State of the Project Report

[Matt Jonkman]

The Phase Two kickoff meeting for Suricata and the OISF was held in San
Francisco last Friday. We had some great discussions, these meetings
have proven to be invaluable. Thanks to all who attended, many great
ideas were exchanged and discussed. The goals of this meeting were to
review where we are in Phase One development, lay out Phase Two major
features, and bring in new ideas and challenges. These were accomplished
quite well!

Below is a discussion of where we believe we should go with Suricata for
Phase Two. This is only the beginning of the conversation, we know
everyone interested can't be at one meeting. So please consider this a
starting point and we'll continue discussion on the mailing lists.



Status

Overall, the engine is in a great state. We are much further into
development than we had expected at this point, we've solved many
technical issues we expected to be pushed to Phase Two. I have to say
I'm honored to be just near a team of developers with such talent and
dedication. Our contributors and consortium members have brought
everything we didn't have available to put this incredibly complex
engine together. My thanks to everyone who's contributed, but we have a
long road ahead of us.

We had originally intended to end Phase One with the 1.0 release and
move directly into Phase Two development. Phase One was the more
traditional features and the base functionality for the engine, then
Phase Two would be the most experimental features. We have decided to
push Phase Two development off a few months to put more time into
stabilizing and performance tuning the base engine. We need the time for
performance tuning, but also our funding for 2010 is due in September,
and the foundation is low on resources. So we're limiting development to
1.0.1 bugfixes and performance tuning for the next month or so. We'd
also like to see how this release performs and works for the community,
so get your feedback in. Phase Two features are very experimental, and
will take significant amounts of time to perfect, so we're gathering our
resources to attack this on all fronts.



So for this Interim period here are our goals:

Complete Architecture Documentation
Significant Performance Optimization
More Easily Configurable Run Mode Support (Endace has offered to
complete this)
Error Code Cleanup and Documentation
Full Documentation (community editable docs)
Advanced Profiling and Engine Statistics Module
Accuracy Improvements
Added Protocol Detections
Classifications Update (support a more elegant definition system)
Full 2.8.6 Syntax Compatibility
Better LibHTP Error Handling
Heavy Inline Testing



The Features to be pursued in Phase Two are:

High Priority:
Max Inspection Time Cutoff Setting (while inline set a packet loose to
avoid latency but still process)
File Capture and Extraction in Stream
REGEX Optimization/Acceleration (possibly using alternate regex libraries)
Live Ruleset Updates
Flow Logging (Netflow output)
Add Replace keyword support
Host attribute scrubbing (strip OS identifying oddities)
URI Matching lookups (stopbadware, websense, etc)
Full CUDA Support

Phase Two Low Priority:
IP Reputation - Explore other items, dns, etc
Distributed Blocking
Global Flowbits and flowvars
Full Stream Capture (rotating pcap support)
Traffic Redirection (bait and switch style)


We have a huge list above, and we need your help. Ideas, code
contributions, help in documentation, help in translating documentation,
and financial and hardware support are needed. We welcome input from any
source!

Please join the OISF mailing lists
(http://lists.openinfosecfoundation.org/mailman/listinfo) for more info,
discussion, and to follow developments. If you'd like more information
about consortium membership or ways you can help out please email
consortium at openinfosecfoundation.org, or myself directly at
jonkman at openinfosecfoundation.org.




----------------------------------------------------
Matthew Jonkman
Emerging Threats
Open Information Security Foundation (OISF)
Phone 765-429-0398
Fax 312-264-0205
http://www.emergingthreats.net
http://www.openinfosecfoundation.org
----------------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc