[Discussion] Suricata 1.4 Available!

[Victor Julien]

The OISF development team is proud to announce Suricata 1.4. This
release is a major improvement over the previous releases with regard to
performance, scalability and accuracy. Also, a number of great features
have been added.

Get the new release here:
http://www.openinfosecfoundation.org/download/suricata-1.4.tar.gz

The biggest new features of this release are the Unix Socket support, IP
Reputation support and the addition of the Luajit keyword. Each of these
new features are still in active development, and should be approached
with some care.

The 1.4 release improves performance and scalability a lot. The IP
Defrag engine was rewritten to scale better, various packet acquisition
methods were improved and various parts of the detection engine were
optimized further.

The configuration file has evolved but backward compatibility is
provided. We thus encourage you to update your suricata configuration
file. Upgrade guidance is provided here:
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Upgrading_Suricata_13_to_Suricata_14

Detailed list of changes:

New features

- Unix socket mode for batched processing of series of pcap (#571, #552)
(experimental)
- Interaction with Suricata via uix socket (#571, #552) (experimental)
- IP Reputation: loading and matching (#647) (experimental)
- New keyword: "luajit" to inspect packet, payload and all HTTP buffers
with a Lua script (#346) (experimental)
- Delayed detect initialization. Starts processing packets right away
and loads detection engine in the background (#522)
- Support for pkt_data keyword was added (#423)
- Improved --list-keywords commandline option gives detailed info for
supported keyword, including doc link (#435)
- User and group to run as can now be set in the config file
- Add stream event to match on overlaps with different data in stream
reassembly (#603)
- Decoding of IPv4-in-IPv6, IPv6-in-IPv6 and Teredo tunnels (#462, #514,
#480)
- Rules can be set to inspect only IPv4 or IPv6 (#494)
- Added ability to control per server HTTP parser settings in much more
detail (#503)
- Make HTTP request and response body inspection sizes configurable per
HTTP server config (#560)
- Filesize keyword for matching on sizes of files in HTTP (#489)
- Custom HTTP logging contributed by Ignacio Sanchez (#530)
- TLS certificate logging and fingerprint computation and keyword by
Jean-Paul Roliers (#443)
- TLS certificate store to disk feature Jean-Paul Roliers (#444)
- AF_PACKET IPS support (#516)
- NFQ fail open support (#507)
- PCAP/AF_PACKET/PF_RING packet stats are now printed in stats.log
(#561, #625)
- Support for Napatech cards through their 3rd generation driver was
added by Matt Keeler from Npulse (#430, #619)
- Endace support improved
- New runmode for users of pcap wrappers (Myricom, PF_RING, others)

Improvements

- Add contrib directory to the dist (#567)
- Performance improvements to signatures with dsize option
- Improved rule analyzer: print fast_pattern along with the rule (#558)
- Fixes to stream engine reducing the number of events generated (#604)
- Stream.inline option new defaults to "auto", meaning enabled in IPS
mode, disabled in IDS mode (#592)
- HTTP handling in OOM condition was greatly improved (#557)
- Filemagic keyword performance was improved (#585)
- Updated bundled libhtp to 0.2.11
- Build system improvements and cleanups
- Live reloads now supports HTTP rule updates better (#522)
- AF_PACKET performance improvements (#197, #415)
- Make defrag more configurable (#517, #528)
- Improve pool performance (#518)
- Improve file inspection keywords by adding a separate API (#531)
- Example threshold.config file provided (#302)

Changes since 1.4rc1

- Decoder event matching fixed (#672)
- Unified2 would overwrite files if file rotation happened within a
second of file creation, leading to loss of events/alerts (#665)
- Add more events to IPv6 extension header anomolies (#678)
- Fix ICMPv6 payload and checksum calculation (#677, #674)
- Clean up flow timeout handling (#656)
- Fix a shutdown bug when using AF_PACKET under high load (#653)
- Fix TCP sessions being cleaned up to early (#652)

Credits

- Jason Ish -- Endace
- Ludovico Cavedon -- Lastline
- Last G
- Matt Keeler -- Npulse
- Chris Wakelin
- Will Metcalf
- Ivan Ristic
- Kyle Creyts
- Michael Hoffrath
- Rmkml
- Jean-Paul Roliers
- Ignacio Sanchez
- Michel Saborde
- Simon Moon
- Coverity

Known issues & missing features

If you encounter issues, please let us know! As always, we are doing our
best to make you aware of continuing development and items within the
engine that are not yet complete or optimal.  With this in mind, please
notice the list we have included of known items we are working on.

See http://redmine.openinfosecfoundation.org/projects/suricata/issues
for an up to date list and to report new issues. See
http://redmine.openinfosecfoundation.org/projects/suricata/wiki/Known_issues
for a discussion and time line for the major issues.

About Suricata

Suricata is a high performance Network IDS, IPS and Network Security
Monitoring engine. Open Source and owned by a community run non-profit
foundation, the Open Information Security Foundation (OISF). Suricata is
developed by the OISF, its supporting vendors and the community.
-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------