[Oisf-devel] Suricata 4.0 released!

Thu Jul 27 14:44:00 UTC 2017

We are thrilled to announce Suricata 4.0. This is a major new release,
improving detection capabilities, adding new output options and more.

*Improved Detection*

Based on valuable feedback from the rule writing teams at Emerging
Threats and Positive Technologies we’ve added and improved many rule
keywords for inspecting HTTP, SSH and other protocols. TLS additions
were contributed by Mats Klepsland at NorCERT, including decoding,
logging and matching on TLS serial numbers. Additionally, Suricata now
allows rule writers to specify who’s the target in a signature. This
information is used in EVE JSON logging to give more context with alerts.

*TLS improved, NFS added*

More on the TLS side: A major new feature is support for STARTTLS in
SMTP and FTP. TLS sessions will now be logged in these cases. More
goodness from Mats Klepsland. Also, TLS session resumption logging is
now supported thanks to the work of Ray Ruvinskiy. Additional TLS
logging improvements were done by Paulo Pacheco.

NFS decoding, logging and file extraction was added as part of the
experimental Rust support. Read on for more information about Rust.

*More EVE JSON*

EVE is extended in several ways:
- in the case of encapsulated traffic both the inner and outer ip
addresses and ports are logged
- the ‘vars’ facility logs flowbits and other vars. This can also be
used to log data extracted from traffic using a PCRE statement in rules
- EVE can now be rotated based on time
- EVE was extended to optionally log the HTTP request and/or response bodies
- the (partial) flow record is added to alert records.

The ‘vars’ facility is one of the main improvements here, as it is now
possible for a signature to accurately extract information for logging.
For instance, a signature can extract an advertised software version or
other information such as the recipient of an email.
[https://blog.inliniac.net/2016/12/20/suricata-bits-ints-and-vars/]

*First Step into a Safer Future*

This is the first release in which we’ve implemented parts in the Rust
language using the Nom parser framework. This work is inspired by Pierre
Chiffliers’ (ANSSI), talk at SuriCon 2016 (pdf). By compiling with
–enable-rust you’ll get a basic NFS parser and a re-implementation of
the DNS parser. Feedback on this is highly appreciated.

The Rust support is still experimental, as we are continuing to explore
how it functions, performs and what it will take to support it in the
community. Additionally we included Pierre Chiffliers Rust parsers work.
This uses external Rust parser ‘crates’ and is enabled by using
–enable-rust-experimental. Initially this adds a NTP parser.

*Under the Hood*

A major TCP stream engine update is included. This should lead to better
performance and less configuration, especially in IPS mode. First steps
in TCP GAP recovery were taken, with implementations for DNS and NFS.

For developers, this release makes extending the detection engine with
high performance keywords a lot easier. Adding a new high performance
keyword using multi pattern matching does now requires only a few lines
of code.

*Documentation*

David Wharton at SecureWorks has created a section in the documentation
for rule writers who have a background in Snort. It documents changes
that are relevant for writing rules.

*Next steps*

Based on the feedback we’ll get we’re expecting to do a 4.0.1 release in
a month or so. Then we’ll start work on the next major release, which is
4.1. This is planned for late fall, ETA before SuriCon in Prague.

*Feature tickets*

Feature #806: Implement STARTTLS support
Feature #2006: tls: decode certificate serial number
Feature #1969: TLS transactions with session resumption are not logged
Feature #2129: nfs: parser, logger and detection
Feature #2130: dns: rust parser with stateless behaviour
Feature #2131: nfs: implement GAP support
Feature #2163: ntp parser
Feature #2164: rust: external parser crate support
Feature #2077: Additional HTTP Header Contents and Negation
Feature #2011: eve.alert: print outside IP addresses on alerts on
traffic inside tunnels
Feature #2095: eve: http body in alert event
Feature #1978: Using date in logs name
Feature #1998: eve.tls: custom TLS logging
Feature #2046: Support custom file permissions per logger
Feature #2123: unix-socket: additional runmodes
Feature #2132: eve: flowbit and other vars logging
Feature #2156: Add app_proto or partial flow entry to alerts
Feature #744: Teredo configuration
Feature #2061: lua: get timestamps from flow
Feature #1953: lua: expose flow_id
Feature #1748: lua: expose tx in alert lua scripts
Feature #1636: Signal rotation of unified2 log file without restart
Feature #2133: unix socket: add/remove hostbits
Feature #805: Add support for applayer change
For all other closed tickets please see the full changelog of 4.0.

*Download*

https://www.openinfosecfoundation.org/download/suricata-4.0.0.tar.gz

*Special thanks*

Mats Klepsland – for his major contributions: many EVE and TLS features

Pierre Chifflier – for paving the way for the Rust experiment and being
very helpful while learning Rust and Nom.

Additionally: Abdullah Ada, Jérémy Beaume, Sebastian Garcia, Alexander
Gozman, Giuseppe Longo, Paulo Pacheco, Selivanov Pavel, Ray Ruvinskiy,
Peter Sanders, David Wharton, Jon Zeolla, the AFL project and Coverity Scan.

*Suricata Trainings and Events*

We have several community events and trainings on the calendar and in
the works for 2017… here are some of the highlights:

5-Day Developer Deep Dive Training – Sept 11 – 15, 2017, Cork, Ireland –
led by Victor Julien, Eric Leblond, and Jason Ish
Rule Writing Training @ DerbyCon – Sept 20 – 24, 2017 – SOLD OUT!
Rule Writing Training @ SuriCon – Nov 13 – 14, 2017
2-Day Suricata Training @ SuriCon – Nov 13 – 14, 2017
SuriCon 2017 – Nov 15 – 17, 2017, Prague
Details and registration for all our events can be found at
https://suricata_events.eventbrite.com. Don’t delay as space is limited.

We also offer custom training events for your team – contact us at
info at oisf.net for details.

*About Suricata*

Suricata is a high performance Network Threat Detection, IDS, IPS and
Network Security Monitoring engine. Open Source and owned by a community
run non-profit foundation, the Open Information Security Foundation
(OISF). Suricata is developed by the OISF, its supporting vendors and
the community.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------