[Oisf-devel] [COMMIT] OISF branch, master, updated. suricata-4.0.1-354-gd0ea147
OISF Git
noreply at openinfosecfoundation.org
Tue Jan 30 18:42:43 UTC 2018
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "OISF".
The branch, master has been updated
via d0ea1472639a77b4e243f7a14507eb45b5e24e9c (commit)
via 6f7e24d3f25ef767aea9d32a002647c6183e0bec (commit)
via 6e82df274dcbc43e6473e5e259ae6df255f0a113 (commit)
via dfae3297a5852c014ec439278f513903b044713a (commit)
via ab939f4aaa04adc32a5feb12d07c32afe7238046 (commit)
via 93b056d89e857c6270d4eeeea48c49aa82363aa0 (commit)
via b659222ea02a2047b861bcb263f21063b442740c (commit)
via 1f47f77bd5381838bbf64d2767f9e62d84d2397d (commit)
via 0e02684634f82b76b3425b84d80fb376b94b30a4 (commit)
via 572a62f35a15d3a2e519e67c83ca4078b408bdc8 (commit)
via 34811cf69e243567afe23266b59f437283db1d15 (commit)
via a23d54ce3ed40bae2e9c531a5c30f8fdcba348ec (commit)
via 3eaca7c239fb62b3c98cebc735f1626910cb545e (commit)
via 790ce3743ba7edb9686bc0906b3075640d26331b (commit)
via 23bbbc5818649dc20e7a69e585a27a6552bfbbbf (commit)
via 4a05160353a744be68037e12b557ac0d58f07ba1 (commit)
via 5da5fc1f7deff03c3dd31ad654ede33c2f4c0bd2 (commit)
via 2247b9aad2dcaab54014ff1ae6d2cd35381b28d5 (commit)
via 885452fc22a37b91f99cb4713e9a9401f925a4a8 (commit)
via b577f4a0c99e6ec6a2287f34e13fe97bd8a8287a (commit)
via 7f5439a3004da75179a3c02a68607753ffc0e0ec (commit)
via 32da579239082efe41d63c837c9a5da5380d59ba (commit)
via 88ac0f2b1a8606ed5c5de8388970800227d2a6ab (commit)
via dd988d9934286535897e3c5e9215d85ff7b1523f (commit)
via 5138f99c580e75d38e0e6cf38253d16e84a374d9 (commit)
via 6f339abdf01f972d213d1c9e76ff3d427e1eb183 (commit)
via e86be22737c93870841c053f20baaad29f33cd95 (commit)
via c63b1ce2c676218132b625eb28df995a3d47768a (commit)
via 07cbbfb0d124e11b99087a40de5056dac7533e5a (commit)
via 9ca71beb031c9d4c76a62449426d9ac91651ce61 (commit)
via daeba48f779443e03383ab00809114b819e0b60a (commit)
via c0d26de665345caa69bcdf1a0a2fcec33fcaa971 (commit)
via e64941144eb57df2fc7ab866f8d26c6b95c8e69f (commit)
via 49927024c6735e85c6a49246ec70324fe5d9e4fb (commit)
via ca67408e791eefbab70ea48546b004647c35c43f (commit)
via d1adf5f7e96a66482da0f21f9d8f9233e0f94ea5 (commit)
via e1ef57c848bbe4e567d5d4b66d346a742e3f77a1 (commit)
via 700781c53b847eb73e35742a3b44b934d4fda9e6 (commit)
via 89dc05d4a6ee05aab03ae6baa6c56be01056f33a (commit)
via 251156e2539396655eea2852182565a3a68b640f (commit)
via 80f2fbac6ebc1b9114175068647d6d1c44b46776 (commit)
via b9cf49e933d8216e31136ec4b64fc46653d6d729 (commit)
via 0ff60f65ec4cf4d71d68a28da4c5d5c161da2176 (commit)
via e8939335eaccd9cf20141f299e8781c210af863b (commit)
via 2c3c8f8b85a0abd8bc12b546fcc9a77d084d6c0b (commit)
via d27ed5957faf709b900bf5a4393f4968dc31f961 (commit)
via d75d9d0b45027c8ab9a57be57855c57e03e53bbb (commit)
via ce08a43bdaf5fc5ea81e76d6c5cd30b459bdaba7 (commit)
via 3a2e4614d074ac66df1f7b5a3ec15d9da9c78660 (commit)
via 45c5030ff070c2ba08409fcd7b6c5a48f0a34f22 (commit)
via 288ddc95acf148837729233f7ca1f870f835b9b8 (commit)
via 4a89d939fcb1a0718916f7b3ffce138be50e7138 (commit)
from cba41207b3f2d8251f7e0f7944683134d9cf8233 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit d0ea1472639a77b4e243f7a14507eb45b5e24e9c
Author: Victor Julien <victor at inliniac.net>
Date: Tue Jan 30 17:28:47 2018 +0100
travis/rust: update rust minimum to 1.21
Ubuntu LTS and CentOS7/EPEL has upgraded to 1.21.
Update highest known working version to 1.23.
commit 6f7e24d3f25ef767aea9d32a002647c6183e0bec
Author: Victor Julien <victor at inliniac.net>
Date: Tue Jan 30 12:20:42 2018 +0100
autogen/rust: remove Cargo.lock
Remove Cargo.lock to avoid issues when updating Cargo.toml
commit 6e82df274dcbc43e6473e5e259ae6df255f0a113
Author: Victor Julien <victor at inliniac.net>
Date: Mon Jan 29 15:02:13 2018 +0100
rust: update dependencies
commit dfae3297a5852c014ec439278f513903b044713a
Author: Victor Julien <victor at inliniac.net>
Date: Mon Jan 29 14:16:01 2018 +0100
rust: don't gen C headers if Rust isn't enabled
commit ab939f4aaa04adc32a5feb12d07c32afe7238046
Author: Jason Ish <ish at unx.ca>
Date: Thu Jan 25 16:20:32 2018 -0600
doc: breakout eve-log section to a partial file
Both the suricata.yaml and eve configuration sections
included the eve-log section from suricata.yaml. First,
sync these up with the actual suricata.yaml then break
it out into its own file, so only one file needs to
be kept in sync with the actual configuration file.
commit 93b056d89e857c6270d4eeeea48c49aa82363aa0
Author: Jason Ish <ish at unx.ca>
Date: Thu Jan 25 15:33:31 2018 -0600
eve/alert: log metadata be default
By default log metadata.
Remove toggles for individual protocol types and just use a
single toggle to control including the app-layer with the
alert.
The metadata (currently app-layer and flow) can be disabled
by setting metadata to a falsey value, but its removed
from the default configuration (but wil be in docs)
commit b659222ea02a2047b861bcb263f21063b442740c
Author: Jason Ish <ish at unx.ca>
Date: Thu Jan 25 14:55:55 2018 -0600
eve/metadata: log flowvars as a list of k/v pairs
To match the pktvars output.
commit 1f47f77bd5381838bbf64d2767f9e62d84d2397d
Author: Jason Ish <ish at unx.ca>
Date: Thu Jan 25 14:49:38 2018 -0600
eve/metadata: special handling for traffic-id labels
Give traffic/id and traffic/label flowbits special handling
in the eve output. Instead of just logging them as flowbits,
give them their own top level object.
{
"traffic": {
"id": ["id0", "id1"],
"label": ["label0", "label1"]
}
}
commit 0e02684634f82b76b3425b84d80fb376b94b30a4
Author: Jason Ish <ish at unx.ca>
Date: Thu Jan 25 08:52:47 2018 -0600
doc: update eve-log section for metadata
commit 572a62f35a15d3a2e519e67c83ca4078b408bdc8
Author: Jason Ish <ish at unx.ca>
Date: Wed Jan 24 16:40:11 2018 -0600
output-json-vars: rename to metadata
No functional change, just rename of files and functions
to reflect the metadata event type now used.
commit 34811cf69e243567afe23266b59f437283db1d15
Author: Jason Ish <ish at unx.ca>
Date: Wed Jan 24 14:51:03 2018 -0600
json-vars: rename to metadata and use new metadata format
commit a23d54ce3ed40bae2e9c531a5c30f8fdcba348ec
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:50:54 2017 -0600
eve: netflow: global metadata config
commit 3eaca7c239fb62b3c98cebc735f1626910cb545e
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:50:42 2017 -0600
eve: http: global metadata config
commit 790ce3743ba7edb9686bc0906b3075640d26331b
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:50:36 2017 -0600
eve: flow: global metadata config
commit 23bbbc5818649dc20e7a69e585a27a6552bfbbbf
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:50:30 2017 -0600
eve: dns: global metadata config
commit 4a05160353a744be68037e12b557ac0d58f07ba1
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:50:25 2017 -0600
eve: alert: global metadata config
Also, remove vars as a subtype. Adding the top level metadata
field is an eve lebel parameter, not alert now.
commit 5da5fc1f7deff03c3dd31ad654ede33c2f4c0bd2
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:50:18 2017 -0600
eve: drop: global metadata config
commit 2247b9aad2dcaab54014ff1ae6d2cd35381b28d5
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:50:07 2017 -0600
eve: email: respect global metadata config
commit 885452fc22a37b91f99cb4713e9a9401f925a4a8
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:49:57 2017 -0600
eve: nfs: respect global metadata config
commit b577f4a0c99e6ec6a2287f34e13fe97bd8a8287a
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:49:51 2017 -0600
eve: smtp: respect global metadata config
commit 7f5439a3004da75179a3c02a68607753ffc0e0ec
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:49:45 2017 -0600
eve: dnp3: respect global metadata config
commit 32da579239082efe41d63c837c9a5da5380d59ba
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:49:40 2017 -0600
eve: ssh: respect global metadata config
commit 88ac0f2b1a8606ed5c5de8388970800227d2a6ab
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:49:34 2017 -0600
eve: tls: respect global metadata config
commit dd988d9934286535897e3c5e9215d85ff7b1523f
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 15:48:14 2017 -0600
eve: metadata setting to enable/disable metadata
This is a top level metadata object containing flowbits,
flowints, pktvars and flowvars.
Enabling it at the top level enables it for all log types.
commit 5138f99c580e75d38e0e6cf38253d16e84a374d9
Author: Jason Ish <ish at unx.ca>
Date: Mon Dec 11 10:16:47 2017 -0600
eve: top level metadata object
Contains:
- flowbits (as array)
- flowints (map)
- flowvars (map)
- pktvars (map)
commit 6f339abdf01f972d213d1c9e76ff3d427e1eb183
Author: Victor Julien <victor at inliniac.net>
Date: Mon Jan 29 16:45:33 2018 +0100
htp: minor debug addition
commit e86be22737c93870841c053f20baaad29f33cd95
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 26 12:05:35 2018 +0100
htp: remove unused field from tx state
commit c63b1ce2c676218132b625eb28df995a3d47768a
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 26 12:00:15 2018 +0100
htp: remove used body operation field
commit 07cbbfb0d124e11b99087a40de5056dac7533e5a
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 26 11:53:27 2018 +0100
htp: code cleanups
commit 9ca71beb031c9d4c76a62449426d9ac91651ce61
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 26 11:48:55 2018 +0100
htp: remove usused file flags
commit daeba48f779443e03383ab00809114b819e0b60a
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 26 11:41:03 2018 +0100
htp: remove usused flags
commit c0d26de665345caa69bcdf1a0a2fcec33fcaa971
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 26 09:14:16 2018 +0100
stream: improve overlap detection
Improve detection of overlapping different data. Keep some data around
even if it was already ACK'd to check if packets have overlap.
commit e64941144eb57df2fc7ab866f8d26c6b95c8e69f
Author: Victor Julien <victor at inliniac.net>
Date: Sun Nov 12 09:16:17 2017 +0100
htp: allow HTTP pickup of response data
Now that libhtp can pick up sessions that start with a response
we can enable support for it as well.
commit 49927024c6735e85c6a49246ec70324fe5d9e4fb
Author: Victor Julien <victor at inliniac.net>
Date: Sun Nov 12 09:15:33 2017 +0100
http: add tests for malformed response lines
commit ca67408e791eefbab70ea48546b004647c35c43f
Author: Victor Julien <victor at inliniac.net>
Date: Wed Jan 24 16:37:27 2018 +0100
stream: set event for suspected data injection during 3whs
This rule will match on the STREAM_3WHS_ACK_DATA_INJECT, that is
set if we're:
- in IPS mode
- get a data packet from the server
- that matches the exact SEQ/ACK expectations for the 3whs
The action of the rule is set to drop as the stream engine will drop.
So the rule action is actually not needed, but for consistency it
is drop.
commit d1adf5f7e96a66482da0f21f9d8f9233e0f94ea5
Author: Victor Julien <victor at inliniac.net>
Date: Wed Jan 24 15:59:57 2018 +0100
stream: handle data on incomplete 3whs
If we have only seen the SYN and SYN/ACK of the 3whs, accept from
server data if it perfectly matches the SEQ/ACK expectations. This
might happen in 2 scenarios:
1. packet loss: if we lost the final ACK, we may get data that fits
this pattern (e.g. a SMTP EHLO message).
2. MOTS/MITM packet injection: an attacker can send a data packet
together with its SYN/ACK packet. The client due to timing almost
certainly gets the SYN/ACK before considering the data packet,
and will respond with the final ACK before processing the data
packet.
In IDS mode we will accept the data packet and rely on the reassembly
engine to warn us if the packet was indeed injected.
In IPS mode we will drop the packet. In the packet loss case we will
rely on retransmissions to get the session back up and running. For
the injection case we blocked this injection attempt.
commit e1ef57c848bbe4e567d5d4b66d346a742e3f77a1
Author: Victor Julien <victor at inliniac.net>
Date: Wed Jan 24 15:59:14 2018 +0100
stream: still inspect packets dropped by stream
The detect engine would bypass packets that are set as dropped. This
seems sane, as these packets are going to be dropped anyway.
However, it lead to the following corner case: stream events that
triggered the drop could not be matched on the rules. The packet
with the event wouldn't make it to the detect engine due to the bypass.
This patch changes the logic to not bypass DROP packets anymore.
Packets that are dropped by the stream engine will set the no payload
inspection flag, so avoid needless cost.
commit 700781c53b847eb73e35742a3b44b934d4fda9e6
Author: Victor Julien <victor at inliniac.net>
Date: Tue Nov 21 10:31:58 2017 +0100
enip: support gaps
Due to a bug in the GAP handling the TCP layer the parser would already
get data after GAPs before.
commit 89dc05d4a6ee05aab03ae6baa6c56be01056f33a
Author: Victor Julien <victor at inliniac.net>
Date: Fri Nov 10 23:03:16 2017 +0100
stream/app-layer: fix GAP handling issue
Fix case where data after GAP was processed as in order data by app-layer.
This happened even if protocol parser did not register to accept GAPs.
commit 251156e2539396655eea2852182565a3a68b640f
Author: Victor Julien <victor at inliniac.net>
Date: Mon Jan 29 11:26:01 2018 +0100
pcre: don't leak memory in data extraction
commit 80f2fbac6ebc1b9114175068647d6d1c44b46776
Author: Pascal Delalande <pdl35 at free.fr>
Date: Tue Jan 23 21:18:41 2018 +0100
rust/tftp: eve logging with rust
commit b9cf49e933d8216e31136ec4b64fc46653d6d729
Author: Clement Galland <clement.galland at epita.fr>
Date: Fri Oct 20 07:42:37 2017 +0000
rust/tftp: add tftp parsing and logging
TFTP parsing and logging written in Rust.
Log on eve.json the type of request (read or write), the name of the file and
the mode.
Example of output:
"tftp":{"packet":"read","file":"rfc1350.txt","mode":"octet"}
commit 0ff60f65ec4cf4d71d68a28da4c5d5c161da2176
Author: Pascal Delalande <pdl35 at free.fr>
Date: Fri Jan 26 22:11:33 2018 +0100
doc: update filestore for file hash extraction
Update for extraction based on md5, sha1 and sha256
commit e8939335eaccd9cf20141f299e8781c210af863b
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 17:55:17 2018 +0100
rust/nfs: explicitly handle GAPs from C
It seems that Rust optimizes this code in such a way that it
passes the null ptr along as real data.
if buf.as_ptr().is_null() && input_len > 0 {
commit 2c3c8f8b85a0abd8bc12b546fcc9a77d084d6c0b
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 15:31:35 2018 +0100
rust/filetracker: if file API return error, trunc file
commit d27ed5957faf709b900bf5a4393f4968dc31f961
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 14:56:05 2018 +0100
rust/nfs: fix read reply handling
READ replies with large data chunks are processed partially to avoid
queuing too much data. When the final chunk was received however, the
start of the chunk would already tag the transaction as 'done'. The
more aggressive tx freeing that was recently merged would cause this
tx to be freed before the rest of the in-progress chunk was done.
This patch delays the tagging of the tx until the final data has been
received.
commit d75d9d0b45027c8ab9a57be57855c57e03e53bbb
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 11:02:33 2018 +0100
file: minor cleanups
commit ce08a43bdaf5fc5ea81e76d6c5cd30b459bdaba7
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 10:32:13 2018 +0100
file: use enum for state
Makes debugging easier.
commit 3a2e4614d074ac66df1f7b5a3ec15d9da9c78660
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 10:05:55 2018 +0100
rust/file: handle file open errors
commit 45c5030ff070c2ba08409fcd7b6c5a48f0a34f22
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 09:56:37 2018 +0100
rust/file: change return type for FileOpenFileWithId
Make it int so we can easily check it in Rust. No consumer used the
File pointer that was returned before anyway.
commit 288ddc95acf148837729233f7ca1f870f835b9b8
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 25 09:47:02 2018 +0100
rust/core: comment cleanup
commit 4a89d939fcb1a0718916f7b3ffce138be50e7138
Author: Jason Ish <ish at unx.ca>
Date: Thu Jan 25 16:19:57 2018 -0600
.gitignore: only ignore *.yaml in root directory
-----------------------------------------------------------------------
Summary of changes:
.gitignore | 2 +-
.travis.yml | 12 +-
autogen.sh | 5 +
doc/userguide/configuration/suricata-yaml.rst | 110 +----
doc/userguide/file-extraction/file-extraction.rst | 25 +
doc/userguide/output/eve/eve-json-format.rst | 20 +
doc/userguide/output/eve/eve-json-output.rst | 112 +----
doc/userguide/partials/eve-log.yaml | 131 ++++++
doc/userguide/rules/differences-from-snort.rst | 2 +-
doc/userguide/rules/file-keywords.rst | 44 ++
rules/files.rules | 5 +
rules/stream-events.rules | 7 +-
rust/Cargo.toml.in | 6 +-
rust/Makefile.am | 11 +-
rust/gen-c-headers.py | 2 +
rust/src/core.rs | 9 +-
rust/src/filecontainer.rs | 5 +-
rust/src/filetracker.rs | 31 +-
rust/src/lib.rs | 1 +
rust/src/nfs/nfs.rs | 73 ++-
rust/src/{lib.rs => tftp/log.rs} | 46 +-
rust/src/{dns => tftp}/mod.rs | 8 +-
rust/src/tftp/tftp.rs | 141 ++++++
src/Makefile.am | 4 +-
src/app-layer-detect-proto.c | 4 +
src/app-layer-enip.c | 8 +
src/app-layer-htp-file.c | 28 --
src/app-layer-htp.c | 509 ++++++++++++++++-----
src/app-layer-htp.h | 27 +-
src/app-layer-nfs-tcp.c | 16 +-
src/app-layer-parser.c | 2 +
src/app-layer-protos.c | 3 +
src/app-layer-protos.h | 1 +
src/app-layer-tftp.c | 331 ++++++++++++++
src/{app-layer-ntp.h => app-layer-tftp.h} | 20 +-
src/app-layer.c | 40 +-
src/decode-events.c | 1 +
src/decode-events.h | 1 +
src/detect-pcre.c | 10 +-
src/detect.c | 7 +-
src/output-json-alert.c | 157 +++----
src/output-json-dnp3.c | 12 +-
src/output-json-dns.c | 12 +
src/output-json-drop.c | 7 +
src/output-json-email-common.h | 1 +
src/output-json-flow.c | 7 +-
src/output-json-http.c | 6 +
src/{output-json-vars.c => output-json-metadata.c} | 107 +++--
src/{output-json-vars.h => output-json-metadata.h} | 10 +-
src/output-json-netflow.c | 9 +
src/output-json-nfs.c | 6 +
src/output-json-smtp.c | 5 +
src/output-json-ssh.c | 6 +
src/output-json-tftp.c | 199 ++++++++
src/{detect-target.h => output-json-tftp.h} | 10 +-
src/output-json-tls.c | 6 +
src/output-json.c | 138 +++++-
src/output-json.h | 3 +-
src/output.c | 7 +-
src/rust.h | 2 +-
src/stream-tcp-list.c | 12 +
src/stream-tcp-reassemble.c | 5 +-
src/stream-tcp.c | 53 ++-
src/suricata-common.h | 3 +-
src/util-file.c | 22 +-
src/util-file.h | 4 +-
src/util-lua-common.c | 8 +
src/util-profiling.c | 2 +-
suricata.yaml.in | 15 +-
69 files changed, 1939 insertions(+), 715 deletions(-)
create mode 100644 doc/userguide/partials/eve-log.yaml
copy rust/src/{lib.rs => tftp/log.rs} (59%)
copy rust/src/{dns => tftp}/mod.rs (90%)
create mode 100644 rust/src/tftp/tftp.rs
create mode 100644 src/app-layer-tftp.c
copy src/{app-layer-ntp.h => app-layer-tftp.h} (68%)
rename src/{output-json-vars.c => output-json-metadata.c} (59%)
rename src/{output-json-vars.h => output-json-metadata.h} (81%)
create mode 100644 src/output-json-tftp.c
copy src/{detect-target.h => output-json-tftp.h} (79%)
hooks/post-receive
--
OISF
More information about the Oisf-devel
mailing list