[Oisf-devel] [COMMIT] OISF branch, master, updated. suricata-4.0.1-293-gd588237
OISF Git
noreply at openinfosecfoundation.org
Tue Jan 23 07:00:38 UTC 2018
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "OISF".
The branch, master has been updated
via d5882372357e957fd38c658f012b5d5943c9923e (commit)
via 07738af8682d55c985a55fefcaaa6c4110b4601f (commit)
via 5e65d79be0473e303af9924ec50dac624dca6904 (commit)
via 11cb84ad35a45748b2192c7a47d5a4797807dcec (commit)
via cf2feeecf430bd7b5f3d2ec1fcf4d7e98981d692 (commit)
via 31648913e6c6763ba13f1aa0c50684f8dc06cbdc (commit)
via 91827568918a6ad84b808d4c3f63100bfab50b4d (commit)
via fdbcf948c1ecac6b4f4c63a7996dcd14f537f6bc (commit)
via f75df19c35b6a36c00083ebaf7d06ecf4efac70e (commit)
via 2b8f34a1b5e7ac7b9b756d5abf1fdbbe518a84dc (commit)
via 1df00749df6b250db79e9ad3760dea1a2b99cfb9 (commit)
via 7a96d18f36817ec8280546b248ef0d0afe16f88a (commit)
via 3d9ade9c351c736e24e2fb4b67cbe63756c04cd8 (commit)
via 5f890296b53afacd729e5d77f09aed9f27540aa5 (commit)
via 4be73fac5e88da9aa14bddc829db2e4410c4b1fa (commit)
via af51e0f5a12865a1d805f7010ee02e8b7444aa38 (commit)
via 8cda2a43510d3465a8134e2263973919ca990223 (commit)
via edb9c59526ffd562a31747568855cb224f714d7f (commit)
via 98eca55241fe80a9ffc400269e828890a1c9b233 (commit)
via 00b0a41b55dc6f876c313a228c470ab2fbdd25a7 (commit)
via d0f19891b4078cd696b28ddb77361b1fbbc9d626 (commit)
via 73b59bda532a6bb232deddf79b5287b6ae0bb20f (commit)
via 1bed6e9caecad5b661874d1166901217609f80dd (commit)
via daeb8fd34306fb7a6e661ba884509fdc42776737 (commit)
via 51d429b3b16863996686da7ad43353182ad6011e (commit)
from d05355db3d6e2752ae0582a7ea8c1a0f08bde91c (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit d5882372357e957fd38c658f012b5d5943c9923e
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 19 16:14:33 2017 +0100
detect/content: implement endswith
commit 07738af8682d55c985a55fefcaaa6c4110b4601f
Author: Victor Julien <victor at inliniac.net>
Date: Sat Dec 16 11:44:18 2017 +0100
detect/content: introduce startswith modifier
Add startswith modifier to simplify matching patterns at the start
of a buffer.
Instead of:
content:"abc"; depth:3;
This enables:
content:"abc"; startswith;
Especially with longer patterns this makes the intention of the rule
more clear and eases writing the rules.
Internally it's simply a shorthand for 'depth:<pattern len>;'.
Ticket https://redmine.openinfosecfoundation.org/issues/742
commit 5e65d79be0473e303af9924ec50dac624dca6904
Author: Victor Julien <victor at inliniac.net>
Date: Thu Nov 2 12:44:29 2017 +0100
detect: bypass merge sort call if possible
commit 11cb84ad35a45748b2192c7a47d5a4797807dcec
Author: Victor Julien <victor at inliniac.net>
Date: Wed Nov 1 17:38:22 2017 +0100
detect: profiling update for new detect code
commit cf2feeecf430bd7b5f3d2ec1fcf4d7e98981d692
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 31 17:48:18 2017 +0100
detect/prefilter: redo profiling
commit 31648913e6c6763ba13f1aa0c50684f8dc06cbdc
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 23 10:48:56 2017 +0200
detect/profiling: postpone setup
Do this to allow for including of runtime buffer registrations.
commit 91827568918a6ad84b808d4c3f63100bfab50b4d
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 23 12:29:55 2017 +0200
detect/fast-pattern: use registered buffers for check
commit fdbcf948c1ecac6b4f4c63a7996dcd14f537f6bc
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 12 08:49:20 2018 +0100
output/filedata: call loggers on both directions
commit f75df19c35b6a36c00083ebaf7d06ecf4efac70e
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 11 20:42:54 2018 +0100
output/file: run file loggers in both directions
This avoids the wait for injected packets when file is already ready
to be logged.
commit 2b8f34a1b5e7ac7b9b756d5abf1fdbbe518a84dc
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 11 13:50:59 2018 +0100
detect/state: clean up old code
commit 1df00749df6b250db79e9ad3760dea1a2b99cfb9
Author: Victor Julien <victor at inliniac.net>
Date: Wed Jan 10 19:17:33 2018 +0100
detect: fix multiple files per tx inspect
Fix the inspection of multiple files in a single TX, where new files
may be added to the TX after inspection started.
Assign the hard coded id DE_STATE_FLAG_FILE_INSPECT to the file
inspect engine.
Make sure that sigs that do file inspection and don't match on the
current file always store a detailed state. This state will include
the DE_STATE_FLAG_FILE_INSPECT flag.
When the app-layer indicates a new file is available, for each sig
that has the DE_STATE_FLAG_FILE_INSPECT flag set, reset part of the
state so that the sig is evaluated again.
commit 7a96d18f36817ec8280546b248ef0d0afe16f88a
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 11 19:02:43 2017 +0200
app-layer: improve async and out of order txs
Free txs that are done out of order if we can. Some protocol
implementations have transactions running in parallel, where it is
possible that a tx that started later finishes earlier than other
transactions. Support freeing those.
Also improve handling on asynchronious transactions. If transactions
are unreplied, e.g. in the dns flood case, the parser may at some
point free transactions on it's own. Handle this case in
the app-layer engine so that the various tracking id's (inspect, log,
and 'min') are updated accordingly.
Next, free txs much more aggressively. Instead of freeing old txs
at the app-layer parsing stage, free all complete txs at the end
of the flow-worker. This frees txs much sooner in many cases.
commit 3d9ade9c351c736e24e2fb4b67cbe63756c04cd8
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 10 12:40:03 2017 +0200
detect/prefilter: show prefilter engine id space
commit 5f890296b53afacd729e5d77f09aed9f27540aa5
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 10 11:54:35 2017 +0200
app-layer: warn that MpmIDs API is no longer used
Remove implementation.
commit 4be73fac5e88da9aa14bddc829db2e4410c4b1fa
Author: Victor Julien <victor at inliniac.net>
Date: Tue Jan 9 17:52:19 2018 +0100
detect/flowbits: apply state knowledge
When stateless rules are depending on a flowbit being set by a stateful
rule, the inspection order is almost certainly wrong.
Switch stateless rules depending on stateful rules to being stateful.
This is used to turn 'TCP stream' inspecting rules (which are stateless
unless mixed with stateful keywords) into stateful rules.
commit af51e0f5a12865a1d805f7010ee02e8b7444aa38
Author: Victor Julien <victor at inliniac.net>
Date: Sun Oct 8 11:42:30 2017 +0200
detect: rewrite of the detect engine
Use per tx detect_flags to track prefilter. Detect flags are used for 2
things:
1. marking tx as fully inspected
2. tracking already run prefilter (incl mpm) engines
This supercedes the MpmIDs API for directionless tracking
of the prefilter engines.
When we have no SGH we have to flag the txs that are 'complete'
as inspected as well.
Special handling for the stream engine:
If a rule mixes TX inspection and STREAM inspection, we can encounter
the case where the rule is evaluated against multiple transactions
during a single inspection run. As the stream data is exactly the same
for each of those runs, it's wasteful to rerun inspection of the stream
portion of the rule.
This patch enables caching of the stream 'inspect engine' result in
the local 'RuleMatchCandidateTx' array. This is valid only during the
live of a single inspection run.
Remove stateful inspection from 'mask' (SignatureMask). The mask wasn't
used in most cases for those rules anyway, as there we rely on the
prefilter. Add a alproto check to catch the remaining cases.
When building the active non-mpm/non-prefilter list check not just
the mask, but also the alproto. This especially helps stateful rules
with negated mpm.
Simplify AppLayerParserHasDecoderEvents usage in detection to only
return true if protocol detection events are set. Other detection is done
in inspect engines.
Move rule group lookup and handling into it's own function. Handle
'post lookup' tasks immediately, instead of after the first detect
run. The tasks were independent of the initial detection.
Many cleanups and much refactoring.
commit 8cda2a43510d3465a8134e2263973919ca990223
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 16 15:30:28 2017 +0200
rust/nfs: add support for detect_flags API
commit edb9c59526ffd562a31747568855cb224f714d7f
Author: Victor Julien <victor at inliniac.net>
Date: Wed Jan 17 15:13:55 2018 +0100
dns: support detect flags
commit 98eca55241fe80a9ffc400269e828890a1c9b233
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 13 08:12:21 2017 +0200
rust/dns: implement detect_flags API
commit 00b0a41b55dc6f876c313a228c470ab2fbdd25a7
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 10 11:54:02 2017 +0200
http: move from MpmIDs to DetectFlags API
commit d0f19891b4078cd696b28ddb77361b1fbbc9d626
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 17 10:19:41 2017 +0200
ssl/tls: use DetectFlags API
commit 73b59bda532a6bb232deddf79b5287b6ae0bb20f
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 16 15:56:53 2017 +0200
smtp: implement DetectFlags API
commit 1bed6e9caecad5b661874d1166901217609f80dd
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 16 15:48:05 2017 +0200
ssh: implement DetectFlags API
commit daeb8fd34306fb7a6e661ba884509fdc42776737
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 10 11:17:52 2017 +0200
app-layer: detect flags API calls
Add API meant to replace the MpmIDs API. It uses a u64 for each direction
in a tx to keep track of 2 things:
1. is inspection done?
2. which prefilter engines (like mpm) are already completed
commit 51d429b3b16863996686da7ad43353182ad6011e
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 13 11:01:38 2017 +0200
flowbits: analyze and dump to json
Analyze flowbits to find which bits are only checked.
Track whether they are set and checked on the same level of 'statefulness'
for later used.
Dump flowbits to json including the sids that set/check etc the bit.
-----------------------------------------------------------------------
Summary of changes:
doc/userguide/rules/payload-keywords.rst | 18 +
rust/src/dns/dns.rs | 28 +
rust/src/nfs/nfs.rs | 31 +
src/Makefile.am | 5 +-
src/app-layer-dns-common.c | 20 +
src/app-layer-dns-common.h | 6 +
src/app-layer-dns-tcp.c | 2 +
src/app-layer-dns-udp-rust.c | 13 +
src/app-layer-dns-udp.c | 2 +
src/app-layer-htp.c | 36 +-
src/app-layer-htp.h | 5 +-
src/app-layer-nfs-tcp.c | 13 +
src/app-layer-nfs-udp.c | 14 +
src/app-layer-parser.c | 315 ++++---
src/app-layer-parser.h | 47 +-
src/app-layer-protos.h | 1 +
src/app-layer-smtp.c | 31 +-
src/app-layer-smtp.h | 4 +
src/app-layer-ssh.c | 21 +-
src/app-layer-ssh.h | 4 +-
src/app-layer-ssl.c | 21 +-
src/app-layer-ssl.h | 5 +-
src/decode.h | 1 -
src/detect-app-layer-event.c | 36 -
src/detect-content.h | 1 +
src/detect-depth.c | 60 ++
src/detect-engine-build.c | 121 +--
src/detect-engine-build.h | 2 +-
src/detect-engine-iponly.c | 4 +-
src/detect-engine-iponly.h | 4 +-
src/detect-engine-payload.c | 7 -
src/detect-engine-prefilter.c | 142 ++-
src/detect-engine-prefilter.h | 19 +-
src/detect-engine-profile.c | 5 +-
src/detect-engine-profile.h | 3 +-
src/detect-engine-register.h | 2 +
src/detect-engine-siggroup.c | 2 +
src/detect-engine-state.c | 1009 +++++++++------------
src/detect-engine-state.h | 104 +--
src/detect-engine.c | 55 +-
src/detect-fast-pattern.c | 31 +
src/detect-fast-pattern.h | 26 +-
src/detect-file-data.c | 5 -
src/detect-flowbits.c | 368 +++++++-
src/detect-http-client-body.c | 1 -
src/detect-http-cookie.c | 11 -
src/detect-http-header-names.c | 9 -
src/detect-http-header.c | 10 -
src/detect-http-headers-stub.h | 9 -
src/detect-http-hh.c | 10 -
src/detect-http-hrh.c | 10 -
src/detect-http-method.c | 9 -
src/detect-http-protocol.c | 9 -
src/detect-http-raw-header.c | 9 -
src/detect-http-raw-uri.c | 1 -
src/detect-http-request-line.c | 10 -
src/detect-http-response-line.c | 10 -
src/detect-http-start.c | 9 -
src/detect-http-stat-code.c | 10 -
src/detect-http-stat-msg.c | 10 -
src/detect-http-ua.c | 10 -
src/detect-http-uri.c | 1 -
src/detect-isdataat.c | 30 +
src/detect-offset.c | 4 +
src/detect-ssh-proto.c | 9 -
src/detect-ssh-software.c | 9 -
src/detect.c | 1224 +++++++++++++++++++-------
src/detect.h | 59 +-
src/flow-worker.c | 4 +
src/output-file.c | 88 +-
src/output-filedata.c | 78 +-
src/suricata-common.h | 10 +-
src/suricata.c | 4 +-
src/tests/detect-engine-content-inspection.c | 44 +
src/util-error.c | 2 +
src/util-error.h | 2 +
src/util-profiling-prefilter.c | 311 +++++++
src/util-profiling.c | 109 +--
src/util-profiling.h | 65 +-
src/util-var-name.c | 13 +-
src/util-var-name.h | 1 +
suricata.yaml.in | 5 +
82 files changed, 2926 insertions(+), 1942 deletions(-)
create mode 100644 src/util-profiling-prefilter.c
hooks/post-receive
--
OISF
More information about the Oisf-devel
mailing list