[Oisf-devel] [COMMIT] OISF branch, master, updated. suricata-3.0.1-100-g9b6e292
OISF Git
noreply at openinfosecfoundation.org
Tue Apr 5 09:14:53 UTC 2016
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "OISF".
The branch, master has been updated
via 9b6e292a28ac862684056f8dee912f0629590cbf (commit)
via f1ee1258107945f6ab8316093e5e7241462f95b5 (commit)
via e2789a87c2a7d3a575fc197cae49a399085e30f7 (commit)
via adc453eec6e61db7520bb3fe0dd8ad4a249486a6 (commit)
via ef6f347f84f556dace8acce4a8fcc39d2c263aba (commit)
via d5c6d08bc816321172992c17e83edb0575adf724 (commit)
via 9bd8197009ff36ffb7dbf8f6362f6f8570b0bd2b (commit)
via e0111fbb904524de93a03cb72b2cb4066887e66f (commit)
via 5f676167a357a7cf679b6481d6c17c4c56dc44a9 (commit)
via d6ba01b1b73dfccc6ee009f5ce3d0880cbc2b6d1 (commit)
via 5b1d75f0bd67175415ed355eb477392d1a0d166f (commit)
via 725d6c37395f21e62c93da34530a383500c07e67 (commit)
via ac2c206359922a2296b5d019be0355ba23ae430b (commit)
via 1dd135d512bb536287d92895cdd6cf33e2ecc573 (commit)
via 6ef27c9f92fd68f7b0a389957e451a73e7274e9f (commit)
via 79a96b2b9096e5850f276d98ba621439d892e7ff (commit)
via 157ca89dd7e25b88a25eb64143b8335663817d69 (commit)
via fdd05e8fb40887a02dc138cbc0aaa1dfdefd6003 (commit)
via 4e91f6b1e65a6a46fbb88dcc411508790a97a801 (commit)
via 2b84387ea4571775d1b3dd64b3170610ae13f072 (commit)
via 0311f01b970d7ca9d4e5786f6f3a3e3782429cb4 (commit)
via a2223bb066955c76848760bd76f3a29bd7994153 (commit)
via 87f3adbe4c25a3047acf40670eba151af49cbd78 (commit)
via 30755265ee4bbb424ce221e06dcb586ece0f8846 (commit)
via af3bf3dc7a6580b6cdca273201eb82998715d94f (commit)
via de273d88ccaf30bcf32e86b392e72534ddc6189c (commit)
via e43c4f3ea2b9d72a58df6e4c4ada9057bcf01101 (commit)
via 58576605682c5f0863fb8e2150db498e17fbf5b3 (commit)
via 6bb2b001a33e542f840dfef0e1b0d01db86f0d15 (commit)
via e57e7d1b961e77f5d8cec28adfcbb24ebf8ec50c (commit)
via eb19fc4c7bbcc00e781dd5fe2ed1f74f13bd9671 (commit)
via caea596ce5682fadcb113a6a7c8ab4a2bc15a1b2 (commit)
via 92c571b26d4518ad630117a06990d80f92dfb4f6 (commit)
via 722e2dbf7cb916845fc50d5408000dcc6da9eabc (commit)
via e310a033be69d40026cb6ae9f605da78699d37c8 (commit)
via c880b79f45b0ddc269e5710a45549b23da748743 (commit)
via c804102a9a2cf3b53bc45fb912cfa462dc52f5b3 (commit)
via 9b3d4f7e2483f907698d6e16aca99a2f4fd5b9ce (commit)
via ba9d43cce56932ea98603084648614e6a4523064 (commit)
via 9e71ef4c3bc878f018cdd8fadfab5e33ada17fb6 (commit)
via 46734ec41be6eb7121c3e1a5b71e2ae245ce8640 (commit)
via c1ad08d11e70f02b6f913c51ec59a6748ce177c5 (commit)
via 4e8e5917153d8bf1b06c7a3f56457f540034a8d1 (commit)
via c87fcb29ffb95feace6c1c8decc668d84c198ac7 (commit)
via 7c94077892cea1b1153c8a814c126b13eb81c08c (commit)
via a7d126738a3f4c82b4b71c5787cae48cabc12bff (commit)
via cbf80de6fe6de423344a40597aca1891b192fed2 (commit)
via a96fa0fc2fca9763ad007fd692abd88b8d003315 (commit)
via e6248b0dbe849274871a7291019ad337fba90e02 (commit)
via 1f7e33a4b002167045e98d4878b45b73d6401194 (commit)
via e8c95980354bc1e0ce2dfdc039b2684095d626d5 (commit)
via 2ab20d0b9bfa8af1688c886869938192e18f2f26 (commit)
via 4ba1ac55f063d488f6f3b0b513b0f93317c753f0 (commit)
via 810d2d3ec692053d860009271a9cb2d3827c7fc0 (commit)
via fa885e1d85f7522283c77eb4f6c9921006eb1bc5 (commit)
via 69d38a3222415dc2c7fc2ea2686f4c1c9b0dd2d4 (commit)
via 4edb03ab9d6ede0afb539687a36829913691f1b2 (commit)
via cd8283bb729c4437f6a532b80c0b62d75ee8b76e (commit)
via b2fcb17859f02dcf6d69c4d2836d44b16a3c73dc (commit)
via a34be2300251b0e0bdc4baaf31751e30f3ba786e (commit)
via b84d6d402f7d73a892a819ba038b67ee2e5a946c (commit)
via 3c184c19cd56be39974463b3d4f2e43ceee9a837 (commit)
via 5772f526dcd235f80d3c9f65bb9fb5a7c1d8c0de (commit)
via 2ce03fbabb3468c72cf7d4c27fa64f557c97d10c (commit)
via e48d745ed71d6a14bfca1b13d088c041a70c9174 (commit)
via 26517b8b61d69ac21ef2d48ee8301639be005ec2 (commit)
via 102a82fc7bc47b7cf3c4c20bfbbd5da8363f987e (commit)
via fac2cc056077c44ed826219bf51004b253dc1e56 (commit)
via f0ba00e51dfdb0f3fd0df78cd065183e8f94ed22 (commit)
via 10b049304f36f55536b28ffb30c818f6ae795ce1 (commit)
via 2ee9bf2aef46856d74d4b1a3dcc1b0ff7f5fd23c (commit)
via 1f70ccfc23cc64baee66f48d9540bc1c84da5abf (commit)
via ae80ed596408ebe0cae58d73537d295ecd9de3b7 (commit)
via df529b13ce7d851ac5ba08c81afeb3e94c3fa212 (commit)
via a3928123029217099dd04dc1ea2d4fbf3ad6632e (commit)
via e570b10abe72b00c1768f3385cc52ae522a17de1 (commit)
via eda9552e9531471b0ad5890698ee2d28ad424a6e (commit)
via d82df4eb8b50064d5e6e5da6b1fe684eb190b08d (commit)
via 4223ce9aba0b8303e7b11092ce7e490ac026610a (commit)
via 9ae4cb9e0234ca7f64c73a30b97598a00ca41a11 (commit)
via fd5a06017d84dfe8d8042cd6e19bd315a959cb7c (commit)
via 27e63a1e1134812b414e189caa90e53d8266ed86 (commit)
via e75a93b1256e604d2528baff76b9c236057dba64 (commit)
via c71c991669d52c28bd2619489d39462206980559 (commit)
via bb662a65f8630723cd4fef361b7d7313fed74e77 (commit)
via 18dd54dfa77eddecc8a94da0a1e30041abfccddf (commit)
via bfe49b60f7a0fdad8ded686b0291d04fde997d8a (commit)
via b7d81fc3b024412ba0975e9bb691ea6f74924319 (commit)
via f720dfd21e72596e0dde9ad6dbe575e4949ad66d (commit)
via 66b3dba676c50bfdc9f83a5d4ae94e6c6dc907e9 (commit)
via b3dcdb10bedb8044ac8e1cad65498df9abcd422b (commit)
via 14d9ce7b2ec6379187d88d651c0d5231d75a5755 (commit)
via 0d3f671b55d918d3e9c44222162c081df5c666b8 (commit)
via 4f8e1f59a6c3d76f49863ddaafb97e04bfecc092 (commit)
via 262abbb49f579073ff5288dca104a5c3dab486f2 (commit)
via 58e533858bff2146ac07a2491314ef36e899961f (commit)
via 0987fd16dce9a05d7f2b0fc61df4ad687368dc60 (commit)
via 9c2e374a3d5115bc392164fd567b3afd9b5502d0 (commit)
via 887ddf1ed89f6d22080bce7b9a99c2c7deeb554e (commit)
from 3781b00dbcc4d8694886e82fef07b24e0e9567b7 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 9b6e292a28ac862684056f8dee912f0629590cbf
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 13:11:04 2016 +0100
mpm: remove unused max pattern len field
commit f1ee1258107945f6ab8316093e5e7241462f95b5
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 12:58:55 2016 +0100
detect-flowvar: shrink mem structure by 8 bytes
commit e2789a87c2a7d3a575fc197cae49a399085e30f7
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 12:55:23 2016 +0100
detect: shrink IPOnlyCIDRItem with 8 bytes
commit adc453eec6e61db7520bb3fe0dd8ad4a249486a6
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 12:48:08 2016 +0100
detect-port: improve comment about sgh pointer
commit ef6f347f84f556dace8acce4a8fcc39d2c263aba
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 12:32:49 2016 +0100
detect-address: remove sgh pointer as it's unused
commit d5c6d08bc816321172992c17e83edb0575adf724
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 12:28:33 2016 +0100
detect-port: remove debug mem counters
commit 9bd8197009ff36ffb7dbf8f6362f6f8570b0bd2b
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 12:27:11 2016 +0100
detect-address: remove debug mem counters
commit e0111fbb904524de93a03cb72b2cb4066887e66f
Author: Victor Julien <victor at inliniac.net>
Date: Fri Mar 25 12:00:52 2016 +0100
detect grouping: remove debug mem counters
commit 5f676167a357a7cf679b6481d6c17c4c56dc44a9
Author: Victor Julien <victor at inliniac.net>
Date: Wed Mar 2 16:38:02 2016 +0100
detect grouping: make json dump configurable
Make the rule grouping dump to rule_group.json configurable.
detect:
profiling:
grouping:
dump-to-disk: false
include-rules: false # very verbose
include-mpm-stats: false
commit d6ba01b1b73dfccc6ee009f5ce3d0880cbc2b6d1
Author: Victor Julien <victor at inliniac.net>
Date: Wed Mar 2 13:37:14 2016 +0100
detect: make port whitelisting configurable
Make the port grouping whitelisting configurable. A whitelisted port
ends up in it's own port group.
detect:
grouping:
tcp-whitelist: 80, 443
udp-whitelist: 53, 5060
No portranges are allowed at this point.
commit 5b1d75f0bd67175415ed355eb477392d1a0d166f
Author: Victor Julien <victor at inliniac.net>
Date: Wed Nov 25 17:27:56 2015 +0100
detect: suppress output
commit 725d6c37395f21e62c93da34530a383500c07e67
Author: Victor Julien <victor at inliniac.net>
Date: Mon Nov 23 19:03:47 2015 +0100
yaml: convert detect-engine to just detect
Instead of detect-engine which used a list for no good reason, use a
simple map now.
detect:
profile: medium
custom-values:
toclient-groups: 3
toserver-groups: 25
sgh-mpm-context: auto
inspection-recursion-limit: 3000
# If set to yes, the loading of signatures will be made after the capture
# is started. This will limit the downtime in IPS mode.
#delayed-detect: yes
commit ac2c206359922a2296b5d019be0355ba23ae430b
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 28 21:47:37 2015 +0100
mpm: clean up builtin mpm setup, enable single/full
commit 1dd135d512bb536287d92895cdd6cf33e2ecc573
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 28 20:52:00 2015 +0100
mpm: always cleanup factory
commit 6ef27c9f92fd68f7b0a389957e451a73e7274e9f
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 28 17:02:40 2015 +0100
mpm: allow app buffer shared/unique
Allow setting of shared or unique setting per app buffer type:
e.g. detect.mpm.http_uri.shared=true
commit 79a96b2b9096e5850f276d98ba621439d892e7ff
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 28 09:17:45 2015 +0100
mpm: refactor 'single' setup handling
commit 157ca89dd7e25b88a25eb64143b8335663817d69
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 28 08:37:28 2015 +0100
mpm: remove useless flag from factory
commit fdd05e8fb40887a02dc138cbc0aaa1dfdefd6003
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 28 08:32:29 2015 +0100
mpm: remove unused app proto factory
commit 4e91f6b1e65a6a46fbb88dcc411508790a97a801
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 28 08:07:28 2015 +0100
mpm: in factory register, consider name const
commit 2b84387ea4571775d1b3dd64b3170610ae13f072
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 27 21:49:00 2015 +0100
detect: work around cocci limitation
commit 0311f01b970d7ca9d4e5786f6f3a3e3782429cb4
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 26 17:26:49 2015 +0100
rule grouping: speed up port based grouping
Create a hash table of unique DetectPort objects before trying to
create a unique list of these objects. This safes a lot of cycles
in the creation of the list.
commit a2223bb066955c76848760bd76f3a29bd7994153
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 26 17:04:47 2015 +0100
mpm: consify packet/stream search
commit 87f3adbe4c25a3047acf40670eba151af49cbd78
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 26 14:18:37 2015 +0100
detect/mpm: unify packet/stream mpm_ctx pointers
SGH's for tcp and udp are now always only per proto and per direction.
This means we can simply reuse the packet and stream mpm pointers.
The SGH's for the other protocols already used a directionless catch
all mpm pointer.
commit 30755265ee4bbb424ce221e06dcb586ece0f8846
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 21 08:59:04 2015 +0200
http_raw_header: improve mpm progress handling
commit af3bf3dc7a6580b6cdca273201eb82998715d94f
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 21 08:35:24 2015 +0200
detect: optimize sgh layout
commit de273d88ccaf30bcf32e86b392e72534ddc6189c
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 21 08:32:00 2015 +0200
detect: remove unused content minlen tracking
commit e43c4f3ea2b9d72a58df6e4c4ada9057bcf01101
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 21 08:19:21 2015 +0200
mpm: optimize calls
For all mpm wrapper functions, check minlen vs the input buffer to see
if we can bypass the mpm search.
Next to this, make all the function inline. Also constify the input and
do other minor cleanups.
commit 58576605682c5f0863fb8e2150db498e17fbf5b3
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 21 07:36:48 2015 +0200
http_uri: mpm cleanup. Use mpm_ctx's minlen
commit 6bb2b001a33e542f840dfef0e1b0d01db86f0d15
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 20 17:49:32 2015 +0200
mpm: cleanup: move mpm funcs into buffer specific files
commit e57e7d1b961e77f5d8cec28adfcbb24ebf8ec50c
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 20 10:55:41 2015 +0200
mpm: cleanup, remove unused structs and prototypes
commit eb19fc4c7bbcc00e781dd5fe2ed1f74f13bd9671
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 20 10:49:10 2015 +0200
mpm: remove unused structure
commit caea596ce5682fadcb113a6a7c8ab4a2bc15a1b2
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 19 17:39:57 2015 +0200
profiling: output post-prefilter matches
Dump a json record containing all sigs that need to be inspected after
prefilter. Part of profiling. Only dump if threshold is met, which is
currently set by:
--set detect.profiling.inspect-logging-threshold=200
A file called packet_inspected_rules.json is created in the default
log dir.
commit 92c571b26d4518ad630117a06990d80f92dfb4f6
Author: Victor Julien <victor at inliniac.net>
Date: Tue Mar 1 17:42:40 2016 +0100
detect: move sm_list to string funcs to parser code
commit 722e2dbf7cb916845fc50d5408000dcc6da9eabc
Author: Victor Julien <victor at inliniac.net>
Date: Sat Oct 17 00:21:00 2015 +0200
profiling: initial rulegroup tracking
Per rule group tracking of checks, use of lists, mpm matches,
post filter counts.
Logs SGH id so it can be compared with the rule_group.json output.
Implemented both in a human readable text format and a JSON format.
commit e310a033be69d40026cb6ae9f605da78699d37c8
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 27 13:34:55 2015 +0100
detect: assign id to sgh
commit c880b79f45b0ddc269e5710a45549b23da748743
Author: Victor Julien <victor at inliniac.net>
Date: Thu Oct 15 15:22:44 2015 +0200
detect: shrink sgh
Turn list of mpm_ctx pointers into a union so that we don't waste
space. The sgh's for tcp and udp are in one direction only, so the
ts and tc ones are now in the union.
commit c804102a9a2cf3b53bc45fb912cfa462dc52f5b3
Author: Victor Julien <victor at inliniac.net>
Date: Thu Oct 15 14:52:01 2015 +0200
detect: move app_mpms array to init data
commit 9b3d4f7e2483f907698d6e16aca99a2f4fd5b9ce
Author: Victor Julien <victor at inliniac.net>
Date: Thu Oct 15 10:31:05 2015 +0200
mpm: unify & localize mpm pattern (id) handling
So far, the patterns as passed to the mpm's would use global id's that
were shared among all buffers, directions. This would lead to a fairly
large pattern id space. As the mpm algo's use the pattern id's to
prevent duplicate matching through a pattern id based bitarray,
shrinking this space will optimize performance.
This patch implements this. It sets a flag before adding the pattern
to the mpm ctx, instructing the mpm to ignore the provided pid and
handle pids management itself. This leads to a shrinking of the
bitarray size.
This is made possible by the previous work that removes the pid logic
from the code.
Next to this, this patch moves the pattern setup stage to common util
functions. This avoids code duplication.
Update ac, ac-bs and ac-ks to use this.
commit ba9d43cce56932ea98603084648614e6a4523064
Author: Victor Julien <victor at inliniac.net>
Date: Thu Oct 1 13:11:44 2015 +0200
mpm: improve negated mpm
The idea is: if mpm is negated, it's both on mpm and nonmpm sid lists
and we can kick it out in that case during the merge sort.
It only works for patterns that are 'independent'. This means that the
rule doesn't need to only match if the negated mpm pattern is limited
to the first 10 bytes for example.
Or more generally, an negated mpm pattern that has depth, offset,
distance or within settings can't be handled this way. These patterns
are not added to the mpm at all, but just to to non-mpm list. This
makes sense as they will *always* need manual inspection.
Similarly, a pattern that is 'chopped' always needs validation. This
is because in this case we only inspect a part of the final pattern.
commit 9e71ef4c3bc878f018cdd8fadfab5e33ada17fb6
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 13 14:56:01 2015 +0200
detect: remove signature pattern id reference
commit 46734ec41be6eb7121c3e1a5b71e2ae245ce8640
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 13 10:39:54 2015 +0200
mpm: remove unused pmq merge function
commit c1ad08d11e70f02b6f913c51ec59a6748ce177c5
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 13 10:27:26 2015 +0200
detect: remove stream pmq array
commit 4e8e5917153d8bf1b06c7a3f56457f540034a8d1
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 13 08:49:23 2015 +0200
detect mpm: mpm store cleanup
Move all rule modification to the fast_pattern assigment.
commit c87fcb29ffb95feace6c1c8decc668d84c198ac7
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 13 08:33:27 2015 +0200
detect mpm: fast_pattern assignment cleanup
commit 7c94077892cea1b1153c8a814c126b13eb81c08c
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 13 08:44:45 2015 +0200
detect mpm: remove unused mpm flags
commit a7d126738a3f4c82b4b71c5787cae48cabc12bff
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 21:54:32 2015 +0200
detect address: remove unused features
commit cbf80de6fe6de423344a40597aca1891b192fed2
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 21:48:41 2015 +0200
detect-port: cleanup
commit a96fa0fc2fca9763ad007fd692abd88b8d003315
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 21:33:44 2015 +0200
detect: remove unused dport sgh hash
commit e6248b0dbe849274871a7291019ad337fba90e02
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 20:05:42 2015 +0200
detect: clean up sgh's at detect engine free
commit 1f7e33a4b002167045e98d4878b45b73d6401194
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 18:39:57 2015 +0200
detect: remove unused flag
commit e8c95980354bc1e0ce2dfdc039b2684095d626d5
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 18:21:42 2015 +0200
detect sgh: remove unused field
commit 2ab20d0b9bfa8af1688c886869938192e18f2f26
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 17:49:25 2015 +0200
detect: free lookup structures
commit 4ba1ac55f063d488f6f3b0b513b0f93317c753f0
Author: Victor Julien <victor at inliniac.net>
Date: Wed Sep 30 09:59:05 2015 +0200
detect: output sgh stats
Output stats for the rule groups into a json format.
commit 810d2d3ec692053d860009271a9cb2d3827c7fc0
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 9 10:12:11 2015 +0200
detect: add list id to string funcs
commit fa885e1d85f7522283c77eb4f6c9921006eb1bc5
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 10:15:16 2015 +0200
mpm: remove pattern id logic
commit 69d38a3222415dc2c7fc2ea2686f4c1c9b0dd2d4
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 10:41:57 2015 +0200
mpm: ac-bs use internal pattern id tracking
commit 4edb03ab9d6ede0afb539687a36829913691f1b2
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 10:39:16 2015 +0200
mpm: ac use internal pattern id tracking
commit cd8283bb729c4437f6a532b80c0b62d75ee8b76e
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 10:14:25 2015 +0200
smtp: use rule_id mpm support instead of pattern id
commit b2fcb17859f02dcf6d69c4d2836d44b16a3c73dc
Author: Victor Julien <victor at inliniac.net>
Date: Sat Oct 10 17:10:13 2015 +0200
proto detect: in mpm switch to rule id
Use the rule id API instead of pattern id API.
commit a34be2300251b0e0bdc4baaf31751e30f3ba786e
Author: Victor Julien <victor at inliniac.net>
Date: Sat Oct 10 14:36:45 2015 +0200
detect: simplify negated mpm handling
commit b84d6d402f7d73a892a819ba038b67ee2e5a946c
Author: Victor Julien <victor at inliniac.net>
Date: Thu Oct 1 15:33:42 2015 +0200
detect grouping: multiple whitelist conditions
Instead of the binary yes/no whitelisting used so far, use different
values for different sorts of whitelist reasons. The port list will
be sorted by whitelist value first, then by rule count.
The goal is to whitelist groups that have weak sigs:
- 1 byte pattern groups
- SYN sigs
Rules that check for SYN packets are mostly scan detection rules.
They will be checked often as SYN packets are very common.
e.g. alert tcp any any -> any 22 (flags:S,12; sid:123;)
This patch adds whitelisting for SYN-sigs, so that the sigs end up
in as unique groups as possible.
- negated mpm sigs
Currently negated mpm sigs are inspected often, so they are quite
expensive. For this reason, try to whitelist them.
These values are set during 'stage 1', rule preprocessing.
commit 3c184c19cd56be39974463b3d4f2e43ceee9a837
Author: Victor Julien <victor at inliniac.net>
Date: Wed Sep 30 18:26:00 2015 +0200
detect grouping: port based group whitelisting
Whitelist some ports in grouping to make sure they get their own group.
commit 5772f526dcd235f80d3c9f65bb9fb5a7c1d8c0de
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 7 07:08:02 2015 +0200
detect grouping: warn on and fix up bad sigs
Only inspect directionless SYN scan sigs toserver. Issue a warning for
those rules.
commit 2ce03fbabb3468c72cf7d4c27fa64f557c97d10c
Author: Victor Julien <victor at inliniac.net>
Date: Thu Oct 1 19:29:45 2015 +0200
detect: split non-mpm list into syn/nosyn
Since SYN inspecting rules are expensive, this patch splits the
'non-mpm' list (i.e. the rules that are always considered) into
a 'syn' and 'non-syn' list. The SYN list is only inspected if the
packet has the SYN flag set, otherwise the non-syn list is used.
The syn-list contains _all_ rules. The non-syn list contains all
minus the rules requiring the SYN bit in a packet.
commit e48d745ed71d6a14bfca1b13d088c041a70c9174
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 7 06:48:44 2015 +0200
mpm: constify search func args
commit 26517b8b61d69ac21ef2d48ee8301639be005ec2
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 6 15:04:33 2015 +0200
detect: mpm store frees mpm_ctx' it owns
commit 102a82fc7bc47b7cf3c4c20bfbbd5da8363f987e
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 5 19:15:58 2015 +0200
detect: use mpm store for app layer mpms
Rework app-layer mpm setup and registration to make this possible.
commit fac2cc056077c44ed826219bf51004b253dc1e56
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 5 10:25:03 2015 +0200
detect: mpm deduplication
Create hash for mpm's that we can reuse. Have packet/stream mpms
use this.
commit f0ba00e51dfdb0f3fd0df78cd065183e8f94ed22
Author: Victor Julien <victor at inliniac.net>
Date: Sat Oct 3 17:02:30 2015 +0200
detect: remove old unused code
commit 10b049304f36f55536b28ffb30c818f6ae795ce1
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 2 20:29:55 2015 +0200
detect: set new defaults for grouping
commit 2ee9bf2aef46856d74d4b1a3dcc1b0ff7f5fd23c
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 2 20:23:50 2015 +0200
detect: rename groupings vars
commit 1f70ccfc23cc64baee66f48d9540bc1c84da5abf
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 2 20:16:25 2015 +0200
detect: remove unused grouping settings
commit ae80ed596408ebe0cae58d73537d295ecd9de3b7
Author: Victor Julien <victor at inliniac.net>
Date: Wed Sep 30 12:41:42 2015 +0200
detect: make port grouping use config limits
commit df529b13ce7d851ac5ba08c81afeb3e94c3fa212
Author: Victor Julien <victor at inliniac.net>
Date: Wed Sep 30 11:56:42 2015 +0200
detect: change port grouping
Update port grouping logic. Previously it would create one consistent
list w/o overlap. It largely still does this, except for the 'catch
all' port group at the end of the list. This port group contains all
the sigs that didn't fit into the other groups.
commit a3928123029217099dd04dc1ea2d4fbf3ad6632e
Author: Victor Julien <victor at inliniac.net>
Date: Tue Sep 29 18:42:16 2015 +0200
detect: sort/group port sigs
commit e570b10abe72b00c1768f3385cc52ae522a17de1
Author: Victor Julien <victor at inliniac.net>
Date: Tue Sep 29 17:40:59 2015 +0200
detect: display unique sgh count
commit eda9552e9531471b0ad5890698ee2d28ad424a6e
Author: Victor Julien <victor at inliniac.net>
Date: Tue Sep 29 16:46:21 2015 +0200
detect: group proto sghs
commit d82df4eb8b50064d5e6e5da6b1fe684eb190b08d
Author: Victor Julien <victor at inliniac.net>
Date: Mon Sep 28 16:03:48 2015 +0200
detect-mpm: make sgh setup proto aware
Allow multi-proto, multi-direction sgh's.
commit 4223ce9aba0b8303e7b11092ce7e490ac026610a
Author: Victor Julien <victor at inliniac.net>
Date: Tue Sep 29 13:20:20 2015 +0200
detect: remove obsolete grouping code
commit 9ae4cb9e0234ca7f64c73a30b97598a00ca41a11
Author: Victor Julien <victor at inliniac.net>
Date: Mon Sep 28 23:20:03 2015 +0200
detect: debug output
commit fd5a06017d84dfe8d8042cd6e19bd315a959cb7c
Author: Victor Julien <victor at inliniac.net>
Date: Fri Nov 7 23:14:26 2014 +0100
detect: per port and proto rule grouping
Replace tree based approach for rule grouping with a per port (tcp/udp)
and per protocol approach.
Grouping now looks like:
+----+
|icmp+--->
+----+
|gre +--->
+----+
|esp +--->
+----+
other|... |
+----->-----+
| |N +--->
| +----+
|
| tcp +----+ +----+
+----->+ 80 +-->+ 139+-->
| +----+ +----+
|
| udp +----+ +----+
+---+----->+ 53 +-->+ 135+-->
| +----+ +----+
|toserver
+--->
|toclient
|
+--->
So the first 'split' in the rules is the direction: toserver or toclient.
Rules that don't have a direction, are in both branches.
Then the split is between tcp/udp and the other protocols. For tcp and
udp port lists are used. For the other protocols, grouping is simply per
protocol.
The ports used are the destination ports for toserver sigs and source
ports for toclient sigs.
commit 27e63a1e1134812b414e189caa90e53d8266ed86
Author: Victor Julien <victor at inliniac.net>
Date: Mon Sep 28 10:10:58 2015 +0200
detect: track direction and ipproto of sgh
Each SGH has a unique ipproto and direction.
commit e75a93b1256e604d2528baff76b9c236057dba64
Author: Victor Julien <victor at inliniac.net>
Date: Mon Sep 28 10:00:36 2015 +0200
detect: pass ipproto to rule grouping funcs
commit c71c991669d52c28bd2619489d39462206980559
Author: Victor Julien <victor at inliniac.net>
Date: Mon Sep 28 16:02:15 2015 +0200
detect: delay sgh cleanup
commit bb662a65f8630723cd4fef361b7d7313fed74e77
Author: Victor Julien <victor at inliniac.net>
Date: Mon Sep 28 10:17:34 2015 +0200
detect: delay sgh mpm setup
commit 18dd54dfa77eddecc8a94da0a1e30041abfccddf
Author: Victor Julien <victor at inliniac.net>
Date: Fri Nov 7 23:43:45 2014 +0100
Start rule inspect with mask check
commit bfe49b60f7a0fdad8ded686b0291d04fde997d8a
Author: Victor Julien <victor at inliniac.net>
Date: Sun Sep 27 10:33:48 2015 +0200
rule analyzer: add no/both direction warning
commit b7d81fc3b024412ba0975e9bb691ea6f74924319
Author: Victor Julien <victor at inliniac.net>
Date: Wed Oct 7 15:38:58 2015 +0200
detect: SYN flags
Add funcs to see if a rule needs a SYN flag in the packet.
commit f720dfd21e72596e0dde9ad6dbe575e4949ad66d
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 9 11:18:36 2015 +0200
detect: validate http_method pattern
Leading and trailing spaces and tabs are invalid as these are not part
of the buffer as returned by libhtp.
commit 66b3dba676c50bfdc9f83a5d4ae94e6c6dc907e9
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 5 19:00:24 2015 +0200
detect: remove dead code
commit b3dcdb10bedb8044ac8e1cad65498df9abcd422b
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 5 12:40:57 2015 +0200
detect mpm: remove dead code
commit 14d9ce7b2ec6379187d88d651c0d5231d75a5755
Author: Victor Julien <victor at inliniac.net>
Date: Sat Oct 3 17:57:27 2015 +0200
detect/mpm: remove unused max_id param from API
commit 0d3f671b55d918d3e9c44222162c081df5c666b8
Author: Victor Julien <victor at inliniac.net>
Date: Mon Sep 28 11:15:09 2015 +0200
detect: constify mpm/detect funcs
commit 4f8e1f59a6c3d76f49863ddaafb97e04bfecc092
Author: Victor Julien <victor at inliniac.net>
Date: Tue Oct 20 10:19:40 2015 +0200
mpm: remove obsolete mpm algos
Remove: ac-gfbs, wumanber, b2g, b3g.
commit 262abbb49f579073ff5288dca104a5c3dab486f2
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 12:49:06 2015 +0200
mpm: fix ac-ks compilation on cygwin
commit 58e533858bff2146ac07a2491314ef36e899961f
Author: Victor Julien <victor at inliniac.net>
Date: Sun Sep 20 10:17:17 2015 +0200
detect mpm: ac-tile/ac-ks default to single
Use sgh-mpm-context single is it is set to 'auto' when ac-ks is used.
commit 0987fd16dce9a05d7f2b0fc61df4ad687368dc60
Author: Victor Julien <victor at inliniac.net>
Date: Mon Oct 12 08:47:37 2015 +0200
ac-ks: 32bit fixes
commit 9c2e374a3d5115bc392164fd567b3afd9b5502d0
Author: Victor Julien <victor at inliniac.net>
Date: Fri Oct 9 16:21:17 2015 +0200
ac-ks: fix mem leaks
commit 887ddf1ed89f6d22080bce7b9a99c2c7deeb554e
Author: Victor Julien <victor at inliniac.net>
Date: Sat Sep 19 22:59:03 2015 +0200
mpm: introduce ac-ks
Introduce 'ac-ks' or the Kenneth Steele AC implementation. It's
actually 'ac-tile' written by Ken for the Tilera platform. This
patch adds support for it on other architectures as well.
Enable ac-tile for other archs as 'ac-ks'.
Fix a bunch of OOB reads in the loops that triggered ASAN.
-----------------------------------------------------------------------
Summary of changes:
src/Makefile.am | 6 +-
src/app-layer-detect-proto.c | 89 +-
src/app-layer-smtp.c | 9 +-
src/detect-ack.c | 31 +-
src/detect-content.c | 10 +-
src/detect-content.h | 12 +
src/detect-distance.c | 1 -
src/detect-dns-query.c | 35 -
src/detect-engine-address-ipv4.c | 158 -
src/detect-engine-address-ipv6.c | 149 -
src/detect-engine-address.c | 429 --
src/detect-engine-address.h | 1 -
src/detect-engine-analyzer.c | 21 +-
src/detect-engine-analyzer.h | 2 +-
src/detect-engine-dns.c | 65 +
src/detect-engine-filedata-smtp.c | 32 +
src/detect-engine-hcbd.c | 31 +
src/detect-engine-hcd.c | 39 +
src/detect-engine-hhd.c | 40 +
src/detect-engine-hhhd.c | 37 +-
src/detect-engine-hmd.c | 34 +-
src/detect-engine-hrhd.c | 53 +-
src/detect-engine-hrhhd.c | 46 +-
src/detect-engine-hrl.c | 38 -
src/detect-engine-hrud.c | 36 +-
src/detect-engine-hsbd.c | 33 +-
src/detect-engine-hscd.c | 35 +-
src/detect-engine-hsmd.c | 33 +-
src/detect-engine-hua.c | 33 +-
src/detect-engine-mpm.c | 3462 +++++------------
src/detect-engine-mpm.h | 40 +-
src/detect-engine-payload.c | 143 +-
src/detect-engine-port.c | 266 +-
src/detect-engine-port.h | 23 +-
src/detect-engine-profile.c | 136 +
src/{detect-depth.h => detect-engine-profile.h} | 12 +-
src/detect-engine-proto.c | 2 +-
src/detect-engine-proto.h | 2 +-
src/detect-engine-siggroup.c | 1253 +-----
src/detect-engine-siggroup.h | 25 +-
src/detect-engine-uri.c | 102 +-
src/detect-engine.c | 466 +--
src/detect-fast-pattern.c | 1 -
src/detect-flags.c | 38 +
src/detect-flags.h | 3 +
src/detect-http-method.c | 37 +-
src/detect-http-method.h | 1 +
src/detect-ipproto.c | 5 +-
src/detect-parse.c | 89 +-
src/detect-parse.h | 5 +-
src/detect-pcre.c | 58 +-
src/detect-rpc.c | 2 -
src/detect-sameip.c | 31 +-
src/detect-uricontent.c | 80 -
src/detect.c | 4727 +++++++----------------
src/detect.h | 257 +-
src/runmode-unix-socket.c | 1 +
src/suricata.c | 15 +-
src/util-bloomfilter.c | 5 +-
src/util-bloomfilter.h | 8 +-
src/util-error.c | 1 +
src/util-error.h | 1 +
src/util-mpm-ac-bs.c | 477 +--
src/util-mpm-ac-bs.h | 32 +-
src/util-mpm-ac-gfbs.c | 2722 -------------
src/util-mpm-ac-gfbs.h | 110 -
src/util-mpm-ac-tile-small.c | 21 +-
src/util-mpm-ac-tile.c | 632 +--
src/util-mpm-ac-tile.h | 39 +-
src/util-mpm-ac.c | 691 +---
src/util-mpm-ac.h | 32 +-
src/util-mpm-b2g.c | 2832 --------------
src/util-mpm-b2g.h | 127 -
src/util-mpm-b3g.c | 1807 ---------
src/util-mpm-b3g.h | 130 -
src/util-mpm-hs.c | 130 +-
src/util-mpm-wumanber.c | 3219 ---------------
src/util-mpm-wumanber.h | 97 -
src/util-mpm.c | 529 +--
src/util-mpm.h | 132 +-
src/util-profiling-keywords.c | 2 +-
src/util-profiling-rulegroups.c | 407 ++
src/util-profiling.h | 15 +
suricata.yaml.in | 80 +-
84 files changed, 5082 insertions(+), 22016 deletions(-)
create mode 100644 src/detect-engine-profile.c
copy src/{detect-depth.h => detect-engine-profile.h} (74%)
delete mode 100644 src/util-mpm-ac-gfbs.c
delete mode 100644 src/util-mpm-ac-gfbs.h
delete mode 100644 src/util-mpm-b2g.c
delete mode 100644 src/util-mpm-b2g.h
delete mode 100644 src/util-mpm-b3g.c
delete mode 100644 src/util-mpm-b3g.h
delete mode 100644 src/util-mpm-wumanber.c
delete mode 100644 src/util-mpm-wumanber.h
create mode 100644 src/util-profiling-rulegroups.c
hooks/post-receive
--
OISF
More information about the Oisf-devel
mailing list