[Oisf-devel] [COMMIT] OISF branch, master, updated. suricata-2.1beta4-257-g834c366
OISF Git
noreply at openinfosecfoundation.org
Thu Jul 23 21:43:12 UTC 2015
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "OISF".
The branch, master has been updated
via 834c36659e881aa9b140683151309e4229600b09 (commit)
via 1235c578b25e2e4ca0eb442dfbe962a7a2f22551 (commit)
via 979bd3527799e657ce6084297e7a78999bc7f9e1 (commit)
via cfeaf42caba71f8f34a506758866146f95ff9f5b (commit)
via f871c0e1b80e0712bf5dd43cc1799561c5ad42c2 (commit)
via e44fd97d720801d2d1796bad3ad392831a76c4ed (commit)
via 99c0a7ad72fbe9ca3ffe7905eebafd8bb418bc5a (commit)
via b7b27684c2e093c0f67adbfa9af0400f7104ac3b (commit)
via eb09118d64003b06edaf455bd2bee0cab1fa467b (commit)
via 82aa419431509bb9c54c22e989b6402d43a3b357 (commit)
via e19c41a80726f27672f9ba889ab9b0444cc09399 (commit)
via 722c56dbf33035576e205ca739078be2f2337171 (commit)
via 4f77e8967bb0baab3b8791b8d3a977cf20c5dac1 (commit)
via a6e3cec9e560f0fe8c9e297e557d9f2f9a939192 (commit)
via 5fff2507497cc0d442a9e9670a145ea42a240e78 (commit)
via 4d0975eeac28956720df86c1e836e5fa474ee135 (commit)
via d8181802d375e50dff5f15a6532a20861c5b0b23 (commit)
via 8673801ea3b7b5bba0c40fb916f33f53898b6069 (commit)
via f4c9915066402852e647346ed8c9d797ba6781f3 (commit)
via 642c267dc4aa8af62fbdc95ddb61c52055093d8e (commit)
via 4e0683c3f520ff73cef7d33779f269b54aaa8e0f (commit)
via 646eb4c2a8387c642f979c78032e064ed88bc1eb (commit)
via a20e43f97c0d91e642c1c02ca18293d15361000f (commit)
via a21e88d95534df8dc70b0500b02fd7c45337625e (commit)
via 216638c3428819dd7b4254027d5cd48ea8af0191 (commit)
via 93f856a1b309f84796f182ae95483a383c3edacc (commit)
via b6f290fac74c51384fb6a89feb94f25bd740d5a8 (commit)
via c72b7f83b87982b7b03600b08e8fd6d75da9bfa0 (commit)
via 1127ad66b46a8322438a99b602974faaf60ff770 (commit)
via 7c581c0ffcaa66b6ebd62b8f3918ad1a9d73072e (commit)
via 6d92e8d220d4e3924c92e202e6ed608e23ca5db9 (commit)
via 1893c5edb1f239985ecaad8afafac08647034485 (commit)
via 0ff6d3dcfd9c61e320fd83d3cdbec371de286722 (commit)
via bc7e9be5c64b191521bdbe9f198392a59a5b84d5 (commit)
via bea66156b9b3255032a29dd1f3f5e73bdf506390 (commit)
via 127d8183d78c9e9f6d75dcdc139c1eb79bca1e4c (commit)
via 98d265f40b63538ca2c15996d0fda1f1e6cb8644 (commit)
via b653479815175aa12377b4293f37b5476a437ff7 (commit)
via 3f12bdd4f8d78d40b7bf21974fdcdf9eb2b8ebeb (commit)
via def2b58725e6876abecceccecb096ba005eb34bc (commit)
via 147a6d2bfd899d16a68d01874bbc8dda71ff791f (commit)
from 3aa58f25ad51a68b57946f06a2423a26e41400c8 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit 834c36659e881aa9b140683151309e4229600b09
Author: Antti Tönkyrä <antti.tonkyra at f-solutions.fi>
Date: Tue May 19 20:07:21 2015 +0000
Allow colon in SSH version, at least some trojaned PuTTY clients have version like Putty-Local: Timestamp HH:MM:SS
commit 1235c578b25e2e4ca0eb442dfbe962a7a2f22551
Author: Victor Julien <victor at inliniac.net>
Date: Tue Jul 21 17:27:26 2015 +0200
xff: support ports and more ipv6 notations
It's not uncommon to see an header like:
X-Forwarded-For: 1.2.3.4:56789
This patch recognizes this case and ignores the port. It also supports
this for IPv6 if the address has the following notation:
X-Forwarded-For: [12::34]:1234
This patch also adds unittests.
commit 979bd3527799e657ce6084297e7a78999bc7f9e1
Author: Victor Julien <victor at inliniac.net>
Date: Mon Jul 20 19:55:38 2015 +0200
detect loader: move to own file
commit cfeaf42caba71f8f34a506758866146f95ff9f5b
Author: Victor Julien <victor at inliniac.net>
Date: Mon Jul 20 18:08:37 2015 +0200
detect-loaders: configurable amount of loaders
commit f871c0e1b80e0712bf5dd43cc1799561c5ad42c2
Author: Victor Julien <victor at inliniac.net>
Date: Wed Apr 22 12:41:57 2015 +0200
debug: packet pool init/destroy validation
Validate packet pool handling:
- pools are initialized before use
- pools are not used after destroy
- pools are not double initialized/destroyed
commit e44fd97d720801d2d1796bad3ad392831a76c4ed
Author: Victor Julien <victor at inliniac.net>
Date: Wed Jul 8 15:57:26 2015 +0200
unittests: use a global packetpool
commit 99c0a7ad72fbe9ca3ffe7905eebafd8bb418bc5a
Author: Victor Julien <victor at inliniac.net>
Date: Tue May 19 15:14:49 2015 +0200
multi-detect: improve memory handling of setup code
commit b7b27684c2e093c0f67adbfa9af0400f7104ac3b
Author: Victor Julien <victor at inliniac.net>
Date: Wed May 13 15:35:47 2015 +0200
multi-detect: detect loader for unix socket
Move the tenant load and reload commands to be executed by the detect
loader thread(s).
Limitation: no yaml parsing in parallel. The Conf API is currently not
thread safe, so don't load the tenant config (yaml) in parallel.
commit eb09118d64003b06edaf455bd2bee0cab1fa467b
Author: Victor Julien <victor at inliniac.net>
Date: Tue May 12 11:35:47 2015 +0200
detect: create loader threads
To speed up startup with many tenants, tenant loading will be parallelized.
As no tempary threads should be used for these memory allocation heavy
tasks, this patch adds new type of 'command' thread that can be used to
load and reload tenants.
This patch hardcodes the number of loaders to 4. Future work will make it
dynamic.
The loader thread essentially sleeps constantly. When a tasks is sent to
it, it will wake up and execute it.
commit 82aa419431509bb9c54c22e989b6402d43a3b357
Author: Victor Julien <victor at inliniac.net>
Date: Mon Apr 13 10:33:11 2015 +0200
multi-detect: set tenant id on pseudo packets
Store the tenant id in the flow and use the stored id when setting
up pesudo packets.
For tunnel and defrag packets, get tenant from parent. This will only
pass tenant_id's set at capture time.
For defrag packets, the tenant selector based on vlan id will still
work as the vlan id(s) are stored in the defrag tracker before being
passed on.
commit e19c41a80726f27672f9ba889ab9b0444cc09399
Author: Victor Julien <victor at inliniac.net>
Date: Thu Apr 9 18:20:28 2015 +0200
multi-detect: hash lookup for tenants
Use hash for storing and looking up det_ctxs.
commit 722c56dbf33035576e205ca739078be2f2337171
Author: Victor Julien <victor at inliniac.net>
Date: Fri Apr 10 09:59:33 2015 +0200
detect: clean up thread free code
Introduce DetectEngineThreadCtxFree that doesn't need a 'ThreadVars'
pointer.
commit 4f77e8967bb0baab3b8791b8d3a977cf20c5dac1
Author: Victor Julien <victor at inliniac.net>
Date: Wed Apr 8 15:59:46 2015 +0200
multi-detect: make threshold prefix aware
Make threshold loading prefix aware, so it can be part of tenant
configuration.
If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
Note: currently per host thresholds are tracked globally and NOT per
tenant.
commit a6e3cec9e560f0fe8c9e297e557d9f2f9a939192
Author: Victor Julien <victor at inliniac.net>
Date: Wed Apr 8 15:31:52 2015 +0200
multi-detect: make reference prefix aware
Make reference loading prefix aware, so it can be part of tenant
configuration.
If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
commit 5fff2507497cc0d442a9e9670a145ea42a240e78
Author: Victor Julien <victor at inliniac.net>
Date: Wed Apr 8 14:55:16 2015 +0200
multi-detect: make classification prefix aware
Make classification loading prefix aware, so it can be part of tenant
configuration.
If the setting is missing from the tenant, the global setting is tried
and if that too is missing, the global default is used.
commit 4d0975eeac28956720df86c1e836e5fa474ee135
Author: Victor Julien <victor at inliniac.net>
Date: Sat Mar 14 20:27:30 2015 +0100
multi-detect: implement reload tenant in suricatasc
commit d8181802d375e50dff5f15a6532a20861c5b0b23
Author: Victor Julien <victor at inliniac.net>
Date: Sat Mar 14 20:23:13 2015 +0100
multi-detect: add reload-tenant command
Allow for a tenant to be reloaded. The command is the same as the
register-tenant command, so with a yaml and tenant-id as argument.
However this replaces an existing tenant.
commit 8673801ea3b7b5bba0c40fb916f33f53898b6069
Author: Victor Julien <victor at inliniac.net>
Date: Sun Mar 1 10:34:53 2015 +0100
multi-detect: add tenant id to alert json output
Add a integer field "tenant_id" to the JSON alert output.
commit f4c9915066402852e647346ed8c9d797ba6781f3
Author: Victor Julien <victor at inliniac.net>
Date: Sun Mar 1 10:09:21 2015 +0100
multi-detect: store tenant id in packet
Store tenant id in the packet so that the output API's can log it.
commit 642c267dc4aa8af62fbdc95ddb61c52055093d8e
Author: Victor Julien <victor at inliniac.net>
Date: Fri Feb 20 10:40:32 2015 +0100
multi-detect: refuse to add duplicate tenant
Generate error if tentant to be added is already loaded.
commit 4e0683c3f520ff73cef7d33779f269b54aaa8e0f
Author: Victor Julien <victor at inliniac.net>
Date: Fri Feb 20 09:33:53 2015 +0100
multi-detect: cleanup, reuse tenant loading code
Reuse tenant loading from YAML code for Unix Socket.
commit 646eb4c2a8387c642f979c78032e064ed88bc1eb
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 19 18:45:10 2015 +0100
multi-detect: load tenants from yaml file
Load tenants and mappings from the suricata.yaml when available.
commit a20e43f97c0d91e642c1c02ca18293d15361000f
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 12 15:30:20 2015 +0100
suricatasc: add unregister-tenant-handler
commit a21e88d95534df8dc70b0500b02fd7c45337625e
Author: Victor Julien <victor at inliniac.net>
Date: Fri Feb 6 15:08:44 2015 +0100
suricatasc: add register-tenant-handler command
Arguments:
- tenant id (int)
- name of handler (string)
- traffic id related to handler (int, optional)
Examples:
- register-tenant-handler 1 vlan 1111
- register-tenant-handler 8 pcap
commit 216638c3428819dd7b4254027d5cd48ea8af0191
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 12 15:56:38 2015 +0100
multi-detect: implement unregister-tenant-handler
Remove a tenant handler from the list and apply it.
commit 93f856a1b309f84796f182ae95483a383c3edacc
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 12 14:13:46 2015 +0100
detect: don't error out on no de_ctx
This can happen on a multi-detect setup with no registered
engines yet.
commit b6f290fac74c51384fb6a89feb94f25bd740d5a8
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 12 12:39:27 2015 +0100
multi-detect: set selector from yaml
Yaml setting is: multi-detect.selector
Implement 'vlan' and 'direct'.
commit c72b7f83b87982b7b03600b08e8fd6d75da9bfa0
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 12 12:08:33 2015 +0100
multi-detect: error on start if no selector registered
Force user to select the method at startup.
commit 1127ad66b46a8322438a99b602974faaf60ff770
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 12 14:14:35 2015 +0100
multi-detect: register counters on 'master' det_ctx
Otherwise counters are only registered after the stats api is
already fixed.
commit 7c581c0ffcaa66b6ebd62b8f3918ad1a9d73072e
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 12 11:04:35 2015 +0100
multi-detect: allow start up with 0 tenants
commit 6d92e8d220d4e3924c92e202e6ed608e23ca5db9
Author: Victor Julien <victor at inliniac.net>
Date: Fri Feb 6 16:01:09 2015 +0100
unix-socket: implement register-tenant-handler
Register tenant handlers/selectors based on what the unix command
"register-tenant-handler" tells.
Check traffic id before adding it. No duplicated registrations for
a traffic id are allowed.
commit 1893c5edb1f239985ecaad8afafac08647034485
Author: Victor Julien <victor at inliniac.net>
Date: Thu Feb 5 18:58:26 2015 +0100
multi-detect: initial selectors for tenants
The Detection Thread has the TenantGetId pointer which allows it
to select a tenant id based on the packet.
commit 0ff6d3dcfd9c61e320fd83d3cdbec371de286722
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 30 15:04:38 2015 +0100
detect: select detect engine at Detect entry
Limited to Pcap only currently.
commit bc7e9be5c64b191521bdbe9f198392a59a5b84d5
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 30 14:27:05 2015 +0100
unix-socket: allow tenant id with pcap-file
Register the tenant id that the pcap-file optionally got.
commit bea66156b9b3255032a29dd1f3f5e73bdf506390
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 30 13:20:12 2015 +0100
pcap-file: set tenant-id if available
Set the id to each packet's 'pcap_v' structure.
commit 127d8183d78c9e9f6d75dcdc139c1eb79bca1e4c
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 30 13:03:13 2015 +0100
suricatasc: allow for tenant id in pcap-file
Allow for an optional 'tenant id' argument to pcap-file. This will
allow us to force the pcap to be inspected by this tenant.
If ommited it's 0, which means it's disabled.
commit 98d265f40b63538ca2c15996d0fda1f1e6cb8644
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 29 16:40:23 2015 +0100
detect: use multi tenant thread init if MT enabled
commit b653479815175aa12377b4293f37b5476a437ff7
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 29 14:39:01 2015 +0100
detect: make multi tenancy a global switch
At start up we will set this flag based on "multi-detect.enabled".
commit 3f12bdd4f8d78d40b7bf21974fdcdf9eb2b8ebeb
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 29 11:33:11 2015 +0100
tenants: apply added/removed tenant
Apply to the engine.
commit def2b58725e6876abecceccecb096ba005eb34bc
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 29 11:20:09 2015 +0100
detect: initial MT lookup logic
In the DetectEngineThreadCtx, store another DetectEngineThreadCtx per
tenant.
Currently it's just a simple array indexed by the tenant id.
commit 147a6d2bfd899d16a68d01874bbc8dda71ff791f
Author: Victor Julien <victor at inliniac.net>
Date: Mon Jan 12 19:00:16 2015 +0100
multi-detect: (un)register-tenant unix socket commands
Make available to live mode and unix socket mode.
register-tenant:
Loads a new YAML, does basic validation.
Loads a new detection engine
Loads rules
Add new de_ctx to master store and stores tenant id in the de_ctx so
we can look it up by tenant id later.
unregister-tenant:
Gets the de_ctx, moves it to the freelist
Removes config
Introduce DetectEngineGetByTenantId, which gets a reference to the
detect engine by tenant id.
-----------------------------------------------------------------------
Summary of changes:
scripts/suricatasc/src/suricatasc.py | 76 +++-
src/Makefile.am | 1 +
src/alert-unified2-alert.c | 1 -
src/app-layer-htp-xff.c | 182 ++++++++-
src/app-layer-htp-xff.h | 2 +
src/app-layer-htp.c | 4 +-
src/decode.c | 2 +
src/decode.h | 4 +
src/detect-engine-loader.c | 300 ++++++++++++++
src/detect-engine-loader.h | 57 +++
src/detect-engine.c | 753 ++++++++++++++++++++++++++++++++++-
src/detect-engine.h | 11 +
src/detect-ssh-software-version.c | 2 +-
src/detect.c | 54 ++-
src/detect.h | 45 +++
src/flow-manager.c | 4 -
src/flow-timeout.c | 1 +
src/flow-util.h | 2 +
src/flow.h | 4 +
src/output-json-alert.c | 10 +-
src/output-json-alert.h | 2 +-
src/output-json-drop.c | 4 +-
src/runmode-unittests.c | 6 +
src/runmode-unix-socket.c | 404 ++++++++++++++++++-
src/runmode-unix-socket.h | 8 +
src/source-pcap-file.c | 13 +
src/source-pcap.h | 1 +
src/stream-tcp.c | 2 +
src/suricata.c | 1 +
src/tm-modules.c | 1 +
src/tm-threads-common.h | 1 +
src/tmqh-packetpool.c | 36 +-
src/tmqh-packetpool.h | 12 +-
src/unix-manager.c | 6 +
src/util-classification-config.c | 26 +-
src/util-error.c | 2 +
src/util-error.h | 2 +
src/util-reference-config.c | 25 +-
src/util-threshold-config.c | 24 +-
39 files changed, 2030 insertions(+), 61 deletions(-)
create mode 100644 src/detect-engine-loader.c
create mode 100644 src/detect-engine-loader.h
hooks/post-receive
--
OISF
More information about the Oisf-devel
mailing list