[Oisf-devel] [COMMIT] OISF branch, master, updated. suricata-2.1beta3-19-gcbe9342
OISF Git
noreply at openinfosecfoundation.org
Wed Feb 4 18:17:21 UTC 2015
This is an automated email from the git hooks/post-receive script. It was
generated because a ref change was pushed to the repository containing
the project "OISF".
The branch, master has been updated
via cbe934267efa0973b042853cd36a99c2b3c83941 (commit)
via c58b2b4b18375781d56384c5c2e123c6a2fa3e31 (commit)
via e58fd3cc6e29312e957f8b78b43a7115883b4eb8 (commit)
via fbe6dac1ae013d4d118cbe3da2fe9351f19dd9d4 (commit)
via 5251ea9ff5a8b419cb3f642606a036210c6e0bd1 (commit)
via 5587372ce15cf3ead98867e1daf3dd010529b4ad (commit)
via a0732d3db23538689af9432e012d26241996b99b (commit)
via 9327b08ab16c10807b70a2fb200d5681ed6aea39 (commit)
via 37b56dca5570313a0bee6f19d1955f8baf40b926 (commit)
via 2bba5eb7044504044b9a0e913e95fa4f542e49e4 (commit)
via 8c1bc7cfb6adddfbfcbad6ee087b5fd45383cd66 (commit)
via 6ca9c8eb32c5a44d52ae4b3d6ad0d61848025739 (commit)
via ff2fecf59099f3d1808d7d89ee2c459bdf59ab63 (commit)
via 97908bcd2d4157536ba03fd724ed0ee6351ab73f (commit)
via e1d134b0275a1fef073cee55b75b43107164a851 (commit)
via 29d2483efb47108efb26728483e94ea502ac7f8c (commit)
via e4cb8715de69b12fbb1605a387b79ff221bf9928 (commit)
via ed791a562eddbcd14b31063c6a1519c2453c3a76 (commit)
via e494336453c16c00f1026da63685ffdc2b9f567f (commit)
from a5641bc7c225d447d21a65927b4077f095d7dca2 (commit)
Those revisions listed above that are new to this repository have
not appeared on any other notification email; so we list those
revisions in full, below.
- Log -----------------------------------------------------------------
commit cbe934267efa0973b042853cd36a99c2b3c83941
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 23 10:22:55 2015 +0100
file: register filedata log before file log
This way the file log can log the 'stored' info that the filedata
log sets.
commit c58b2b4b18375781d56384c5c2e123c6a2fa3e31
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 23 10:22:01 2015 +0100
file: improve file pruning
Check if file has been logged/stored before considering it 'done'.
commit e58fd3cc6e29312e957f8b78b43a7115883b4eb8
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 23 10:20:37 2015 +0100
runmodes: add funcs to check if file loggers enabled
Add functions to check if file/filedata loggers are enabled.
commit fbe6dac1ae013d4d118cbe3da2fe9351f19dd9d4
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 22 19:24:35 2015 +0100
file: optimize file pruning
FilePrune would clear the files, but not free them and remove them
from the list. This lead to ever growing lists in some cases.
Especially in HTTP sessions with many transactions, this could slow
us down.
commit 5251ea9ff5a8b419cb3f642606a036210c6e0bd1
Author: Victor Julien <victor at inliniac.net>
Date: Sat Jan 24 14:10:11 2015 +0100
flow: lockless flow manager checks
Until this point, the flow manager would check for timed out flows
by walking the flow hash, locking first the hash row and then each
individual flow to get it's state and timestamp. To not be too
intrusive trylocks were used so that a busy flow wouldn't cause the
flow manager to wait for a long time while holding the hash row lock.
Building on the changes in handling of the flow state and lastts
fields, this patch changes the flow managers behavior.
It can now get a flows state atomically and the lastts can be safely
read while holding just the flow hash row lock. This allows the flow
manager to do the basic time out check much more cheaply:
1. it doesn't have to wait for getting a lock
2. it doesn't interupt the packet path
As a consequence the trylock is now also gone. A flow that returns
'true' on timeout is pretty much certainly not going to be busy so
we can safely lock it unconditionally. This also means the flow
manager now walks the entire row unconditionally and is guaranteed
to inspect each flow in the row.
To make sure the functions called before the flow lock don't
accidentally change the flow (which would require a lock) the args
to these flows are changed to const pointers.
commit 5587372ce15cf3ead98867e1daf3dd010529b4ad
Author: Victor Julien <victor at inliniac.net>
Date: Sat Jan 24 13:46:25 2015 +0100
flow: modify lastts update logic
In the lastts timeval struct field in the flow the timestamp of the
last packet to update is recorded. This allows for tracking the timeout
of the flow. So far, this value was updated under the flow lock and also
read under the flow lock.
This patch moves the updating of this field to the FlowGetFlowFromHash
function, where it updated at the point where both the Flow and the
Flow Hash Row lock are held. This guarantees that the field is only
updated when both locks are held.
This makes reading the field safe when either lock is held, which is the
purpose of this patch.
The flow manager, while holding the flow hash row lock, can now safely
read the lastts value. This allows it to do the flow timeout check
without actually locking the flow.
commit a0732d3db23538689af9432e012d26241996b99b
Author: Victor Julien <victor at inliniac.net>
Date: Sat Jan 24 13:18:51 2015 +0100
flow: change flow state logic
A flow has 3 states: NEW, ESTABLISHED and CLOSED.
For all protocols except TCP, a flow is in state NEW as long as just one
side of the conversation has been seen. When both sides have been
observed the state is moved to ESTABLISHED.
TCP has a different logic, controlled by the stream engine. Here the TCP
state is leading.
Until now, when parts of the engine needed to know the flow state, it
would invoke a per protocol callback 'GetProtoState'. For TCP this would
return the state based on the TcpSession.
This patch changes this logic. It introduces an atomic variable in the
flow 'flow_state'. It defaults to NEW and is set to ESTABLISHED for non-
TCP protocols when we've seen both sides of the conversation.
For TCP, the state is updated from the TCP engine directly.
The goal is to allow for access to the state without holding the Flow's
main mutex lock. This will later allow the Flow Manager(s) to evaluate
the Flow w/o interupting it.
commit 9327b08ab16c10807b70a2fb200d5681ed6aea39
Author: Victor Julien <victor at inliniac.net>
Date: Thu Jan 22 13:43:31 2015 +0100
tcp: add stream.reassembly.zero-copy-size option
The option sets in bytes the value at which segment data is passed to
the app layer API directly. Data sizes equal to and higher than the
value set are passed on directly.
Default is 128.
commit 37b56dca5570313a0bee6f19d1955f8baf40b926
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 16 12:12:38 2015 +0100
tcp: add debug stats about reassembly fast paths
Only shown if --enable-debug is passed to configure.
commit 2bba5eb7044504044b9a0e913e95fa4f542e49e4
Author: Victor Julien <victor at inliniac.net>
Date: Fri Jan 16 11:53:29 2015 +0100
tcp: zero copy fast path in app-layer reassembly
Create 2 'fast paths' for app layer reassembly. Both are about reducing
copying. In the cases described below, we pass the segment's data
directly to the app layer API, instead of first copying it into a buffer
than we then pass. This safes a copy.
The first is for the case when we have just one single segment that was
just ack'd. As we know that we won't use any other segment this round,
we can just use the segment data.
The second case is more aggressive. When the segment meets a certain
size limit (currently hardcoded at 128 bytes), we pass it to the
app-layer API directly. Thus invoking the app-layer somewhat more often
to safe some copies.
commit 8c1bc7cfb6adddfbfcbad6ee087b5fd45383cd66
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 19:08:14 2014 +0100
stream: move raw stream gap handling into util func
commit 6ca9c8eb32c5a44d52ae4b3d6ad0d61848025739
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 19:01:03 2014 +0100
stream: move raw reassembly into util func
commit ff2fecf59099f3d1808d7d89ee2c459bdf59ab63
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 15:39:49 2014 +0100
stream: remove StreamTcpReassembleInlineAppLayer
Function is now unused.
commit 97908bcd2d4157536ba03fd724ed0ee6351ab73f
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 15:27:58 2014 +0100
stream: unify inline and non-inline applayer assembly
Unifiy inline and non-inline app layer stream reassembly to aid
maintainability of the code.
commit e1d134b0275a1fef073cee55b75b43107164a851
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 14:36:05 2014 +0100
stream: remove STREAM_SET_FLAGS
Use the unified StreamGetAppLayerFlags instead.
commit 29d2483efb47108efb26728483e94ea502ac7f8c
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 14:26:35 2014 +0100
stream: update inline tests
Make sure inline tests set the stream_inline flag.
commit e4cb8715de69b12fbb1605a387b79ff221bf9928
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 14:25:34 2014 +0100
stream: replace STREAM_SET_INLINE_FLAGS macro
Replace it by a generic function StreamGetAppLayerFlags, that can
be used both by inline and non-inline.
commit ed791a562eddbcd14b31063c6a1519c2453c3a76
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 13:48:00 2014 +0100
stream: track data sent to app-layer
commit e494336453c16c00f1026da63685ffdc2b9f567f
Author: Victor Julien <victor at inliniac.net>
Date: Tue Dec 9 12:56:55 2014 +0100
stream: move reassembly loop into util funcs
Move IDS per segment reassembly and gap handling into utility functions.
-----------------------------------------------------------------------
Summary of changes:
src/app-layer.c | 8 +-
src/flow-hash.c | 14 +-
src/flow-manager.c | 31 +-
src/flow-private.h | 19 -
src/flow-util.h | 3 +
src/flow.c | 28 +-
src/flow.h | 15 +-
src/runmodes.c | 46 +-
src/runmodes.h | 5 +
src/stream-tcp-reassemble.c | 1841 +++++++++++++++++++------------------------
src/stream-tcp-reassemble.h | 9 +-
src/stream-tcp-util.c | 7 +
src/stream-tcp-util.h | 2 +
src/stream-tcp.c | 56 +-
src/stream-tcp.h | 3 +
src/util-file.c | 35 +-
suricata.yaml.in | 6 +
17 files changed, 982 insertions(+), 1146 deletions(-)
hooks/post-receive
--
OISF
More information about the Oisf-devel
mailing list