[Oisf-devel] Some questions about module development

[Victor Julien]

On 02/20/2013 06:13 PM, Jörg Vehlow wrote:
> Actually I'm interested in every kind of communication, http was just
> one example.
> I did hook into the Applayer parser, managing the flags that control the
> behavior of the reassembler myself and buffering the data to be able to
> feed it to the applayer parser the way it was before I hooked into.
> 
> I've got one more questions: Which flags control the behavior of the
> reassembler that can be changed by app layer parsers: I identified
> FLOW_NO_APPLAYER_INSPECTION and STREAMTCP_FLAG_APPPROTO_DETECTION_COMPLETED.
> These two flags are controlled by my hook. I.e. unsetting
> FLOW_NO_APPLAYER_INSPECTION and setting
> STREAMTCP_FLAG_APPPROTO_DETECTION_COMPLETED before I return control  to
> the reassembler. By doing this I achieve getting the hole stream
> reassembled, independent of the app layer parsers state and only getting
> the data of the newly arrived packets.

This will probably mess up the existing parsers.

> Are there more flags I have to consider?

You should probably check:

/** Flag to avoid stream reassembly/app layer inspection for the stream */
#define STREAMTCP_STREAM_FLAG_NOREASSEMBLY      0x02
/** Flag to pause stream reassembly / app layer inspection for the stream.*/
#define STREAMTCP_STREAM_FLAG_PAUSE_REASSEMBLY  0x04
/** Stream has reached it's reassembly depth, all further packets are
ignored */
#define STREAMTCP_STREAM_FLAG_DEPTH_REACHED     0x08

from stream-tcp-private.h as well.

Cheers,
Victor

> 
> 
> Am 20.02.2013 12:00, schrieb Victor Julien:
>> On 02/01/2013 03:30 PM, Jörg Vehlow wrote:
>>> What I'm trying to do is to devide the stream into messages. A message
>>> is a continous part of the stream (stream means bidrectional stream
>>> here) in one direction.
>>> For an HTTP session with one request there will be two messages, the
>>> request and the response.
>>> If there is another request in the http session it will have four
>>> messages. (Request, response, request, response)
>>> The order in which the messages appear is important. Together with the
>>> payloadof a message, the time of the first frame and direction should be
>>> saved.
>>>
>>> Something like:
>>> struct Message {
>>>     char* data;
>>>     uint length;
>>>     int direction;
>>>
>>>     Message * next;
>>> }
>>>
>>> The benefit of this representation of the stream is that you can do
>>> statistical matches on the length of the individual messages or the time
>>> between two messsages. I hope to be able to detect some malware c'n'c
>>> traffic with this. Often you have limited message lengths or somewhat
>>> fixed intervals between messages.
>>>
>>> The rules to devide a bidirectional tcp stream into messages are as follows:
>>>  - begin of stream -> create new message
>>>  - change of transmission direction -> create new message
>>>  - timeout after last received packet (say if no packet receives within
>>> 2 seconds) -> create new message
>>>
>>>
>>> The only thing I need at the moment is the pakets of a tcp stream in the
>>> correct order. This essentially what the reassembler should to I think...
>> I think that if you're interested in HTTP only, the best way to achieve
>> this would be to hook into the app-layer-htp entry functions. You can
>> then use the parser's knowledge of when one request ends and the next
>> starts (in case of request pipelining).
>>
> 


-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------