[Oisf-devel] Some questions about module development

[Jörg Vehlow]

Actually I'm interested in every kind of communication, http was just
one example.
I did hook into the Applayer parser, managing the flags that control the
behavior of the reassembler myself and buffering the data to be able to
feed it to the applayer parser the way it was before I hooked into.

I've got one more questions: Which flags control the behavior of the
reassembler that can be changed by app layer parsers: I identified
FLOW_NO_APPLAYER_INSPECTION and STREAMTCP_FLAG_APPPROTO_DETECTION_COMPLETED.
These two flags are controlled by my hook. I.e. unsetting
FLOW_NO_APPLAYER_INSPECTION and setting
STREAMTCP_FLAG_APPPROTO_DETECTION_COMPLETED before I return control  to
the reassembler. By doing this I achieve getting the hole stream
reassembled, independent of the app layer parsers state and only getting
the data of the newly arrived packets.

Are there more flags I have to consider?


Am 20.02.2013 12:00, schrieb Victor Julien:
> On 02/01/2013 03:30 PM, Jörg Vehlow wrote:
>> What I'm trying to do is to devide the stream into messages. A message
>> is a continous part of the stream (stream means bidrectional stream
>> here) in one direction.
>> For an HTTP session with one request there will be two messages, the
>> request and the response.
>> If there is another request in the http session it will have four
>> messages. (Request, response, request, response)
>> The order in which the messages appear is important. Together with the
>> payloadof a message, the time of the first frame and direction should be
>> saved.
>>
>> Something like:
>> struct Message {
>>     char* data;
>>     uint length;
>>     int direction;
>>
>>     Message * next;
>> }
>>
>> The benefit of this representation of the stream is that you can do
>> statistical matches on the length of the individual messages or the time
>> between two messsages. I hope to be able to detect some malware c'n'c
>> traffic with this. Often you have limited message lengths or somewhat
>> fixed intervals between messages.
>>
>> The rules to devide a bidirectional tcp stream into messages are as follows:
>>  - begin of stream -> create new message
>>  - change of transmission direction -> create new message
>>  - timeout after last received packet (say if no packet receives within
>> 2 seconds) -> create new message
>>
>>
>> The only thing I need at the moment is the pakets of a tcp stream in the
>> correct order. This essentially what the reassembler should to I think...
> I think that if you're interested in HTTP only, the best way to achieve
> this would be to hook into the app-layer-htp entry functions. You can
> then use the parser's knowledge of when one request ends and the next
> starts (in case of request pipelining).
>