[Oisf-users] http.log log format

[Victor Julien]

On 04/05/2012 03:45 PM, Martin Holste wrote:
> Yes, but we also lose info if the log doesn't get parsed right by
> whatever log solution is reading it in.  I understand the desire to
> keep as much forensic information available as possible, but I think
> that's better suited for packets.  For the log, normalization is
> important because misparsed logs can mean missed log searches.

I see the point, but I don't want to loose the extra information it
gives. Maybe we can add that to the log in a different way?

Victor

> On Thu, Apr 5, 2012 at 3:23 AM, Victor Julien <victor at inliniac.net> wrote:
>> On 04/05/2012 10:14 AM, Geert Alberghs wrote:
>>> Hello,
>>>
>>> http logging has been enabled in our environment. The purpose is to
>>> parse these logs for URL's up to and including the path. (so no query
>>> and/or fragment part) The problem is that in http.log I encounter 2 log
>>> formats:
>>>
>>>  1. TIMESTAMP HOSTNAME [**] COMPLETE URL [**]
>>>  2. TIMESTAMP HOSTNAME [**] URL without SCHEME&HOSTNAME [**]
>>>
>>> In case 1 I only need COMPLETE URL and strip of query and/or fragment
>>> In case 2 I need to concat "SCHEME", "HOSTNAME" and "URL without
>>> SCHEME&HOSTNAME" and then strip of query and/or fragment.
>>>
>>> Is there any logic in why there are 2 different cases? Personally I
>>> think log format 1 is preferable.
>>
>> The URL is expressed as it appears in the request. These are both valid:
>>
>> GET / HTTP/1.1
>>
>> GET http://somehost/ HTTP/1.1
>>
>> The host name is taken from the Host header.
>>
>> The 2nd URL format is used in case of proxy request usually, but is also
>> valid for "normal" request by RFC.
>>
>> If we leave it out we miss some info, especially if the host part of the
>> URL would not match the value of the Host header.
>>
>> --
>> ---------------------------------------------
>> Victor Julien
>> http://www.inliniac.net/
>> PGP: http://www.inliniac.net/victorjulien.asc
>> ---------------------------------------------
>>
>> _______________________________________________
>> Oisf-users mailing list
>> Oisf-users at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> 


-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------