[Oisf-users] http.log log format

[Martin Holste]

Yes, but we also lose info if the log doesn't get parsed right by
whatever log solution is reading it in.  I understand the desire to
keep as much forensic information available as possible, but I think
that's better suited for packets.  For the log, normalization is
important because misparsed logs can mean missed log searches.

On Thu, Apr 5, 2012 at 3:23 AM, Victor Julien <victor at inliniac.net> wrote:
> On 04/05/2012 10:14 AM, Geert Alberghs wrote:
>> Hello,
>>
>> http logging has been enabled in our environment. The purpose is to
>> parse these logs for URL's up to and including the path. (so no query
>> and/or fragment part) The problem is that in http.log I encounter 2 log
>> formats:
>>
>>  1. TIMESTAMP HOSTNAME [**] COMPLETE URL [**]
>>  2. TIMESTAMP HOSTNAME [**] URL without SCHEME&HOSTNAME [**]
>>
>> In case 1 I only need COMPLETE URL and strip of query and/or fragment
>> In case 2 I need to concat "SCHEME", "HOSTNAME" and "URL without
>> SCHEME&HOSTNAME" and then strip of query and/or fragment.
>>
>> Is there any logic in why there are 2 different cases? Personally I
>> think log format 1 is preferable.
>
> The URL is expressed as it appears in the request. These are both valid:
>
> GET / HTTP/1.1
>
> GET http://somehost/ HTTP/1.1
>
> The host name is taken from the Host header.
>
> The 2nd URL format is used in case of proxy request usually, but is also
> valid for "normal" request by RFC.
>
> If we leave it out we miss some info, especially if the host part of the
> URL would not match the value of the Host header.
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users