[Oisf-devel] Support for HTTP range requests

[Victor Julien]

On 08-10-15 21:02, Christian Kreibich wrote:
>> We'll need to update libhtp a bit to start supporting this. I do think
>> the 'reassembly' would better be done inside Suricata.
> 
> Mhmm yeah, this seems a bit tricky. There could be various policies you
> want to use to drive the reassembly. My intuition would be to have most
> of it happen in libhtp since the content-ranges are an HTTP-internal
> aspect that the library user may not care about at all.

The thing that makes this tricky is storing the cross flow data in a
thread safe and performant way. Libhtp currently just works on single
flows, which fits nicely in Suricata's flow structure and leaves all the
synchronisation issues to us.

For byte-ranges we'll need cross flow storage and lookups.

We could of course look at making the actual storage/lookup a set of
callbacks that Suricata would register. But since we're the only serious
(known & contributing) user of libhtp, I don't want to spend many extra
cycles on making the lib nicer just for the sake of it. Of course I'd
welcome contribs here (and elsewhere).

>> We do need to think on how to do the data storage. For simple cases we
>> may look at the IPPair data structure, but for sites with multiple hosts
>> serving a domain we'd need something else. Perhaps a new data store
>> searchable by hostname + url.
> 
> In case it helps, RFC 7233 has this to say:
> 
>   When a 206 response is generated, the server MUST generate the
>   following header fields, in addition to those required above, if the
>   field would have been sent in a 200 (OK) response to the same
>   request: Date, Cache-Control, ETag, Expires, Content-Location, and
>   Vary.
> 
> So there are several ways to track the entity overall.

That's useful, thanks!

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------