[Oisf-devel] http buffers

[Will Metcalf]

PCRE should be avoided like the plague wherever possible... In my
testing even with JIT it's orders of magnitude slower than most
content matches.

Regards,

Will

On Tue, Feb 14, 2012 at 12:55 PM, Martin Holste <mcholste at gmail.com> wrote:
> Yeah, a user-agent buffer would probably be invoked by thousands of
> sigs.  I suppose the question is how much more efficient is searching
> 10-50 bytes versus 50-200.  Or, will 1/4 as much buffer to search
> translate to 4x improvement?
>
> On Tue, Feb 14, 2012 at 11:35 AM, Will Metcalf
> <william.metcalf at gmail.com> wrote:
>> +1 for being able to match on common named http headers.  In some
>> cases this means we don't have to invoke the PCRE performance
>> devil....
>>
>> Regards,
>>
>> Will
>>
>> On Tue, Feb 14, 2012 at 11:29 AM, Anoop Saldanha
>> <anoopsaldanha at gmail.com> wrote:
>>> On Tue, Feb 14, 2012 at 7:11 PM, Victor Julien <victor at inliniac.net> wrote:
>>>> On 02/14/2012 01:12 PM, Chris Wakelin wrote:
>>>>> On 14/02/12 11:52, Victor Julien wrote:
>>>>>> Noticed quite a few ET rules match completely in HTTP buffers (uri,
>>>>>> headers, etc) except for the request and/or response protocol.
>>>>>>
>>>>>> What about adding http_proto (http request proto) and http_stat_proto
>>>>>> (http response proto) in Suricata? Any use for that?
>>>>>>
>>>>>> They would contain "HTTP/1.1" or "HTTP/1.0".
>>>>>
>>>>> Sounds useful!
>>>>>
>>>>> What about other common headers such as "Host:" and "Referer:"? Would we
>>>>> get a performance boost?
>>>>
>>>> Preparing the buffer would require some cycles, we'd safe some on
>>>> inspection. My guess is a minor speedup, but hard to predict.
>>>>
>>>> Adding more buffers is certainly possible. User-agent would be a good
>>>> candidate too I think.
>>>>
>>>>>
>>>>>> The HTTP/0.9 case is interesting, as it commits this field. We could
>>>>>> forge it, so you can match on "HTTP/0.9". Thoughts?
>>>>>
>>>>> You mean "omits" rather than "commits" I think? Forging it makes sense,
>>>>> I guess.
>>>>
>>>> Ya omits indeed :)
>>>>
>>>>>
>>>>> One thing I wondered about the HTTP engine is would it be possible to
>>>>> have single rule check both request and response headers, or even
>>>>> request headers and response body, or will we need to use flowbits?
>>>>
>>>> Technically it's possible, and in some cases it may even already work.
>>>> We've been enforcing adding a direction on some keywords though.
>>>>
>>>> I think we never really made a decision on where to go with this. The
>>>> rule language and current rule writers think in one direction I think.
>>>> But bidirectional, especially for HTTP, is possible since we inspect the
>>>> full HTTP state.
>>>>
>>>
>>> We had a sorta discussion on this in one of our devel threads for
>>> response headers.  I like the idea of inspection of http state across
>>> both directions irrespective of sig direction.  More like the
>>> direction for keywords overrides sig direction.  This way we don't
>>> have to resort to flowbits to write such rules.
>>>
>>>>> e.g. I have a couple of rules using flowbits that look for an executable
>>>>> downloaded from "/" without a referrer.
>>>>
>>>> Ya I can see the use. Have you tried a sig like this? I wonder about
>>>> performance too.
>>>
>>> --
>>> Anoop Saldanha
>>> _______________________________________________
>>> Oisf-devel mailing list
>>> Oisf-devel at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>> _______________________________________________
>> Oisf-devel mailing list
>> Oisf-devel at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel