[Oisf-users] Suricata 1.4 http keywords in rule options, how does matching occur for http_header?

[Victor Julien]

On 01/26/2013 04:22 PM, Anoop Saldanha wrote:
> Thanks for the review Brian.
> 
> Have updated the patch to now use tx->parsed_uri->hostname.  Will
> cover the unittests for the various hostname cases shortly.
> 
> Note: Since the hostname is normalized to lowercase, I'm invalidating
> signatures which doesn't have a nocase attached to the http_header
> content.  Same for pcre.

I assume you mean http_host content?

Is there a way to get the hostname as it appeared in the request?

> * This isn't an incremental patch.  Have amended to the old commit
> instead.  You'll have to undo my previous patch, before applying this
> one.
> 
> On Sat, Jan 26, 2013 at 8:58 AM, Will Metcalf <william.metcalf at gmail.com> wrote:
>> endswith sure seems a lot cleaner :)...
>>
>> Regards,
>>
>> Will
>>
>> On Fri, Jan 25, 2013 at 9:22 PM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>> On Fri, Jan 25, 2013 at 8:53 PM, Will Metcalf <william.metcalf at gmail.com> wrote:
>>>> While we are talking about having len and endswith would be really
>>>> useful for at least http_uri, http_user_agent, and http_host_header.
>>>> The first for performing exact matches i.e.
>>>>
>>>> content:"Mozilla"; http_user_agent; http_user_agent_len:7;
>>>>
>>>> to match
>>>>
>>>> User-Agent: Mozilla\r\n
>>>>
>>>
>>> Sounds good.
>>>
>>>> or
>>>>
>>>> content:".exe"; http_uri; endswith;
>>>>
>>>> to match
>>>>
>>>> GET /blah/blat/foo.exe HTTP/1.1\r\n
>>>>
>>>
>>> Can be done with isdataat?  content:"exe"; http_uri; isdataat:!0, relative;
>>>
>>>>
>>>> etc... Want a feature request? :)
>>>>
>>>> Regards,
>>>>
>>>> Will
>>>>
>>>> On Fri, Jan 25, 2013 at 8:47 AM, William Metcalf
>>>> <william.metcalf at gmail.com> wrote:
>>>>> If the CRLF is normalized out from a rule writers perspective we probably continue to use http_header in most cases as generally you want to match on say google.com what you really want to match is .google.com\r\n and not .somethinggoogle.com.ru. Would it be possible to add a endswith modifier for this normalized buffer?
>>>>>
>>>
>>> Yes we can add, but it probably doesn't make sense since we might
>>> probably pick the hostname from the uri section as well?
>>>
>>> One can still do it using isdataat actually - content:"google.com";
>>> http_header; isdataat:!0, relative;
>>>
>>>>> Regards,
>>>>>
>>>>> Will
>>>>> On Jan 25, 2013, at 7:19 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>>>
>>>>>> Try this patch out(you can apply the patch using "git am -3 <patch>")
>>>>>>
>>>>>> It introduces a new keyword + pcre modifier that would inspect just
>>>>>> the host header.
>>>>>>
>>>>>> The keyword being "http_host" and the pcre modifier being "W"
>>>>>>
>>>>>> You can now use it in a rule like this -
>>>>>>
>>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
>>>>>> pcre:"/\.businessweek.com/W"; sid:1;)
>>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
>>>>>> content:".businessweek.com"; http_host; sid:2;)
>>>>>>
>>>>>> Let me know how it works with the above rules.
>>>>>>
>>>>>> On Fri, Jan 25, 2013 at 8:47 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>>>>> On Thu, Jan 24, 2013 at 10:29 PM, Vincent Fang <vincent.y.fang at gmail.com> wrote:
>>>>>>>> Here's the new results, I will run the tests again to see if it's consistent
>>>>>>>> but using the wireshark filter
>>>>>>>>
>>>>>>>> http contains "businessweek.com"
>>>>>>>>
>>>>>>>> there were 75 matches
>>>>>>>>
>>>>>>>> and in the fast.log there were 138 total alerts from the two new rules you
>>>>>>>> specified
>>>>>>>> grep -c "http header" fast.log -> 69 lines
>>>>>>>> grep -c "pcre version" fast.log -> 69 lines
>>>>>>>>
>>>>>>>> so they're both the same. Ran suricata in offline mode and the results were
>>>>>>>> the same so that's good since they're consistent.
>>>>>>>>
>>>>>>>> Here's a copy of the two rules
>>>>>>>>
>>>>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
>>>>>>>> pcre:"/\s.*?\.businessweek.com/H"; sid:1;)
>>>>>>>>
>>>>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
>>>>>>>> content:".businessweek.com"; http_header; sid:2;)
>>>>>>>>
>>>>>>>> In the next few runs I also plan to change the protocol to http instead of
>>>>>>>> ip, and I technically should get the same numbers yes?
>>>>>>>
>>>>>>> Yes, you should.
>>>>>>>
>>>>>>> Keep in mind that the above rules can also match on other headers
>>>>>>> containing businessweek.com, for example the referer header.
>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 24, 2013 at 9:44 AM, Anoop Saldanha <anoopsaldanha at gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Sound good.  Will open a feature request for "http_host" keyword;
>>>>>>>>>
>>>>>>>>> On Thu, Jan 24, 2013 at 7:45 PM, Matt <matt at somedamn.com> wrote:
>>>>>>>>>> I would find that useful, especially if it increases efficiency in the
>>>>>>>>>> same
>>>>>>>>>> way as http_user_agent.  Among other things, I use Suricata to match
>>>>>>>>>> blacklists of known bad URLs, and all those rules include a content
>>>>>>>>>> match
>>>>>>>>>> for the HTTP Host.
>>>>>>>>>>
>>>>>>>>>> Matt
>>>>>>>>>>
>>>>>>>>>> On 1/24/2013 3:13 AM, Peter Manev wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 24, 2013 at 9:11 AM, Anoop Saldanha
>>>>>>>>>> <anoopsaldanha at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 24, 2013 at 1:37 PM, Peter Manev <petermanev at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> However, any of the techniques mentioned above isn't a foolproof way
>>>>>>>>>>>>> to match on the host header.  The right way would be to provide a
>>>>>>>>>>>>> new
>>>>>>>>>>>>> keyword called "http_host".
>>>>>>>>>>>> Anoop or Vincent would you please put in feature request for that?
>>>>>>>>>>>
>>>>>>>>>>> We should probably consult users/rule-writers if such a keyword would
>>>>>>>>>>> be useful to them?
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Anoop Saldanha
>>>>>>>>>>
>>>>>>>>>> sure
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Regards,
>>>>>>>>>> Peter Manev


-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------