[Oisf-users] Suricata 1.4 http keywords in rule options, how does matching occur for http_header?

Sat Jan 26 15:22:26 UTC 2013

Thanks for the review Brian.

Have updated the patch to now use tx->parsed_uri->hostname.  Will
cover the unittests for the various hostname cases shortly.

Note: Since the hostname is normalized to lowercase, I'm invalidating
signatures which doesn't have a nocase attached to the http_header
content.  Same for pcre.

* This isn't an incremental patch.  Have amended to the old commit
instead.  You'll have to undo my previous patch, before applying this
one.

On Sat, Jan 26, 2013 at 8:58 AM, Will Metcalf <william.metcalf at gmail.com> wrote:
> endswith sure seems a lot cleaner :)...
>
> Regards,
>
> Will
>
> On Fri, Jan 25, 2013 at 9:22 PM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>> On Fri, Jan 25, 2013 at 8:53 PM, Will Metcalf <william.metcalf at gmail.com> wrote:
>>> While we are talking about having len and endswith would be really
>>> useful for at least http_uri, http_user_agent, and http_host_header.
>>> The first for performing exact matches i.e.
>>>
>>> content:"Mozilla"; http_user_agent; http_user_agent_len:7;
>>>
>>> to match
>>>
>>> User-Agent: Mozilla\r\n
>>>
>>
>> Sounds good.
>>
>>> or
>>>
>>> content:".exe"; http_uri; endswith;
>>>
>>> to match
>>>
>>> GET /blah/blat/foo.exe HTTP/1.1\r\n
>>>
>>
>> Can be done with isdataat?  content:"exe"; http_uri; isdataat:!0, relative;
>>
>>>
>>> etc... Want a feature request? :)
>>>
>>> Regards,
>>>
>>> Will
>>>
>>> On Fri, Jan 25, 2013 at 8:47 AM, William Metcalf
>>> <william.metcalf at gmail.com> wrote:
>>>> If the CRLF is normalized out from a rule writers perspective we probably continue to use http_header in most cases as generally you want to match on say google.com what you really want to match is .google.com\r\n and not .somethinggoogle.com.ru. Would it be possible to add a endswith modifier for this normalized buffer?
>>>>
>>
>> Yes we can add, but it probably doesn't make sense since we might
>> probably pick the hostname from the uri section as well?
>>
>> One can still do it using isdataat actually - content:"google.com";
>> http_header; isdataat:!0, relative;
>>
>>>> Regards,
>>>>
>>>> Will
>>>> On Jan 25, 2013, at 7:19 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>>
>>>>> Try this patch out(you can apply the patch using "git am -3 <patch>")
>>>>>
>>>>> It introduces a new keyword + pcre modifier that would inspect just
>>>>> the host header.
>>>>>
>>>>> The keyword being "http_host" and the pcre modifier being "W"
>>>>>
>>>>> You can now use it in a rule like this -
>>>>>
>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
>>>>> pcre:"/\.businessweek.com/W"; sid:1;)
>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
>>>>> content:".businessweek.com"; http_host; sid:2;)
>>>>>
>>>>> Let me know how it works with the above rules.
>>>>>
>>>>> On Fri, Jan 25, 2013 at 8:47 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>>>> On Thu, Jan 24, 2013 at 10:29 PM, Vincent Fang <vincent.y.fang at gmail.com> wrote:
>>>>>>> Here's the new results, I will run the tests again to see if it's consistent
>>>>>>> but using the wireshark filter
>>>>>>>
>>>>>>> http contains "businessweek.com"
>>>>>>>
>>>>>>> there were 75 matches
>>>>>>>
>>>>>>> and in the fast.log there were 138 total alerts from the two new rules you
>>>>>>> specified
>>>>>>> grep -c "http header" fast.log -> 69 lines
>>>>>>> grep -c "pcre version" fast.log -> 69 lines
>>>>>>>
>>>>>>> so they're both the same. Ran suricata in offline mode and the results were
>>>>>>> the same so that's good since they're consistent.
>>>>>>>
>>>>>>> Here's a copy of the two rules
>>>>>>>
>>>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
>>>>>>> pcre:"/\s.*?\.businessweek.com/H"; sid:1;)
>>>>>>>
>>>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
>>>>>>> content:".businessweek.com"; http_header; sid:2;)
>>>>>>>
>>>>>>> In the next few runs I also plan to change the protocol to http instead of
>>>>>>> ip, and I technically should get the same numbers yes?
>>>>>>
>>>>>> Yes, you should.
>>>>>>
>>>>>> Keep in mind that the above rules can also match on other headers
>>>>>> containing businessweek.com, for example the referer header.
>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 24, 2013 at 9:44 AM, Anoop Saldanha <anoopsaldanha at gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Sound good.  Will open a feature request for "http_host" keyword;
>>>>>>>>
>>>>>>>> On Thu, Jan 24, 2013 at 7:45 PM, Matt <matt at somedamn.com> wrote:
>>>>>>>>> I would find that useful, especially if it increases efficiency in the
>>>>>>>>> same
>>>>>>>>> way as http_user_agent.  Among other things, I use Suricata to match
>>>>>>>>> blacklists of known bad URLs, and all those rules include a content
>>>>>>>>> match
>>>>>>>>> for the HTTP Host.
>>>>>>>>>
>>>>>>>>> Matt
>>>>>>>>>
>>>>>>>>> On 1/24/2013 3:13 AM, Peter Manev wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 24, 2013 at 9:11 AM, Anoop Saldanha
>>>>>>>>> <anoopsaldanha at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 24, 2013 at 1:37 PM, Peter Manev <petermanev at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> However, any of the techniques mentioned above isn't a foolproof way
>>>>>>>>>>>> to match on the host header.  The right way would be to provide a
>>>>>>>>>>>> new
>>>>>>>>>>>> keyword called "http_host".
>>>>>>>>>>> Anoop or Vincent would you please put in feature request for that?
>>>>>>>>>>
>>>>>>>>>> We should probably consult users/rule-writers if such a keyword would
>>>>>>>>>> be useful to them?
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Anoop Saldanha
>>>>>>>>>
>>>>>>>>> sure
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Regards,
>>>>>>>>> Peter Manev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>>>>>>>> Site: http://suricata-ids.org | Support:
>>>>>>>>> http://suricata-ids.org/support/
>>>>>>>>> List:
>>>>>>>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>>>>>> OISF: http://www.openinfosecfoundation.org/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Anoop Saldanha
>>>>>>>> _______________________________________________
>>>>>>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>>>>>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>>>>>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>>>>> OISF: http://www.openinfosecfoundation.org/
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Anoop Saldanha
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Anoop Saldanha
>>>>> <0001-Add-support-for-a-new-keyword-to-inspect-http_host-h.patch>
>>>>> _______________________________________________
>>>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>> OISF: http://www.openinfosecfoundation.org/
>>
>>
>>
>> --
>> Anoop Saldanha

-- 
Anoop Saldanha
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Add-support-for-a-new-keyword-to-inspect-http_host-h.patch
Type: application/octet-stream
Size: 208813 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130126/009b453c/attachment-0002.obj>