[Oisf-users] Suricata 1.4 http keywords in rule options, how does matching occur for http_header?

[Anoop Saldanha]

On Mon, Jan 28, 2013 at 4:00 PM, Victor Julien <lists at inliniac.net> wrote:
> On 01/26/2013 04:22 PM, Anoop Saldanha wrote:
>> Thanks for the review Brian.
>>
>> Have updated the patch to now use tx->parsed_uri->hostname.  Will
>> cover the unittests for the various hostname cases shortly.
>>
>> Note: Since the hostname is normalized to lowercase, I'm invalidating
>> signatures which doesn't have a nocase attached to the http_header
>> content.  Same for pcre.
>
> I assume you mean http_host content?
>
> Is there a way to get the hostname as it appeared in the request?
>

Yes, we can.  Shall we introduce it as a new keyword?  http_raw_host; ?

>> * This isn't an incremental patch.  Have amended to the old commit
>> instead.  You'll have to undo my previous patch, before applying this
>> one.
>>
>> On Sat, Jan 26, 2013 at 8:58 AM, Will Metcalf <william.metcalf at gmail.com> wrote:
>>> endswith sure seems a lot cleaner :)...
>>>
>>> Regards,
>>>
>>> Will
>>>
>>> On Fri, Jan 25, 2013 at 9:22 PM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>> On Fri, Jan 25, 2013 at 8:53 PM, Will Metcalf <william.metcalf at gmail.com> wrote:
>>>>> While we are talking about having len and endswith would be really
>>>>> useful for at least http_uri, http_user_agent, and http_host_header.
>>>>> The first for performing exact matches i.e.
>>>>>
>>>>> content:"Mozilla"; http_user_agent; http_user_agent_len:7;
>>>>>
>>>>> to match
>>>>>
>>>>> User-Agent: Mozilla\r\n
>>>>>
>>>>
>>>> Sounds good.
>>>>
>>>>> or
>>>>>
>>>>> content:".exe"; http_uri; endswith;
>>>>>
>>>>> to match
>>>>>
>>>>> GET /blah/blat/foo.exe HTTP/1.1\r\n
>>>>>
>>>>
>>>> Can be done with isdataat?  content:"exe"; http_uri; isdataat:!0, relative;
>>>>
>>>>>
>>>>> etc... Want a feature request? :)
>>>>>
>>>>> Regards,
>>>>>
>>>>> Will
>>>>>
>>>>> On Fri, Jan 25, 2013 at 8:47 AM, William Metcalf
>>>>> <william.metcalf at gmail.com> wrote:
>>>>>> If the CRLF is normalized out from a rule writers perspective we probably continue to use http_header in most cases as generally you want to match on say google.com what you really want to match is .google.com\r\n and not .somethinggoogle.com.ru. Would it be possible to add a endswith modifier for this normalized buffer?
>>>>>>
>>>>
>>>> Yes we can add, but it probably doesn't make sense since we might
>>>> probably pick the hostname from the uri section as well?
>>>>
>>>> One can still do it using isdataat actually - content:"google.com";
>>>> http_header; isdataat:!0, relative;
>>>>
>>>>>> Regards,
>>>>>>
>>>>>> Will
>>>>>> On Jan 25, 2013, at 7:19 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>>>>
>>>>>>> Try this patch out(you can apply the patch using "git am -3 <patch>")
>>>>>>>
>>>>>>> It introduces a new keyword + pcre modifier that would inspect just
>>>>>>> the host header.
>>>>>>>
>>>>>>> The keyword being "http_host" and the pcre modifier being "W"
>>>>>>>
>>>>>>> You can now use it in a rule like this -
>>>>>>>
>>>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
>>>>>>> pcre:"/\.businessweek.com/W"; sid:1;)
>>>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
>>>>>>> content:".businessweek.com"; http_host; sid:2;)
>>>>>>>
>>>>>>> Let me know how it works with the above rules.
>>>>>>>
>>>>>>> On Fri, Jan 25, 2013 at 8:47 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>>>>>> On Thu, Jan 24, 2013 at 10:29 PM, Vincent Fang <vincent.y.fang at gmail.com> wrote:
>>>>>>>>> Here's the new results, I will run the tests again to see if it's consistent
>>>>>>>>> but using the wireshark filter
>>>>>>>>>
>>>>>>>>> http contains "businessweek.com"
>>>>>>>>>
>>>>>>>>> there were 75 matches
>>>>>>>>>
>>>>>>>>> and in the fast.log there were 138 total alerts from the two new rules you
>>>>>>>>> specified
>>>>>>>>> grep -c "http header" fast.log -> 69 lines
>>>>>>>>> grep -c "pcre version" fast.log -> 69 lines
>>>>>>>>>
>>>>>>>>> so they're both the same. Ran suricata in offline mode and the results were
>>>>>>>>> the same so that's good since they're consistent.
>>>>>>>>>
>>>>>>>>> Here's a copy of the two rules
>>>>>>>>>
>>>>>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
>>>>>>>>> pcre:"/\s.*?\.businessweek.com/H"; sid:1;)
>>>>>>>>>
>>>>>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
>>>>>>>>> content:".businessweek.com"; http_header; sid:2;)
>>>>>>>>>
>>>>>>>>> In the next few runs I also plan to change the protocol to http instead of
>>>>>>>>> ip, and I technically should get the same numbers yes?
>>>>>>>>
>>>>>>>> Yes, you should.
>>>>>>>>
>>>>>>>> Keep in mind that the above rules can also match on other headers
>>>>>>>> containing businessweek.com, for example the referer header.
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Jan 24, 2013 at 9:44 AM, Anoop Saldanha <anoopsaldanha at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Sound good.  Will open a feature request for "http_host" keyword;
>>>>>>>>>>
>>>>>>>>>> On Thu, Jan 24, 2013 at 7:45 PM, Matt <matt at somedamn.com> wrote:
>>>>>>>>>>> I would find that useful, especially if it increases efficiency in the
>>>>>>>>>>> same
>>>>>>>>>>> way as http_user_agent.  Among other things, I use Suricata to match
>>>>>>>>>>> blacklists of known bad URLs, and all those rules include a content
>>>>>>>>>>> match
>>>>>>>>>>> for the HTTP Host.
>>>>>>>>>>>
>>>>>>>>>>> Matt
>>>>>>>>>>>
>>>>>>>>>>> On 1/24/2013 3:13 AM, Peter Manev wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Jan 24, 2013 at 9:11 AM, Anoop Saldanha
>>>>>>>>>>> <anoopsaldanha at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Jan 24, 2013 at 1:37 PM, Peter Manev <petermanev at gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, any of the techniques mentioned above isn't a foolproof way
>>>>>>>>>>>>>> to match on the host header.  The right way would be to provide a
>>>>>>>>>>>>>> new
>>>>>>>>>>>>>> keyword called "http_host".
>>>>>>>>>>>>> Anoop or Vincent would you please put in feature request for that?
>>>>>>>>>>>>
>>>>>>>>>>>> We should probably consult users/rule-writers if such a keyword would
>>>>>>>>>>>> be useful to them?
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Anoop Saldanha
>>>>>>>>>>>
>>>>>>>>>>> sure
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Regards,
>>>>>>>>>>> Peter Manev
>
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/



-- 
Anoop Saldanha