[Oisf-users] Suricata 1.4 http keywords in rule options, how does matching occur for http_header?

[Vincent Fang]

Wanted to give the results of the latest patch from Anoop on testing.

Wireshark filter:

http && http.host contains "businessweek.com"

wireshark caught 45 packets that matched this criteria

http-test.rules"
alert http $HOME_NET any -> any any (msg:"pcre rule fired"; pcre:"/\.
businessweek.com/iW"; sid:1;)
alert http $HOME_NET any -> any any(msg:"http rule fired"; content:".
businessweek.com"; nocase; http_host; sid:2;)

fast.log had 90 lines
45 from pcre
45 from http

One thing I wanted to comment was Brian saying how for some clients that
still use HTTP/1.0 with no header host field. There is an http keyword in
Suricata http_uri; I'm guessing that would mean that I would need to create
another rule to complement the http_host if I wanted to catch all dns names
of clients going to the businessweek site?

alert http $HOME_NET any -> any any(msg:"http uri rule fired"; content:"
businessweek.com"; http_uri; sid:3;)

This rule seems to work although I have to take off the leading . from the
domain name.


On Mon, Jan 28, 2013 at 5:30 AM, Victor Julien <lists at inliniac.net> wrote:

> On 01/26/2013 04:22 PM, Anoop Saldanha wrote:
> > Thanks for the review Brian.
> >
> > Have updated the patch to now use tx->parsed_uri->hostname.  Will
> > cover the unittests for the various hostname cases shortly.
> >
> > Note: Since the hostname is normalized to lowercase, I'm invalidating
> > signatures which doesn't have a nocase attached to the http_header
> > content.  Same for pcre.
>
> I assume you mean http_host content?
>
> Is there a way to get the hostname as it appeared in the request?
>
> > * This isn't an incremental patch.  Have amended to the old commit
> > instead.  You'll have to undo my previous patch, before applying this
> > one.
> >
> > On Sat, Jan 26, 2013 at 8:58 AM, Will Metcalf <william.metcalf at gmail.com>
> wrote:
> >> endswith sure seems a lot cleaner :)...
> >>
> >> Regards,
> >>
> >> Will
> >>
> >> On Fri, Jan 25, 2013 at 9:22 PM, Anoop Saldanha <
> anoopsaldanha at gmail.com> wrote:
> >>> On Fri, Jan 25, 2013 at 8:53 PM, Will Metcalf <
> william.metcalf at gmail.com> wrote:
> >>>> While we are talking about having len and endswith would be really
> >>>> useful for at least http_uri, http_user_agent, and http_host_header.
> >>>> The first for performing exact matches i.e.
> >>>>
> >>>> content:"Mozilla"; http_user_agent; http_user_agent_len:7;
> >>>>
> >>>> to match
> >>>>
> >>>> User-Agent: Mozilla\r\n
> >>>>
> >>>
> >>> Sounds good.
> >>>
> >>>> or
> >>>>
> >>>> content:".exe"; http_uri; endswith;
> >>>>
> >>>> to match
> >>>>
> >>>> GET /blah/blat/foo.exe HTTP/1.1\r\n
> >>>>
> >>>
> >>> Can be done with isdataat?  content:"exe"; http_uri; isdataat:!0,
> relative;
> >>>
> >>>>
> >>>> etc... Want a feature request? :)
> >>>>
> >>>> Regards,
> >>>>
> >>>> Will
> >>>>
> >>>> On Fri, Jan 25, 2013 at 8:47 AM, William Metcalf
> >>>> <william.metcalf at gmail.com> wrote:
> >>>>> If the CRLF is normalized out from a rule writers perspective we
> probably continue to use http_header in most cases as generally you want to
> match on say google.com what you really want to match is .google.com\r\n
> and not .somethinggoogle.com.ru. Would it be possible to add a endswith
> modifier for this normalized buffer?
> >>>>>
> >>>
> >>> Yes we can add, but it probably doesn't make sense since we might
> >>> probably pick the hostname from the uri section as well?
> >>>
> >>> One can still do it using isdataat actually - content:"google.com";
> >>> http_header; isdataat:!0, relative;
> >>>
> >>>>> Regards,
> >>>>>
> >>>>> Will
> >>>>> On Jan 25, 2013, at 7:19 AM, Anoop Saldanha <anoopsaldanha at gmail.com>
> wrote:
> >>>>>
> >>>>>> Try this patch out(you can apply the patch using "git am -3
> <patch>")
> >>>>>>
> >>>>>> It introduces a new keyword + pcre modifier that would inspect just
> >>>>>> the host header.
> >>>>>>
> >>>>>> The keyword being "http_host" and the pcre modifier being "W"
> >>>>>>
> >>>>>> You can now use it in a rule like this -
> >>>>>>
> >>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
> >>>>>> pcre:"/\.businessweek.com/W"; sid:1;)
> >>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
> >>>>>> content:".businessweek.com"; http_host; sid:2;)
> >>>>>>
> >>>>>> Let me know how it works with the above rules.
> >>>>>>
> >>>>>> On Fri, Jan 25, 2013 at 8:47 AM, Anoop Saldanha <
> anoopsaldanha at gmail.com> wrote:
> >>>>>>> On Thu, Jan 24, 2013 at 10:29 PM, Vincent Fang <
> vincent.y.fang at gmail.com> wrote:
> >>>>>>>> Here's the new results, I will run the tests again to see if it's
> consistent
> >>>>>>>> but using the wireshark filter
> >>>>>>>>
> >>>>>>>> http contains "businessweek.com"
> >>>>>>>>
> >>>>>>>> there were 75 matches
> >>>>>>>>
> >>>>>>>> and in the fast.log there were 138 total alerts from the two new
> rules you
> >>>>>>>> specified
> >>>>>>>> grep -c "http header" fast.log -> 69 lines
> >>>>>>>> grep -c "pcre version" fast.log -> 69 lines
> >>>>>>>>
> >>>>>>>> so they're both the same. Ran suricata in offline mode and the
> results were
> >>>>>>>> the same so that's good since they're consistent.
> >>>>>>>>
> >>>>>>>> Here's a copy of the two rules
> >>>>>>>>
> >>>>>>>> alert ip $HOME_NET any -> any any (msg:"pcre version rule fired";
> >>>>>>>> pcre:"/\s.*?\.businessweek.com/H"; sid:1;)
> >>>>>>>>
> >>>>>>>> alert ip $HOME_NET any -> any any (msg:"http header rule fired";
> >>>>>>>> content:".businessweek.com"; http_header; sid:2;)
> >>>>>>>>
> >>>>>>>> In the next few runs I also plan to change the protocol to http
> instead of
> >>>>>>>> ip, and I technically should get the same numbers yes?
> >>>>>>>
> >>>>>>> Yes, you should.
> >>>>>>>
> >>>>>>> Keep in mind that the above rules can also match on other headers
> >>>>>>> containing businessweek.com, for example the referer header.
> >>>>>>>
> >>>>>>>>
> >>>>>>>> On Thu, Jan 24, 2013 at 9:44 AM, Anoop Saldanha <
> anoopsaldanha at gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Sound good.  Will open a feature request for "http_host" keyword;
> >>>>>>>>>
> >>>>>>>>> On Thu, Jan 24, 2013 at 7:45 PM, Matt <matt at somedamn.com> wrote:
> >>>>>>>>>> I would find that useful, especially if it increases efficiency
> in the
> >>>>>>>>>> same
> >>>>>>>>>> way as http_user_agent.  Among other things, I use Suricata to
> match
> >>>>>>>>>> blacklists of known bad URLs, and all those rules include a
> content
> >>>>>>>>>> match
> >>>>>>>>>> for the HTTP Host.
> >>>>>>>>>>
> >>>>>>>>>> Matt
> >>>>>>>>>>
> >>>>>>>>>> On 1/24/2013 3:13 AM, Peter Manev wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Jan 24, 2013 at 9:11 AM, Anoop Saldanha
> >>>>>>>>>> <anoopsaldanha at gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Jan 24, 2013 at 1:37 PM, Peter Manev <
> petermanev at gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> However, any of the techniques mentioned above isn't a
> foolproof way
> >>>>>>>>>>>>> to match on the host header.  The right way would be to
> provide a
> >>>>>>>>>>>>> new
> >>>>>>>>>>>>> keyword called "http_host".
> >>>>>>>>>>>> Anoop or Vincent would you please put in feature request for
> that?
> >>>>>>>>>>>
> >>>>>>>>>>> We should probably consult users/rule-writers if such a
> keyword would
> >>>>>>>>>>> be useful to them?
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Anoop Saldanha
> >>>>>>>>>>
> >>>>>>>>>> sure
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Regards,
> >>>>>>>>>> Peter Manev
>
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130128/e95c1760/attachment-0002.html>