[Oisf-users] Suricata 1.4 http keywords in rule options, how does matching occur for http_header?

[Anoop Saldanha]

There's a mistake in the rules.  Can you retry the above experiment
with the rules -

alert ip $HOME_NET any -> any any (msg:"alert fired";
content:".businessweek.com"; http_header; sid:1;)
alert ip $HOME_NET any -> any any (msg:"alert fired";
pcre:"/\s.*?\.businessweek.com/H"; sid:2;)

The first rule content string doesn't have a "Host" in it, so remove
it from the pcre as well.  We are basically asking it to give us a
match as long we have a sub-domained businessweek.com in any header.
Does this give you the same amount of alerts?

-----
Next-

Modify sig 1 and sig 2 to include "Host" as well.  Your rules file
would now be -
alert ip $HOME_NET any -> any any (msg:"alert fired"; content:"Host:";
content:".businessweek.com"; within:40; http_header; sid:1;)
alert ip $HOME_NET any -> any any (msg:"alert fired";
pcre:"/Host:\s.*?\.businessweek.com/H"; sid:2;)

The content string would try to match on businessweek.com when present
in the host header(more like an approximate).  It will try to find the
string ".businessweek.com" within the next 40 bytes after "Host:"
string.  Do note that the 40 bytes is an approximate hoping that all
sub-domains's length would be within that range.

Check if you still get the same no of alerts?

-----

Try these above rules loaded, once against your pcap and next against
live traffic and see if you get the same no of alerts.

-----

However, any of the techniques mentioned above isn't a foolproof way
to match on the host header.  The right way would be to provide a new
keyword called "http_host".

On Wed, Jan 23, 2013 at 11:54 PM, Anoop Saldanha
<anoopsaldanha at gmail.com> wrote:
> * comments inline
>
> On Wed, Jan 23, 2013 at 11:27 PM, Vincent Fang <vincent.y.fang at gmail.com> wrote:
>> How should I send the files to you?
>
> You can mail it to me.
>
>> I did a restart, and then turned on
>> wireshark and had it capturing on eth0, turned on suricata with both rules
>> on, and then opened up chrome to visit the businessweek webpage for the
>> first time and the number of http alerts vs pcre alerts are not the same. I
>> saved the pcap file along with the logs and then tried to run suricata in
>> offline mode using the -r option but in this run it generated no data in the
>> fast.log or http.log.
>
> - have you checked the pcap if you have all the packets from the flow?
> - does wireshark show the checksums for the packets as invalid?  If so
> you can disable checksum-validation(set it to "no") in the conf file
> and re-run suricata.
>
>>
>>
>> On Tue, Jan 22, 2013 at 5:14 AM, Anoop Saldanha <anoopsaldanha at gmail.com>
>> wrote:
>>>
>>> On Tue, Jan 22, 2013 at 2:27 AM, Vincent Fang <vincent.y.fang at gmail.com>
>>> wrote:
>>> > Looking at the snort documentation, regex, and what you suggested, I
>>> > modified my rule to look like this now and I set the protocol to ip to
>>> > catch
>>> > everything going out of my machine
>>> >
>>> > alert ip $HOME_NET any -> any any (msg:"alert fired";
>>> > pcre:"/Host:\s.*?\.businessweek.com/H";)
>>> >
>>> > and when I start up my suricata with just that 1 rule and visit business
>>> > week on the front page, no alerts show. I click on some subsequent links
>>> > and
>>> > the fast.log still doesn't populate. Only after I click on a few more
>>> > links
>>> > do I see the rules fire. But it's significantly less alerts than if I
>>> > used
>>> > the rule
>>> >
>>> > alert ip $HOME_NET any -> any any (msg:"alert fired";
>>> > content:"businessweek.com"; http_header;)
>>> >
>>> > Can anyone explain why I'm not seeing it fire when I visit the front
>>> > page?
>>> >
>>> > My setup is running wireshark, suricata, and google-chrome on VMPlayer
>>> > with
>>> > the OS being Fedora 17 and the host machine is Windows 7. I can't
>>> > understand
>>> > why the number of alerts isn't matching the number of packets I see
>>> > matching
>>> > in my wireshark with the filter http && ip.src == 192.168.0.0/16 &&
>>> > http.host contains "businessweek.com"
>>> >
>>>
>>> Any chance that you accessed the website first with rule containing
>>> "content:"businessweek.com"" loaded, and then restarted the engine
>>> with pcre rule.  If so, the connection might have been still alive
>>> from the browser and it would have reused the same old flow and
>>> suricata may not have picked it up(midstream flow).
>>>
>>> To rule out any alert discrepancy from the above case, try this - Load
>>> both the rules at once, instead of conducting separate trials for each
>>> rule.  So your rules file would now contain -
>>>
>>> alert ip $HOME_NET any -> any any (msg:"alert fired";
>>> content:"businessweek.com"; http_header; sid:1;)
>>> alert ip $HOME_NET any -> any any (msg:"alert fired";
>>> pcre:"/Host:\s.*?\.businessweek.com/H"; sid:2;)
>>>
>>> Now run it through the traffic.  You should see an equal no of alerts
>>> for both sig 1 and sig 2.  If you don't, please capture a pcap and
>>> send it over.  Will have it checked.
>>>
>>> >
>>> > On Tue, Jan 15, 2013 at 3:11 AM, Anoop Saldanha
>>> > <anoopsaldanha at gmail.com>
>>> > wrote:
>>> >>
>>> >> On Tue, Jan 15, 2013 at 3:30 AM, Vincent Fang
>>> >> <vincent.y.fang at gmail.com>
>>> >> wrote:
>>> >> > Since I cannot rely on external ip addresses, I've been testing out a
>>> >> > new
>>> >> > rule that examines the http content, specificially the http header.
>>> >> > Looking
>>> >> > at the http-keyword page on redmine, this is the new rule I created
>>> >> > for
>>> >> > testing
>>> >> >
>>> >> > alert http any any -> any any (msg:"rule fired";
>>> >> > content:"businessweek.com";
>>> >> > http_header;)
>>> >> >
>>> >> > Looking at the example, it explains it would look at the http header
>>> >> > for
>>> >> > any
>>> >> > matching and create an alert if it matches. My one question is how
>>> >> > exactly
>>> >> > does it do the matching?
>>> >> >
>>> >> >
>>> >> > For instance, the businessweek.com address can appear in two places
>>> >> > based on
>>> >> > what I examined in the wireshark http packets
>>> >> >
>>> >> > Host: www.businessweek.com\r\n
>>> >> >
>>> >> > or
>>> >> >
>>> >> > Referer: http://www.businessweek.com/\r\n
>>> >> >
>>> >> > The examples shown in redmine only show it matching to the Host field
>>> >> > but
>>> >> > would this same rule also match if the address showed up in the
>>> >> > Referer
>>> >> > field? Right now I'm running into issues with Chrome caching which is
>>> >> > fuddling my results so I can't tell based on my testing, but I was
>>> >> > wondering
>>> >> > if anyone had any experience and knows what Suricata is doing with
>>> >> > the
>>> >> > content matching in the header field.
>>> >> >
>>> >> > Also is it possible to specify a wildcard * in the content:
>>> >> > "*businessweek.com" for matching?
>>> >>
>>> >> No.
>>> >>
>>> >> >
>>> >> > The end goal is for me to maybe specify that
>>> >> >
>>> >> > content:"Host: *businessweek.com"; http_header;
>>> >>
>>> >> This would work instead, although you are stuck with the subdomain -
>>> >>
>>> >> content:"Host: www.businessweek.com"; http_header;
>>> >>
>>> >> Contents can have fixed strings only.
>>> >>
>>> >> If you'd want to use wildcards you can use pcre -
>>> >>
>>> >> pcre:"/Host: .*businessweek.com"/H";
>>> >>
>>> >> The option "H" refers to matching on the http header filed.  We have
>>> >> other such pcre options to match on different http fields/buffers.
>>> >>
>>> >> >
>>> >> > so that Suricata would match all packets going to this address. I ran
>>> >> > a
>>> >> > test
>>> >> > with the modified rule
>>> >> >
>>> >> > alert http any any -> any any (msg:"rule fired";
>>> >> > content:"*businessweek.com"; http_header;)
>>> >> >
>>> >> > and fast.log isn't populating. I'm not sure if I'm doing it wrong or
>>> >> > if
>>> >> > wildcards are not supported in Suricata rules.
>>> >> >
>>> >>
>>>
>>>
>>>
>>> --
>>> Anoop Saldanha
>>
>>
>
>
>
> --
> Anoop Saldanha



-- 
Anoop Saldanha