[Oisf-users] Suricata 1.4 http keywords in rule options, how does matching occur for http_header?

[Anoop Saldanha]

* comments inline

On Wed, Jan 23, 2013 at 11:27 PM, Vincent Fang <vincent.y.fang at gmail.com> wrote:
> How should I send the files to you?

You can mail it to me.

> I did a restart, and then turned on
> wireshark and had it capturing on eth0, turned on suricata with both rules
> on, and then opened up chrome to visit the businessweek webpage for the
> first time and the number of http alerts vs pcre alerts are not the same. I
> saved the pcap file along with the logs and then tried to run suricata in
> offline mode using the -r option but in this run it generated no data in the
> fast.log or http.log.

- have you checked the pcap if you have all the packets from the flow?
- does wireshark show the checksums for the packets as invalid?  If so
you can disable checksum-validation(set it to "no") in the conf file
and re-run suricata.

>
>
> On Tue, Jan 22, 2013 at 5:14 AM, Anoop Saldanha <anoopsaldanha at gmail.com>
> wrote:
>>
>> On Tue, Jan 22, 2013 at 2:27 AM, Vincent Fang <vincent.y.fang at gmail.com>
>> wrote:
>> > Looking at the snort documentation, regex, and what you suggested, I
>> > modified my rule to look like this now and I set the protocol to ip to
>> > catch
>> > everything going out of my machine
>> >
>> > alert ip $HOME_NET any -> any any (msg:"alert fired";
>> > pcre:"/Host:\s.*?\.businessweek.com/H";)
>> >
>> > and when I start up my suricata with just that 1 rule and visit business
>> > week on the front page, no alerts show. I click on some subsequent links
>> > and
>> > the fast.log still doesn't populate. Only after I click on a few more
>> > links
>> > do I see the rules fire. But it's significantly less alerts than if I
>> > used
>> > the rule
>> >
>> > alert ip $HOME_NET any -> any any (msg:"alert fired";
>> > content:"businessweek.com"; http_header;)
>> >
>> > Can anyone explain why I'm not seeing it fire when I visit the front
>> > page?
>> >
>> > My setup is running wireshark, suricata, and google-chrome on VMPlayer
>> > with
>> > the OS being Fedora 17 and the host machine is Windows 7. I can't
>> > understand
>> > why the number of alerts isn't matching the number of packets I see
>> > matching
>> > in my wireshark with the filter http && ip.src == 192.168.0.0/16 &&
>> > http.host contains "businessweek.com"
>> >
>>
>> Any chance that you accessed the website first with rule containing
>> "content:"businessweek.com"" loaded, and then restarted the engine
>> with pcre rule.  If so, the connection might have been still alive
>> from the browser and it would have reused the same old flow and
>> suricata may not have picked it up(midstream flow).
>>
>> To rule out any alert discrepancy from the above case, try this - Load
>> both the rules at once, instead of conducting separate trials for each
>> rule.  So your rules file would now contain -
>>
>> alert ip $HOME_NET any -> any any (msg:"alert fired";
>> content:"businessweek.com"; http_header; sid:1;)
>> alert ip $HOME_NET any -> any any (msg:"alert fired";
>> pcre:"/Host:\s.*?\.businessweek.com/H"; sid:2;)
>>
>> Now run it through the traffic.  You should see an equal no of alerts
>> for both sig 1 and sig 2.  If you don't, please capture a pcap and
>> send it over.  Will have it checked.
>>
>> >
>> > On Tue, Jan 15, 2013 at 3:11 AM, Anoop Saldanha
>> > <anoopsaldanha at gmail.com>
>> > wrote:
>> >>
>> >> On Tue, Jan 15, 2013 at 3:30 AM, Vincent Fang
>> >> <vincent.y.fang at gmail.com>
>> >> wrote:
>> >> > Since I cannot rely on external ip addresses, I've been testing out a
>> >> > new
>> >> > rule that examines the http content, specificially the http header.
>> >> > Looking
>> >> > at the http-keyword page on redmine, this is the new rule I created
>> >> > for
>> >> > testing
>> >> >
>> >> > alert http any any -> any any (msg:"rule fired";
>> >> > content:"businessweek.com";
>> >> > http_header;)
>> >> >
>> >> > Looking at the example, it explains it would look at the http header
>> >> > for
>> >> > any
>> >> > matching and create an alert if it matches. My one question is how
>> >> > exactly
>> >> > does it do the matching?
>> >> >
>> >> >
>> >> > For instance, the businessweek.com address can appear in two places
>> >> > based on
>> >> > what I examined in the wireshark http packets
>> >> >
>> >> > Host: www.businessweek.com\r\n
>> >> >
>> >> > or
>> >> >
>> >> > Referer: http://www.businessweek.com/\r\n
>> >> >
>> >> > The examples shown in redmine only show it matching to the Host field
>> >> > but
>> >> > would this same rule also match if the address showed up in the
>> >> > Referer
>> >> > field? Right now I'm running into issues with Chrome caching which is
>> >> > fuddling my results so I can't tell based on my testing, but I was
>> >> > wondering
>> >> > if anyone had any experience and knows what Suricata is doing with
>> >> > the
>> >> > content matching in the header field.
>> >> >
>> >> > Also is it possible to specify a wildcard * in the content:
>> >> > "*businessweek.com" for matching?
>> >>
>> >> No.
>> >>
>> >> >
>> >> > The end goal is for me to maybe specify that
>> >> >
>> >> > content:"Host: *businessweek.com"; http_header;
>> >>
>> >> This would work instead, although you are stuck with the subdomain -
>> >>
>> >> content:"Host: www.businessweek.com"; http_header;
>> >>
>> >> Contents can have fixed strings only.
>> >>
>> >> If you'd want to use wildcards you can use pcre -
>> >>
>> >> pcre:"/Host: .*businessweek.com"/H";
>> >>
>> >> The option "H" refers to matching on the http header filed.  We have
>> >> other such pcre options to match on different http fields/buffers.
>> >>
>> >> >
>> >> > so that Suricata would match all packets going to this address. I ran
>> >> > a
>> >> > test
>> >> > with the modified rule
>> >> >
>> >> > alert http any any -> any any (msg:"rule fired";
>> >> > content:"*businessweek.com"; http_header;)
>> >> >
>> >> > and fast.log isn't populating. I'm not sure if I'm doing it wrong or
>> >> > if
>> >> > wildcards are not supported in Suricata rules.
>> >> >
>> >>
>>
>>
>>
>> --
>> Anoop Saldanha
>
>



-- 
Anoop Saldanha