[Oisf-users] real time alert on tcp stream and flowint
Peter Manev
petermanev at gmail.com
Tue Feb 14 09:29:50 UTC 2012
On Tue, Feb 14, 2012 at 10:21 AM, Victor Julien <victor at inliniac.net> wrote:
> On 02/12/2012 08:15 AM, Nikolay Denev wrote:
> >
> > On Feb 11, 2012, at 10:11 PM, Peter Manev wrote:
> >
> >>
> >>
> >> On Sat, Feb 11, 2012 at 8:27 PM, Nikolay Denev <ndenev at gmail.com
> >> <mailto:ndenev at gmail.com>> wrote:
> >>
> >>
> >> On Feb 11, 2012, at 9:14 PM, Peter Manev wrote:
> >>
> >>>
> >>>
> >>> On Sat, Feb 11, 2012 at 7:42 PM, Nikolay Denev <ndenev at gmail.com
> >>> <mailto:ndenev at gmail.com>> wrote:
> >>>
> >>>
> >>> On Feb 11, 2012, at 7:52 PM, Peter Manev wrote:
> >>>
> >>>>
> >>>>
> >>>> On Sat, Feb 11, 2012 at 4:31 PM, Nikolay Denev
> >>>> <ndenev at gmail.com <mailto:ndenev at gmail.com>> wrote:
> >>>>
> >>>>
> >>>> On Feb 11, 2012, at 12:11 PM, Peter Manev wrote:
> >>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Feb 10, 2012 at 6:43 AM, Nikolay Denev
> >>>>> <ndenev at gmail.com <mailto:ndenev at gmail.com>> wrote:
> >>>>>
> >>>>>
> >>>>> On Feb 9, 2012, at 10:04 PM, Nikolay Denev wrote:
> >>>>>
> >>>>> > On Feb 9, 2012, at 10:03 PM, Nikolay Denev wrote:
> >>>>> >
> >>>>> >> Hi all,
> >>>>> >>
> >>>>> >> It's probably stupid question and I'm missing
> >>>>> something but I don't seem to be able
> >>>>> >> to generate alert immediately when for example a
> >>>>> given string is found inside a TCP stream.
> >>>>> >> When the TCP connection closes, suricata
> >>>>> immediately prints the alert in fast.log.
> >>>>> >> How can I make the alert be generated
> >>>>> immediately when the rule condition is matched?
> >>>>> >>
> >>>>> >> Also I don't know if its because of this I don't
> >>>>> seem to be able to trigger the rule to match
> >>>>> several times on the same stream,
> >>>>> >> while I have the string that should fire the
> >>>>> alert several times in the stream.
> >>>>> >>
> >>>>> >> Here's an example :
> >>>>> >>
> >>>>> >> alert tcp $HOME_NET 6666 -> any any \
> >>>>> >> (msg:"got one"; content:"something";
> >>>>> flowint:something,notset; flowint:something,=,1;
> >>>>> sid:10;)
> >>>>> >>
> >>>>> >> alert tcp $HOME_NET 6666 -> any any \
> >>>>> >> (msg:"got five or more";
> >>>>> content:"something"; flowint:something,isset;
> >>>>> flowint:something,+,1; flowint:something,>,5;
> sid:11;)
> >>>>> >>
> >>>>> >> This never works, I just have the first rule
> >>>>> fire once when the TCP session is terminated.
> >>>>> >>
> >>>>> >>
> >>>>> >> P.S.: As a side note the wiki should be updated
> >>>>> to include probably "sid"s for the rules, as
> >>>>> currently when I try to run the examples
> >>>>> >> suricata complains about duplicated rules.
> >>>>> >>
> >>>>> >> Thanks,
> >>>>> >>
> >>>>> >
> >>>>> > I'm running 1.2.1 RELEASE on FreeBSD-9.0-STABLE.
> >>>>>
> >>>>> This seems to work :
> >>>>>
> >>>>> alert tcp $HOME_NET 6666 -> any any \
> >>>>> (msg:"got one"; content:"something";
> >>>>> flowint:something,notset; flowint:something,=,1;
> >>>>> noalert; sid:10; priority: 1;)
> >>>>>
> >>>>> alert tcp $HOME_NET 6666 -> any any \
> >>>>> (msg:"got more"; content:"something";
> >>>>> flowint:something,isset; flowint:something,+,1;
> >>>>> noalert; sid:11; priority: 2;)
> >>>>>
> >>>>>
> >>>>> alert tcp $HOME_NET 6666 -> any any \
> >>>>> (msg:"got too many"; content:"something";
> >>>>> flowint:something,isset; flowint:something,>,2;
> >>>>> sid:12; priority: 3;)
> >>>>>
> >>>>>
> >>>>> _______________________________________________
> >>>>> Oisf-users mailing list
> >>>>> Oisf-users at openinfosecfoundation.org
> >>>>> <mailto:Oisf-users at openinfosecfoundation.org>
> >>>>>
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> >>>>>
> >>>>>
> >>>>>
> >>>>> Hi Nikolay,
> >>>>> I think this is the way it is supposed to work. (last
> >>>>> example, by you).
> >>>>>
> >>>>> When you take out "noalert" form sid 11 - does it fire ?
> >>>>>
> >>>>> And are these the only rules that are loaded in terms
> >>>>> of flowint or you have others before that?
> >>>>>
> >>>>> thanks
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Peter Manev
> >>>>
> >>>>
> >>>> Yes, It fires, the problem I have is that it doesn't
> >>>> fire for each occurence of "content".
> >>>> Is alert supposed to fire once per packet if it matches,
> >>>> or for each match in the stream?
> >>>>
> >>>> For example now I'm using these rules to catch if there
> >>>> are more than some defined amount of email addresses in
> >>>> a given stream :
> >>>>
> >>>>
> >>>> alert tcp $HOME_NET 80 -> any any \
> >>>> (msg:"got one email addr"; content:"|40|";
> >>>> pcre:"/[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}/i"; \
> >>>> flow:established,from_server;
> >>>> flowint:something,notset; flowint:something,=,1; sid:10;
> >>>> priority:3; noalert;)
> >>>>
> >>>> alert tcp $HOME_NET 80 -> any any \
> >>>> (msg:"got more email addrs"; content:"|40|";
> >>>> pcre:"/[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}/i"; \
> >>>> flow:established,from_server;
> >>>> flowint:something,isset; flowint:something,+,1; sid:11;
> >>>> priority:2; noalert;)
> >>>>
> >>>> alert tcp $HOME_NET 80 -> any any \
> >>>> (msg:"Got too many email addrs!";
> >>>> content:"|40|";
> >>>> pcre:"/[a-z0-9._%+-]+@[a-z0-9.-]+\.[a-z]{2,4}/i"; \
> >>>> flow:established,from_server;
> >>>> flowint:something,isset; flowint:something,>,10; sid:12;
> >>>> priority:1; classtype:policy-violation;)
> >>>>
> >>>>
> >>>> This for example works, but would not match for a simple
> >>>> plain text file with 10 email adresses, I need to have
> >>>> maybe 40-50 or more for this to match.
> >>>> Maybe I'm missing something…
> >>>>
> >>>> And yes, these are my only rules that I'm testing with.
> >>>> No other rules with or without flowint whatsoever.
> >>>>
> >>>>
> >>>> Hi ,
> >>>> Just so I understand you correctly - you have a text file
> >>>> (in the stream) and in that text file you have 10 e-mail
> >>>> addresses and it wold not fire. correct ?
> >>>>
> >>>>
> >>>> thanks
> >>>>
> >>>>
> >>>> --
> >>>> Peter Manev
> >>>
> >>> Exactly.
> >>>
> >>> For example if I try to fetch the file emails.txt via http
> >>> which has the following content :
> >>>
> >>> # cat emails.txt
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>>
> >>> $ curl http://testserver/emails.txt
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> edin at email.com <mailto:edin at email.com>
> >>> $
> >>>
> >>> And I also remove the "noalert" option from the rules, this
> >>> is what I get in fast.log :
> >>>
> >>> 02/11/2012-20:37:23.988271 [**] [1:10:0] got one email addr
> >>> [**] [Classification: (null)] [Priority: 3] {TCP} X.X.X.X:80
> >>> -> Y.Y.Y.Y:57923
> >>> 02/11/2012-20:37:23.988271 [**] [1:11:0] got more email
> >>> addrs [**] [Classification: (null)] [Priority: 2] {TCP}
> >>> X.X.X.X:80 -> Y.Y.Y.Y:57923
> >>>
> >>>
> >>> If I change the third rule to fire if the flowint var is more
> >>> than 1, it is being triggered.
> >>>
> >>> If I insert some random data between the email addresses in
> >>> the text file, then I get 4 maybe 5 matches. Doesn't it have
> >>> to match all 10 of them?
> >>>
> >>>
> >>> 1. What happens if you take out the PCRE expressions from all
> >>> the rules ?
> >>> 2. sid:12 - should not fire because you have >10 , and there are
> >>> exactly 10 e-mails in the file
> >>> 3. how big is the stream itself? i think it is below 2KB, correct?
> >>> 4. is the PCRE matching the e-mails, under the unix shell ?
> >>> 5. yes i think you should get more sid:11 alerts - but first lets
> >>> investigate the above 4.
> >>>
> >>> thanks
> >>>
> >>> --
> >>> Peter Manev
> >>
> >> The file with only the 10 emails is 160 bytes. Even without pcre I
> >> get the same result :
> >>
> >> alert tcp $HOME_NET 80 -> any any \
> >> (msg:"got one email addr"; content:"|40|"; \
> >> flow:established,from_server; flowint:something,notset;
> >> flowint:something,=,1; sid:10; priority:3;)
> >>
> >> alert tcp $HOME_NET 80 -> any any \
> >> (msg:"got more email addrs"; content:"|40|"; \
> >> flow:established,from_server; flowint:something,isset;
> >> flowint:something,+,1; sid:11; priority:2;)
> >>
> >> alert tcp $HOME_NET 80 -> any any \
> >> (msg:"Got too many email addrs!"; content:"|40|"; \
> >> flow:established,from_server; flowint:something,isset;
> >> flowint:something,>,9; sid:12; priority:1;
> >> classtype:policy-violation;)
> >>
> >>
> >> alerts I get :
> >>
> >> 02/11/2012-21:23:14.567194 [**] [1:10:0] got one email addr [**]
> >> [Classification: (null)] [Priority: 3] {TCP} X.X.X.X:80 ->
> >> Y.Y.Y.Y:58158
> >> 02/11/2012-21:23:14.567194 [**] [1:11:0] got more email addrs
> >> [**] [Classification: (null)] [Priority: 2] {TCP} X.X.X.X:80 ->
> >> Y.Y.Y.Y:58158
> >>
> >> If I put some '#' symbols between the emails in the file so that
> >> it gets about 9K big and I fetch it I get these alerts :
> >>
> >> 02/11/2012-21:25:37.755214 [**] [1:10:0] got one email addr [**]
> >> [Classification: (null)] [Priority: 3] {TCP} X.X.X.X:80 ->
> >> Y.Y.Y.Y:58166
> >> 02/11/2012-21:25:37.755214 [**] [1:11:0] got more email addrs
> >> [**] [Classification: (null)] [Priority: 2] {TCP} X.X.X.X:80 ->
> >> Y.Y.Y.Y:58166
> >> 02/11/2012-21:25:37.761077 [**] [1:11:0] got more email addrs
> >> [**] [Classification: (null)] [Priority: 2] {TCP} X.X.X.X:80 ->
> >> Y.Y.Y.Y:58166
> >> 02/11/2012-21:25:37.764451 [**] [1:11:0] got more email addrs
> >> [**] [Classification: (null)] [Priority: 2] {TCP} X.X.X.X:80 ->
> >> Y.Y.Y.Y:58166
> >>
> >>
> >>
> >> Hi Nikolay,
> >>
> >>
> >> Can you please post this as a bug - please be detailed (as you were in
> >> your 2 previous e-mails).
> >> Personally i think here sid 11 is the problem , may be it does not
> >> count/increment correctly....
> >> thanks
> >>
> >>
> >> --
> >> Peter Manev
> >
> > Yes I will post this as a bug. But I've just found a much simpler case.
> >
> > Let's for example have only this rule in suricata :
> >
> > alert tcp $HOME_NET 6666 -> any any (msg:"match"; content:"|40|";)
> >
> > Then on a monitored machine from the $HOME_NET range I do :
> >
> > echo "@ @ @ @ @ @ @ @ @" | nc -l 6666
> >
> > And on different host I do :
> >
> > nc testserver 6666
> >
> > This gets the ten @ chars transferred, and I get only one alert.
> > But for example if I echo more @ chars, like 5000 or something, I get
> > 3-6 alerts.
> > I have to check what is actually the number of packets with payload,
> > probably the rule
> > is matched once per packet? But this could not explain that I get
> > different number of alerts on different runs.
>
> The behavior is by design. TCP data by default is inspected in the
> stream context, which means the "@ @ @ @ @ @ @ @ @" buffer is inspected
> at once.
> Suricata will not try to find every possible match in a
> payload, but just one.
>
That's good to know - clears out a few questions of mine....
but then a PCRE (matching on 10 "@") should match all of them - correct?
having in mind they are in the same "chunk".
>
> The reason you get more alerts if you increase the payload
> significantly, is that the stream is inspected in chunks. The size of
> those chunks is determined by your stream toserver_chunk_size setting.
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
--
Peter Manev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20120214/abbe5173/attachment-0002.html>
More information about the Oisf-users
mailing list