[Discussion] Rules Syntax

[Martin Holste]

I agree with Claudio regarding the plugin hooks.  That's what's so
inspirational to me in the Nfsen project: they made the plugins really quite
simple to integrate, yet very powerful.  A good example is the Botnet
plugin, which is what we've had the most success with thus far.  It works by
grabbing a blacklist (I am using the Emerging Threats Bot CC list), parsing
it out of Snort rule format and putting it into Nfsen filter format, then
checking the traffic every five minutes to see if there are any hits.  I've
customized it slightly to exclude port 25 and 53 traffic (since that ends up
being way too chatty) and I've tweaked the configuration to only bother me
when there are more than a few packets.  It then logs an alert to the alert
database and sends me an email.  My SIM can query the database to grab the
latest alerts and do any follow-up incident creation, etc.

This setup has proven to be very useful, and has saved our butts a couple of
times with extra-careful malware.  (As in unpacked, sandbox-defeating
malware).  The project itself is getting on in years, but the code
architecture is really effective and well-suited to its purpose.  Namely, it
doesn't try to do everything, it's clear and concise, and it has a simple
way to extend it: create a package which exports functions from the short
and predefined list of plugin functions (e.g. alert_condition,
lookup_address, etc.), drop your package in the plugins dir, and boom, it's
now part of the system and can be implemented from the web GUI.  I'm looking
to create a meta-blacklist plugin in the near future that would implement
some basic scoring across many different blacklists.

My point is that though the Nfsen project's default install was reasonably
useful, it's true helpfulness has been the ease with which it can be
extended and integrated with our environment.

--Martin

On Wed, Jan 7, 2009 at 2:27 AM, Claudio Criscione <
c.criscione at securenetwork.it> wrote:

> Happy new year ;-)
>
> On Sunday 04 January 2009 01:53:49 Martin Holste wrote:
> > I think that Claudio's English signatures are good examples to start
> with.
> > Even a repository of paragraphs like that would be valuable to me.
>
> I'm not sure a repository would really be useful - apart from inspiration.
> I
> feel we need some sort of engine able to parse user-provided rules like
> that
> one. Maybe something "a-la Peakflow", but far more flexible: this kind of
> alerts
> are more about what's normal in your network than what's anomalous (i.e.
> related to an attack). So it's really important to be able to let each user
> build their own rulebase, or even better to be able to infer rules from
> actual
> network flows...
>
> > Regarding statistical analysis, I highly encourage the group to check out
> > the NFSen project (nfsen.sourceforge.net) for inspiration.  They have an
> [...]
> > existing SIM.  I would think the Holt-Winters plugin code could be
> modified
> > to analyze a lot of input types.
>
> Seems an interesting project, if a bit outdated. As of today, which kind of
> attacks can you detect in your organization with this tool?
>
> > space.  By default, Time Machine will store only the first N bytes of
> > traffic per connection, so you can get a really great profile of a ton of
> > connections with a reasonable amount of disk.  For those of you with any
>
> Which brings us to the "size of the window" issue... but that's another
> story.
>
> Matt wrote:
> > > How do we go after that then? (Call to the db experts out there)
> > > Use a sliding scale so as the user-defined storage space allocated
> fills
> > > up older data drops out?
>
> I would definitly thing of some aggregation scheme... I can't remember the
> proper name but I'm sure any warehousing expert could enlighten me. We
> could
> have a greater resolution of data on, say, a week. Then we start to
> aggregate,
> for instance instead of saving the data we only record the protocol and the
> ports for up to one month, and we compress tcpdump data for a week. After
> one
> month, we only record IP endpoints and delete everything else.
> In such a way, we somehow mitigate "rotation" issues...
>
> > > integration/AD, netbios login monitoring, etc. It's possible, but it's
> a
> > > big thing to tackle. And likely we'd have patent conflicts. We can
> > > explore that though if there's a large enough driver to get it.
> Thoughts?
>
> I think that we should build as many "plugin hooks" as possible. Let those
> who
> have the patents do the work, if they care... but an integration with AD
> (which is one of the most widely used "IM solutions") would be useful and
> interesting.
>
> > > contract or grant fund a real statistician or group of such. Maybe we
> > > could get it made into a class project at a university somewhere under
> > > the guidance of an experienced statistician.
>
> I'm sure many universities would be interested but, in order to have useful
> results, very very very detailed requirements should be provided :)
>
> --
> Claudio Criscione
>
> Secure Network S.r.l
> Via Venezia, 23 - 20099 Sesto San Giovanni (MI) - Italia
> Tel: +39 02.24126788 Mob: +39 392 3389178
> email: c.criscione at securenetwork.it
> web: www.securenetwork.it
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20090115/f6f1d0a6/attachment-0002.html>