[Discussion] Rules Syntax

[Matt Jonkman]

Martin Holste wrote:
> I think that Claudio's English signatures are good examples to start
> with.  Even a repository of paragraphs like that would be valuable to me.
> 
> A port-tracking database wouldn't be that big (as in, only a few gigs)
> to store last remote port on a per-local-IP basis.  With decent
> indexing, it would be extremely fast.

Maybe some kind of hash lookup, or bloom filter kinda thing?

> 
> Regarding statistical analysis, I highly encourage the group to check
> out the NFSen project (nfsen.sourceforge.net
> <http://nfsen.sourceforge.net>) for inspiration.  They have an entire
> framework for statistical alerting on Netflow contained in a small
> amount of very pluggable/extensible Perl packages. I'm extremely
> impressed with its efficiency and the overall software architecture. 
> I've recently implemented it in my org and it took almost zero effort. 
> In particular, they have a plugin which applies a Holt-Winters
> exponential smoothing algorithm to flows and alerts based on those
> statisical anomalies.  Email alerts are standard, but there is a plugin
> for MySQL alert logging as well.  The quality of code is exceptional (I
> was very impressed to see so much effort being spent on input
> validation!) and I plan on bolting it onto my existing SIM.  I would
> think the Holt-Winters plugin code could be modified to analyze a lot of
> input types.

This also looks promising. Anyone used it? Thoughts?

Matt
> 
> On Sun, Dec 21, 2008 at 10:11 AM, Matt Jonkman <jonkman at jonkmans.com
> <mailto:jonkman at jonkmans.com>> wrote:
> 
> 
>     Claudio Criscione wrote:
>     >
>     > Well, what about:
>     >
>     > "if someone in my organization has never started any ftp traffic
>     in the last
>     > three months starts an ftp connection, notify me and start
>     watching more
>     > carefully that person. "
> 
>     I like this too. How do we store that kind of data for long term? Even
>     if we were to just store last timestamp we saw that port in use on this
>     IP that'd be a significant amount of data on the average net to go back
>     months, no?
> 
>     How do we go after that then? (Call to the db experts out there)
> 
>     Use a sliding scale so as the user-defined storage space allocated fills
>     up older data drops out?
> 
>     Use a limited range of ports, and/or group together high port ranges?
> 
> 
>     >
>     >  - Someone vs some machine
>     > Using the IP address is still the only way to go in most cases,
>     but we need
>     > more sophisticate means to identify who's who as networks evolve
>     (think about
>     > whole cities behind a NAT)
> 
>     I think we should think more inside the firewall for these issues, no?
> 
>     There are ways, and several commercial products that track a user to an
>     IP in realtime. Cisco I believe does, and others surely. LDAP
>     integration/AD, netbios login monitoring, etc. It's possible, but it's a
>     big thing to tackle. And likely we'd have patent conflicts. We can
>     explore that though if there's a large enough driver to get it.
>     Thoughts?
> 
> 
>     > - in the last three monts can actually be translated to "is not
>     used to"
>     > or "does not usually"
>     > The issue with statistical approaches is that you really have to
>     develope
>     > custom models. What about "signature based statistical models"?
> 
>     Yes, statistical approaches are tough. I'd like to see what is available
>     out there in this area these days as far as open research. As I
>     mentioned, I think it'll be a good use of some of our grant money to
>     contract or grant fund a real statistician or group of such. Maybe we
>     could get it made into a class project at a university somewhere under
>     the guidance of an experienced statistician.
> 
>     >
>     > - "watching more carefully"
>     > I'm not sure we always want the same "resolution" on network
>     traffic, and I
>     > feel it would be great to be able to zoom on suspicious activity
>     > automatically without having to carry the burden of logging everything
>     > everytime
>     >
> 
>     Another good point. Most folks these days do that with rotating
>     tcpdumps, but you're time limited there. If you don't get to that alert
>     before the pcap rotates out you've lost it. Are there better approaches
>     out there?
> 
>     Matt
> 
> 
>     --
>     --------------------------------------------
>     Matthew Jonkman
>     Emerging Threats
>     Phone 765-429-0398
>     Fax 312-264-0205
>     http://www.emergingthreats.net
>     --------------------------------------------
> 
>     PGP: http://www.jonkmans.com/mattjonkman.asc
> 
> 
>     _______________________________________________
>     Discussion mailing list
>     Discussion at openinfosecfoundation.org
>     <mailto:Discussion at openinfosecfoundation.org>
>     http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
> 
> 

-- 
--------------------------------------------
Matthew Jonkman
Emerging Threats
Phone 765-429-0398
Fax 312-264-0205
http://www.emergingthreats.net
--------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc