[Discussion] Virtues of a Multi-Sensor Environment

[Seth Hall]

On Oct 29, 2008, at 2:13 PM, David J. Bianco wrote:

> There was some discussion on this list about Bro.  I'm a big fan of  
> Bro.
> It performs very well in the pattern analysis and policy-based  
> detection
> arenas, in which Snort (and thus, Sguil) do poorly.  Conversely, Bro's
> signature matching abilities aren't very good, and it certainly  
> doesn't have
> a robust sig library like Snort does.

Bro's signature matching is only poor because it isn't used much by  
anyone that uses Bro heavily.  I don't believe that it would be too  
much of a project to flesh out the signature matching capabilities of  
Bro to mirror Snort.  One area where Bro's signature matching already  
bests Snort is in its ability to have bidirectional signatures.  You  
can define a signature that only triggers if another signature is seen  
in the opposite direction of the connection.  For example, if you're  
matching something that requests a certain URL, maybe you only care  
about it the server responds to the client with a 200 status code.   
This is easy to do with bidirectional signatures.  You can also have  
signature matches thrown into Bro policy script code so that you can  
do any number of extended heuristics based on other information Bro  
has detected.

> My view is that they are both complementary technologies, and should  
> be
> deployed in tandem.  It has been suggested on this list that Bro be  
> the
> engine for future OISF implementation.  Great idea!  But I propose  
> that
> we consider our project a framework, or suite of tools, and that Bro  
> be
> a critical *piece* of that framework, but  not the only framework.

In my opinion, Bro would actually make a very general framework to  
build upon.  Bro is already more or less a general network analysis  
framework.  It has support for lots of various technologies that  
people want in order to do network traffic analysis and typically it's  
fairly easy to add new functionality building on top of existing  
functionality.  As an example of this, yesterday I wrote some small  
extensions that will identify windows executables being transferred  
over HTTP with libmagic and once a windows executable is identified it  
will begin building an MD5 hash of the data.  It then takes the MD5  
hash and checks it with the newly announced Team Cymru Malware Hash  
Repository[1] in realtime to see if they have that binary file in  
their malware repository.  If you would like to see how little code I  
had to write to make it work, here's the link to my changes:
   http://github.com/sethhall/bro_scripts/tree/master/md5_hash_malware

Malware Hash Registry docs:
   http://www.team-cymru.org/Services/MHR/

My point is just that Bro is very much in a position to be a central  
framework.  Much more so than any other tools currently available.

> I can envision a scenario where I'd deploy, for example, signature  
> based
> IDS (Snort), policy based IDS (Bro), correlated event alerts (OSSEC)  
> and
> perhaps even application specific sensors (mod_security, database IDS,
> spear phishing detection, etc).

The problem is that you're replicating effort there.  You'd have to  
maintain multiple machines just to run different tools (assuming  
you're looking at too much traffic to analyze on a single machine with  
multiple tools).  As I mentioned above, the problems with Bro's  
signature support are not due to a major failing in the software it's  
just temporarily suffering due to a lack of interest from the Bro  
community. :)

It probably makes sense to feed the data from other tools like OSSEC  
into Bro first too because it has very good support for writing  
scripts that deal with data expiration over time and catching time  
based events.  We use it to look for anomalies and normalize our  
authentication logs by pushing a lot of them into Bro with the  
Broccoli library (it's included with Bro).  I know of another location  
that was using instrumented sshd servers to forward the content of all  
shell sessions to their Bro instance and looking for anomalies with  
the commands being typed over ssh to catch compromised accounts.

After Bro has a chance to look at the data, it can then pass  
normalized and aggregated data (and maybe more notices if it caught  
anything strange in the data) into Squil and its database.

> All these events should go to the same
> analysis tool, they should be organized effectively, searchable,
> cross-referenced, and tied to forensic data (network sessions, packet
> captures, log files, etc).

I definitely agree with this!  Bro doesn't have anything yet that is  
end user focused towards doing forensics with the data being  
generated.  If Squil could give it that frontend, that would be awesome.

> In fact, we're much of the way there already.  I released an agent to
> slurp OSSEC events into Sguil quite some time ago, and we have done  
> the
> same with other data sources as well.  Communication between Bro and  
> Sguil
> is on my TODO list as well.  So really, the idea of deploying multiple
> types of sensors feeding into the same analysis tool is a practical  
> one.
> I think this should be a design goal of the project.

If you'd like assistance with integrating Squil and Bro, I'd be glad  
to give some pointers or lend a hand with it. :)

   .Seth

---
Seth Hall
Network Security - Office of the CIO
The Ohio State University
Phone: 614-292-9721