[Oisf-users] Understanding Suricata's "xbit" implementation.

[Victor Julien]

On 24-02-17 21:16, Champ Clark III wrote:
> Over the past year I have become quite the Suricata fan.  So much so
> that I ended up in Washington DC to attend Suricon 2016.  It was a great
> experience and I plan on attending Suricon 2017!

Great! See you in Prague :)

> My question has to do with how Suricata handles "xbits".  I work on a
> project that uses "Snort like" rules for log analysis called "Sagan"
> (http://sagan.io). It is multi-threaded and also supports the concept of
> "xbits".  My goal is to align how Sagan works with "xbits" with how
> Suricata works with "xbits".  After a little bit of research,  I've
> found some Suricata examples of "xbits" usage. For example:
> 
> xbits:set,EXAMPLEBIT, track ip_dst, expire 60;
> xbits:isset, EXAMPLEBIT, track ip_dst;
> 
> 1) My question is, when the xbit is "set",  is the "track ip_dst"
> needed?  In Sagan,  this is what I would do:
> 
> xbits: set,EXAMPLEBIT,60;
> xbits: isset,by_src,EXAMPLEBIT;
> 
> When we "set" an xbit,  both the source and destination are stored in
> memory.   This way when we use a "isset", "isnotset" or "unset",  we can
> do so by "by_src", "by_dst", "both" or "reverse" (Suricata's
> "toggle"?).  This means that Sagan really only uses the src/dst on
> "isset", "isnotset", "unset" xbits and not with "set".

We currently support: ip_src, ip_dst and ip_pair:

ip_src and ip_dst are wrappers around our 'hostbits', which stores per
host (host == ipaddress here). ip_src means we store the bit for the src
ip of the matching packet. We don't have a 'both' yet.

ip_pair means we store it in a different place, the ip pair table. This
is meant to store bits per 2 talkers.

> 2) Is it possible to use multiple "xbits" with isset/isnotset/etc.  In
> Sagan,  this is what I would do:
> 
> 
> xbits: isset,by_src,EXAMPLEBIT1|EXAMPLEBIT2|EXAMPLEBIT3;
> 
> 
> This would mean, "if the source (by_src) has EXAMPLEBIT1 or EXAMPLEBIT2
> or EXAMPLEBIT3 isset".   You could also do:
> 
> 
> xbits: isset, by_src, EXAMPLEBIT1&EXAMPLEBIT2&EXAMPLEBIT3;
> 
> 
> This would mean, "if the source (by_src) has EXAMPLEBIT1 and EXAMPLEBIT2
> and EXAMPLEBIT3 isset".


We don't use this syntax, but instead you'd have multiple instances of
the keyword in a rule. Your syntax suggestion would make sense to
implement though.

> 
> Hopefully my questions are clear.
> 
> We've found the "xbit" concept to be insanely powerful.  One of my goals
> this year is to store "xbit" data from both Sagan and Suricata into a
> Redis database.   This way,  xbit data could be shared globally from a
> log and packet analysis level.
> 

In our git master I've added a 'vars' log. Currently it only does 'flow'
vars and 'pkt' vars, not host and ippair yet. But it's basically an EVE
log for these things. It can be configured to go into redis like all EVE
outputs.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------