[Oisf-users] Detection Algorithms

[Cooper F. Nelson]

Yeah that's kind of the problem.  Most people are using Intel's
Hyperscan library with suricata these days, which is a high-performance
SIMD multi-pattern matcher.  This really isn't compatible with a ML
approach.

I've been working on a 'fast fuzzy hashing' algorithm to try and find a
happy medium between regex and ML approaches, but I'm running into the
same problem you have.  There isn't a public shared repo of packet
captures available for training.  Funny enough, the best source would be
from VirusTotal, which was acquired by Google in 2012.  So asking the
TensorFlow (also Google) people might help as well.

I was thinking whether or not you could use an IDS testing tool (e.g.
sneeze or snot (yes that is what they are called)) to create traffic
from suricata sigs, but that really wouldn't be a good approach for
machine learning.  You really need to train it against 'live' packet
captures to get the effect you are looking for. 

I'm also concerned that in the modern era there would be tons of
false-positives, as HTTP CnC is often designed to look as much like
normal http as possible to evade detection.  In this case, a fixed regex
is almost certainly a better approach vs. ML.

-Coop

On 10/20/2017 3:25 PM, Bat Finkler wrote:
> Thanks Cooper
>
> That's a shame I was looking to compare a few ML algorithms to see
> which detects a particular attack vector the best. I was hoping that I
> could drop a couple into suricata to work with live data, rather than
> using a dataset and something like weka.
>
> Any advice How I might be able to do the above to allow me to work on
> live data ?
>
> Cheers
>

-- 
Cooper Nelson
Network Security Analyst
UCSD ITS Security Team
cnelson at ucsd.edu x41042


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 488 bytes
Desc: OpenPGP digital signature
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20171023/c3163a29/attachment-0002.sig>