[Oisf-users] Suricata, 10k rules, 10Gbit/sec and lots of RAM

[Andreas Moe]

Copper, when you say that you are sampleing, how much? 1/10, 1/100, 1/200,
etc? And the speed, 7Gbit/s is that the network speed before the selection
of port 80 and the sampeling, or is this after all this is performed?

- In the case that this speed is before the sub-selection, what is the
actual speeds that are being analyzed on sampled port 80 traffic?
- In the case that this speed is after the sub-selection, what is the
actual speeds that are being sampeld?

Sorry for all the questions, so here is a bonus one, hehe. Have you tried
to compare timeperiods of real-time analysis results to playbacked /
re-spooled / "suricata -r" pcaps from fullcapture / tcpdump to disk, of the
same traffic?

The branch that is being talked about, is this "dev-detect-grouping-v170" ?

/AndreasM

2015-12-04 18:03 GMT+01:00 Cooper F. Nelson <cnelson at ucsd.edu>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> We are running the grouping code branch as well, ~7gbit traffic and
> sampling port 80 flows.  Using groups of 1000.
>
> Performance so far is very good, currently running 27,568 ETPRO signatures.
>
> - -Coop
>
> On 12/3/2015 4:56 PM, Michal Purzynski wrote:
> > I kind of feel responsible here and should answer this question.
> >
> > The grouping code branch will make it to Suricata post 3.0. Give. The
> new release schedule, this should be quick.
> >
> > I'm testing it on production traffic, more than 20gbit, two sensors
> (peak, but frequent, long and crazy. Average is between 3 to 6gbit/sec).
> >
> > In order to stress the code I run it with even more insane settings,
> like this
> >
> > detect-engine:
> >   - profile: custom
> >   - custom-values:
> >       toclient-src-groups: 2000
> >       toclient-dst-groups: 2000
> >       toclient-sp-groups: 2000
> >       toclient-dp-groups: 3000
> >       toserver-src-groups: 2000
> >       toserver-dst-groups: 4000
> >       toserver-sp-groups: 2000
> >       toserver-dp-groups: 2500
> >   - sgh-mpm-context: full
> >   - inspection-recursion-limit: 3000
> >   - rule-reload: true
> >
> > Note - do not try this at home. Or work. It kills kittens on 2.x
> >
> > And it just works on the new branch that's yet to be merged :)
> >
> > Note - I have over 16500 rules now.
>
>
> - --
> Cooper Nelson
> Network Security Analyst
> UCSD ACT Security Team
> cnelson at ucsd.edu x41042
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.17 (MingW32)
>
> iQEcBAEBAgAGBQJWYcddAAoJEKIFRYQsa8FWBt0H/0sh5R412AvdWkMlhTgxTI9v
> VP09We6pjr4iKzJtahKVBaeI/ilcZlUndHWbWPzJA/cD/94sXQMlm8rYsBRfEbVZ
> FnVTXHWUvglGPo0WtgklLX2a66auN4OF+shfE0wh1eP578/KYm7RERYIyelSDkHA
> H74cGHEGgW9xyPR5Kp/JxA7x1D+HO3NC0vfkOJDpvCqsdmqIbYjNIp+Iux7w7JCG
> TycUq2M/QhnNF1lFNziDiGWUMcmCBIi3ZJoMKK5/SRnsWDhdXC4hjvoulVmxZquH
> CmvNl7EFMGi9hyRZEaJIyPbbxsqxIxVueVRznKioKzad4irQAdjduUs5itLce6w=
> =Cn9D
> -----END PGP SIGNATURE-----
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 4 & 5 in Barcelona:
> http://oisfevents.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20151204/e59aff57/attachment-0002.html>