[Oisf-users] Help with good configuration for Suricata install with Napatech card

Fri Oct 9 19:47:49 UTC 2015

Victor (or any one else for that matter), do you have any guides or
examples of how to set "best-practice" traffic distributions with regards
to Suricata processing? I have seen the same as described in this thread:
Less output on live network analysis VS output playback of the same traffic.

Scenario:
- A collection of VLANs aggreagted to a SPAN session, sent on two
interfaces (each interface is monitored by different machines)
- Interface 1 is Suricata (only running IDS) and tcpdump to capture the data
- Interface 2 is tcpdump.
- pcaps from VM/Interface 1 and VM/Interface 2 are compared and found
identical.
- Lots of RAM and CPU to go on, Tested on 10GBit/s and 1Gbit/s cards, with
different drivers. Tested traffic was only at between 100 and 600Mbit/s.
- Found alerts of VM/Interface 1 (live listening mode) is compared to
Suricata replay-pcap-mode of pcaps from VM/Interface 1 and VM/Interface 2
- Results indicate lower detection rate on the live running mode.

So, this traffic distribution / making sure that the same flow is sent to
the correct thread, any pointers on that? cluster_cpu, cluster_flow,
ethtool settings, os settings (linux/RHEL), etc?
I know its not a dirrectly associated sceniario, since the OP of the thread
was with regards to Napatech cards, which my exaplained scenario was not
utilizing.

TL;DR What i found nice to do, was to check the expected output (based on
suricata -r mode) vs live capture mode, and comparing outputs, and not just
blindly looking at the stats counters saying that there is no drop (either
kernel, memcap, or other).

2015-10-09 18:01 GMT+02:00 Victor Julien <lists at inliniac.net>:

> On 09-10-15 17:15, Stephen Castellarin wrote:
> > Yes there still is progress to make.  Looking at CPU utilization through
> > SAR, for today I'm seeing an average of 88.86 %idle, so they're not
> > being overworked.  As far as memory consumption, stats are showing I'm
> > using roughly 50gb of 128gb available.  So I know I have plenty of
> > breathing room from the hardware's perspective.
>
> One thing to check is how the card does the traffic distributions. You
> need to make sure all packets from a flow are delivered to the same
> Suricata thread. IIRC napatech cards give you a lot of control there.
>
> Cheers,
> Victor
>
>
> > To your point about the rules, I know I've stripped down a whole bunch
> > of the ETPRO rules - only sticking with the exploit, malware, scan,
> > trojan, current_events, web_server and web_specific_apps rules.  The
> > largest number of rules from that list are in the trojan.rules (~9763),
> > web_specific_apps.rules (~5603) and current_events.rules(~2505).  When I
> > cut down to that list of rule files from the full ETPRO rule list that
> > definitely cut out unnecessary stuff for us.  It's going to be real
> > tough to dig through the remainder to see what is pertinent to us and
> > what isn't.
> >
> >
> >
> > On Friday, October 9, 2015 10:32 AM, Rob MacGregor
> > <rob.macgregor at gmail.com> wrote:
> >
> >
> > On Fri, Oct 9, 2015 at 3:05 PM Stephen Castellarin <castle1126 at yahoo.com
> > <mailto:castle1126 at yahoo.com>> wrote:
> >
> >     Sorry for the quick reply yeaterday, I forgot to hit Reply All.
> >
> >     As for the tuning, I know my current, underpowered Suricata system
> >     is missing events, as is my new hardware/configuration.  I base this
> >     on some attack traffic I saw from one IP yesterday.
> >
> >     So our configuration is a front end router feeding an inline IPS
> >     which then is tapped - one tap to my old Suricata system and the
> >     second to my new Suricata system.  From a full take packet capture I
> >     see 45 attempts to issue malicious POST attempts to a webserver we
> >     have.  My original Suricata system triggered on 10 of those while my
> >     new Suricata triggered on 15.  I then took the pcap I extracted and
> >     ran it through Suricata on the new system and that system showed it
> >     trigger on all 45.  So that's giving me a feeling that I'm not
> >     tuning something correct - causing the running Suricata to miss
> things.
> >
> >
> > So, things are improving but there's still progress to make?
> >
> > I'd look at things like CPU and RAM usage - are you maxing out your
> > CPUs/RAM?
> >
> > Also, really look at those rules, are they really all relevant to your
> > network? Also, if you strip it down to just the rules that'd catch those
> > POST attempts, does it fire for every event?
> >
> > --
> >  Rob
> >
> >
> >
> >
> > _______________________________________________
> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> > Site: http://suricata-ids.org | Support:
> http://suricata-ids.org/support/
> > List:
> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> > Suricata User Conference November 4 & 5 in Barcelona:
> http://oisfevents.net
> >
>
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 4 & 5 in Barcelona:
> http://oisfevents.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20151009/28684726/attachment-0002.html>