[Oisf-users] Help with good configuration for Suricata install with Napatech card

Wed Oct 14 10:50:39 UTC 2015

Hi Jesper,

Thank you for this.  I worked with a local Napatech engineer to use NTPL to  set up more streams (10), set up the 5-Tuple index and make sure those streams are tied to Numa 1 because of the interrupts seen in /proc/interrupts.  He told me of the 'profiling' command which would show if there are any drops per stream - which there are.  I've also been running the monitoring tool, which may be showing drops (I previously posted I had no drops looking at that tool, but I think I was viewing the stats from that tool incorrectly).

So based on your comments, seeing drops in each stream shows Suricata isn't configured appropriately to keep up with the host buffers.  I've tried different worker configurations in suricata.yaml but no luck.  Same goes for running with "autofp".  Each time I restart Suricata with a reconfigured yaml I see drops via the profiling tool in a short amount of time, and those drops grow in number.

Do you have any suggestions on a way to configure?

Thanks!!

> On Oct 14, 2015, at 6:14 AM, Jesper Lyager Nielsen <jln at napatech.com> wrote:
> 
> Hi Steve.
> 
> I think what Victor means is that with a Napatech card you are able to distribute your traffic to different hostbuffers, fx based on a 5 tuple index. You can then have a Suricata thread for each hostbuffer, and hereby distribute your load.
> 
> First thing I would check in your setup is if your ‘monitoring’ tool (default location: '/opt/napatech3/bin/monitoring’) are showing drops or errors. If that is the case Suricata is not taking packets of the Napatech hostbuffers fast enough.
> 
> Best Regards
> Jesper
> 
>> On 13 Oct 2015, at 19:59, Stephen Castellarin <castle1126 at yahoo.com> wrote:
>> 
>> Hi Victor,
>> 
>> I'm trying to understand what you meant by making "sure all packets from a flow are delivered to the same Suricata thread".
>> 
>> Right now I'm looking at the /proc/interrupts and it shows that CPU 1 is handling the interrupts for the Napatech card (based on lscpu NUMA node 1 is handling CPUs 1,3,5,7,9,11,13,15,17,19).  I've set the Napatech card to assign its HostBuffersRx to NUMA node 1.  Would it be wise to set the CPU affinity for receive, decode and stream-cpu-set to all the CPUs on NUMA node 1?  And if so, then should I assign the detect-cpu-set to the other CPUs on NUMA node 0?
>> 
>> Steve
>> 
>> 
>> 
>> On Friday, October 9, 2015 12:01 PM, Victor Julien <lists at inliniac.net> wrote:
>> 
>> 
>> On 09-10-15 17:15, Stephen Castellarin wrote:
>> > Yes there still is progress to make.  Looking at CPU utilization through
>> > SAR, for today I'm seeing an average of 88.86 %idle, so they're not
>> > being overworked.  As far as memory consumption, stats are showing I'm
>> > using roughly 50gb of 128gb available.  So I know I have plenty of
>> > breathing room from the hardware's perspective.
>> 
>> One thing to check is how the card does the traffic distributions. You
>> need to make sure all packets from a flow are delivered to the same
>> Suricata thread. IIRC napatech cards give you a lot of control there.
>> 
>> Cheers,
>> Victor
>> 
>> 
>> > To your point about the rules, I know I've stripped down a whole bunch
>> > of the ETPRO rules - only sticking with the exploit, malware, scan,
>> > trojan, current_events, web_server and web_specific_apps rules.  The
>> > largest number of rules from that list are in the trojan.rules (~9763),
>> > web_specific_apps.rules (~5603) and current_events.rules(~2505).  When I
>> > cut down to that list of rule files from the full ETPRO rule list that
>> > definitely cut out unnecessary stuff for us.  It's going to be real
>> > tough to dig through the remainder to see what is pertinent to us and
>> > what isn't.
>> > 
>> > 
>> > 
>> > On Friday, October 9, 2015 10:32 AM, Rob MacGregor
>> > <rob.macgregor at gmail.com> wrote:
>> > 
>> > 
>> > On Fri, Oct 9, 2015 at 3:05 PM Stephen Castellarin <castle1126 at yahoo.com
>> > <mailto:castle1126 at yahoo.com>> wrote:
>> 
>> > 
>> >    Sorry for the quick reply yeaterday, I forgot to hit Reply All.
>> > 
>> >    As for the tuning, I know my current, underpowered Suricata system
>> >    is missing events, as is my new hardware/configuration.  I base this
>> >    on some attack traffic I saw from one IP yesterday.  
>> > 
>> >    So our configuration is a front end router feeding an inline IPS
>> >    which then is tapped - one tap to my old Suricata system and the
>> >    second to my new Suricata system.  From a full take packet capture I
>> >    see 45 attempts to issue malicious POST attempts to a webserver we
>> >    have.  My original Suricata system triggered on 10 of those while my
>> >    new Suricata triggered on 15.  I then took the pcap I extracted and
>> >    ran it through Suricata on the new system and that system showed it
>> >    trigger on all 45.  So that's giving me a feeling that I'm not
>> >    tuning something correct - causing the running Suricata to miss things.
>> > 
>> > 
>> > So, things are improving but there's still progress to make?
>> > 
>> > I'd look at things like CPU and RAM usage - are you maxing out your
>> > CPUs/RAM?
>> > 
>> > Also, really look at those rules, are they really all relevant to your
>> > network? Also, if you strip it down to just the rules that'd catch those
>> > POST attempts, does it fire for every event?
>> > 
>> > -- 
>> >  Rob
>> 
>> > 
>> > 
>> > 
>> > 
>> > _______________________________________________
>> > Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> > Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> > List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> > Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net
>> > 
>> 
>> 
>> -- 
>> ---------------------------------------------
>> Victor Julien
>> http://www.inliniac.net/
>> PGP: http://www.inliniac.net/victorjulien.asc
>> ---------------------------------------------
>> 
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net
>> 
>> 
>> 
>> _______________________________________________
>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>> Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net
> 
> Disclaimer: This email and any files transmitted with it may contain confidential information intended for the addressee(s) only. The information is not to be surrendered or copied to unauthorized persons. If you have received this communication in error, please notify the sender immediately and delete this e-mail from your system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20151014/31545e69/attachment-0002.html>