[Oisf-users] Discrepancies in Snort and Suricata alerts

Tue Oct 30 18:45:33 UTC 2018

The box has 96GB of memory, and currently ~12% in use..

On Tue, Oct 30, 2018 at 2:39 PM Albert Whale <
albert.whale at it-security-solutions.com> wrote:

> Quick question Fatema, how much memory is in the system you are using?
>
> Sent from my iPhone-tiny keyboards introduce Interesting mistakes.
> Autodisrupt is Active.
>
> On Oct 30, 2018, at 1:23 PM, fatema bannatwala <
> fatema.bannatwala at gmail.com> wrote:
>
> Hi Everyone,
>
> So finally after a lot of time spent on the troubleshooting from  Peter,
> Michal, Victor and Travis , figured out what was the exact cause of missing
> TCP alerts in suricata was.
> It was because of the under assignment of memory for the "flow" and
> "stream" sections in suricata.yaml, which were set to default values, and
> suricata was assigning the memory at the run-time for them.
>
> *Cause:*
> It was caught while looking at the stats.log file, which was having pretty
> high values for  tcp.ssn_memcap_drop, tcp.segment_memcap_drop,
> tcp.reassembly_gap.
> That indicated that the TCP sessions were not properly tracked and
> processed due to the stream.memcap/stream.reassemble.memcap setting being
> too low.
>
> *Solution:*
> So I changed these following settings to ~10x in suricata.yaml:
>
> flow.memcap : 2048mb  # changing the default value from 128mb
> flow.hash size : 655360  # changing the default value from 65536
> stream.memcap: 1000mb  # changing the default value from 64mb
> stream.reassembly.memcap: 4000mb  # changing the default value from 256mb
>
> And the TCP alerts started getting triggered nicely!
> I also had an increased capture loss because of this, which got resolved
> by re-compiling suricata with disabled debugging and profiling options
> (Yes, I had those enabled when I built suricata before).
>
> BIG THANKS to the patience and co-operation of Peter, Michal, Victor and
> Travis in helping me out to troubleshoot the issue!
>
> Hope this helps to any future suricata newbie who is unaware of those
> memcap default settings and how it's so crucial in configuring suricata
> correctly for the given environment :)
>
> Thanks everyone for the help and responses!
>
> Fatema.
>
> On Tue, Oct 16, 2018 at 7:59 PM Michał Purzyński <
> michalpurzynski1 at gmail.com> wrote:
>
>> The number of suricata threads should be indeed equal to what you have
>> pfring configured to. Not sure if pfring requires configuration in two
>> places, though.
>>
>> Some parts of the Septun tuning, like pinning workers to cores, correct
>> memory settings and making sure you’re not allocating memory during runtime
>> would still apply.
>>
>> Ethtool, etc - depending on what pfring says.
>>
>> They basically bypassed most of the Linux network stack and it’s hard to
>> say what it is they did there.
>>
>> The only advantage I can find of pfring is that it should work on your
>> kernel version.
>>
>> On Oct 16, 2018, at 11:53 AM, fatema bannatwala <
>> fatema.bannatwala at gmail.com> wrote:
>>
>> Had a quick question regarding using pf_ring, I think I am doing
>> something wrong:
>> So in the suricata.yaml it says:
>>
>> "#Number of receive threads (>1 will enable experimental flow pinned
>> runmode)"
>>   #threads: 1
>>
>> And I am using 18 as the threads value here in pf_ring section. So should
>> I change it to 1?
>>
>> Also, I am running kernel Linux 3.10 and for h/w NIC based RSS hashing it
>> requires at least  Linux 3.14, so I can't use it unless I upgrade the
>> kernel.
>> But I think if I can set the pf_ring the correct way it should work.
>> Do I have to undo the ethtool settings mentioned in Septune to use
>> pf_ring over af_packet? I believe those settings were mentioned for using
>> af_packet and shouldn't be needed/set for pf_ring?
>>
>> Thanks,
>> Fatema
>>
>> On Tue, Oct 16, 2018 at 1:30 PM fatema bannatwala <
>> fatema.bannatwala at gmail.com> wrote:
>>
>>> I tried with pf_ring, but couldn't get the "SearchProtect" (sid:2022813)
>>> to trigger still.
>>> Next I am going to try the hardware hashing and af_packet in qm mode and
>>> see if there's any difference, will keep posted..
>>>
>>> On Fri, Oct 12, 2018 at 2:45 PM Michał Purzyński <
>>> michalpurzynski1 at gmail.com> wrote:
>>>
>>>> After talking a bit more with Fatema, we learned there might be a
>>>> problem with the kernel version on that server.
>>>>
>>>> Victor's patch shows 30% of packets sent to a wrong thread. Bro and
>>>> Snort use pf_ring.
>>>>
>>>> Suggestions would be to use hardware hashing and af_packet in QM mode.
>>>> Note - the XDP part is not needed for this setup to work.
>>>>
>>>> *https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst#setup-symmetric-hashing-on-the-nic
>>>> <https://github.com/pevma/SEPTun-Mark-II/blob/master/SEPTun-Mark-II.rst#setup-symmetric-hashing-on-the-nic>*
>>>>
>>>> just do the symmetric hashing part and then change Suricata's
>>>> configuration to use cluster_qm
>>>>
>>>>   cluster-type: cluster_qm # symmetric hashing  is a must!
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Fri, Oct 5, 2018 at 2:20 PM Michał Purzyński <
>>>> michalpurzynski1 at gmail.com> wrote:
>>>>
>>>>> How about we make Suricata write us a pcap in afpacket workers mode?
>>>>> I’m pretty sure a rule can do that.
>>>>>
>>>>> > On Oct 5, 2018, at 8:17 PM, fatema bannatwala <
>>>>> fatema.bannatwala at gmail.com> wrote:
>>>>> >
>>>>> > Changing $HOME_NET to any in sid 2022813 didn't help though, still
>>>>> not getting that alert fired.
>>>>> > One difference I had in suricata.yaml when running in offline pcap
>>>>> reading mode was, I set runmode to "single", while when suricata runs in
>>>>> packet sniffing mode it's set to "workers".
>>>>> >
>>>>> > I tried to set it to "runmode:single" while on interface sniffing
>>>>> mode but was hit by ~60% capture loss, which makes sense as single threaded
>>>>> suricata can't handle the traffic flowing through the interface.
>>>>> >
>>>>> > The fact that alerts are fired when in offline single threaded mode
>>>>> and same alerts are not fired when online packet sniffing multi-threaded
>>>>> mode, makes me think it has to do with multi-threading vs single threaded
>>>>> mode and how "workers" are capturing packets.
>>>>> >
>>>>> > I will keep looking.
>>>>> >
>>>>> > (The good thing is that Interrupt/IRQ pinning has helped to reduce
>>>>> capture loss to 0%)
>>>>> >
>>>>> > Thanks,
>>>>> > Fatema
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > Suricata IDS Users mailing list:
>>>>> oisf-users at openinfosecfoundation.org
>>>>> > Site: http://suricata-ids.org | Support:
>>>>> http://suricata-ids.org/support/
>>>>> > List:
>>>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>> >
>>>>> > Conference: https://suricon.net
>>>>> > Trainings: https://suricata-ids.org/training/
>>>>>
>>>> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
> Conference: https://suricon.net
> Trainings: https://suricata-ids.org/training/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20181030/21f04356/attachment-0001.html>