[Oisf-users] threshold.conf with rate_limit or drop rules

Sat Sep 16 16:16:54 UTC 2017

On Sat, Sep 16, 2017 at 5:07 PM, Jeff Dyke <jeff.dyke at gmail.com> wrote:
> please all, don't get me wrong, I have had wonderful success, and respect
> for the folks that contribute, on many mailing lists starting with apache,
> mysql in the 90's and contributing as possible.  I like to follow OSS
> project mailing lists that i have an active interest in, i won't list them
> here.  That is the main reason i asked if there was another avenue, b/c it
> seemed like a simple question, perhaps i underestimated, my apologies if i
> was terse.
>
> I found there was a PR merged in 2016 (current memory) which updated the
> documentation for rate_filter, I've also been looking through the source and
> while i'm not a C programmer, it looked well supported.  Also I now have
> confirmation from one of the developers, via IRC, that it is indeed
> supported other than bug 425.  Now to dig more into the code and see if more
> logging can be enabled for it, as it looks functional from the stats.log
> output, but waiting on confirmation on that.  Hopefully i'll be able to
> report back some further information.  If its enough to suggest a doc update
> i'll be sure to do that as it looks like a great feature for this product
> and understanding it fully could help folks trying to solve similar issues.

Jeff,

Can you try the following as a test  -

for example for one of the ET Open rules that you had in mind -
sid:2019876 - remove the threshold part like so:
alert ssh $EXTERNAL_NET any -> $HOME_NET any (msg:"ET SCAN SSH
BruteForce Tool with fake PUTTY version"; flow:established,to_server;
ssh.softwareversion:"PUTTY"; classtype:network-scan; sid:2019876;
rev:4; metadata:created_at 2014_12_05, updated_at 2014_12_05;)

and use the rate_filter in your threshold.com as you have originally specified:
rate_filter gen_id 1, sig_id 2019876, track by_rule, count 3, seconds
120, new_action drop, timeout 14400

and confirm if it behaves as expected?

Thank you

>
> Best,
> Jeff
>
> On Fri, Sep 15, 2017 at 7:43 PM, Chris Boley <ilgtech75 at gmail.com> wrote:
>>
>> Jeff, adding to Amar's comments; I was doing some unorthodox inline
>> scanning over a tagged vlan link in router-on-a-stick topology once upon a
>> time with hsrp configurations that traversed the transparent Linux bridge. I
>> asked about others experiences with similar scenarios and got crickets.
>> Sometimes we're pioneers. Sometimes the venture ends in frustration. Overall
>> I'm thankful for this list. I love seeing everyone's ideas.
>> Chris
>>
>> On Fri, Sep 15, 2017 at 6:27 AM amar countersnipe.com
>> <amar at countersnipe.com> wrote:
>>>
>>> Jeff! Are you saying we are not good enough on this list :-)
>>>
>>> On a serious note, I do understand your point. There are many users of
>>> Suricata, but perhaps we could do with more contribution from some of them
>>> in response to the questions that are posted here...just to ensure that this
>>> list is done some justice.
>>>
>>> The whole concept of Open Source is there so that as people use the
>>> software, try out some new features, they can share their knowledge with
>>> others. I think you might just be one of the leaders for rate_filter trials
>>> :-)
>>>
>>> I do believe that documentation is open for anyone to update.
>>>
>>> Amar
>>>
>>> On September 14, 2017 at 7:25 PM Jeff Dyke <jeff.dyke at gmail.com> wrote:
>>>
>>> Is there a better mailing list for suricata users only?  I feel like this
>>> information is not covered in the documents and is glazed over in both the
>>> rate_limit and IPS/Inline mode, sections.  and should be pretty standard in
>>> using suricata as an IPS.  It appears not.  If you're not running an
>>> interface on top of all of this, like securityonion or running in the cloud
>>> with headless servers, there are some missing portions.  I'd love to help
>>> others out, help document, but i first need to understand.
>>>
>>> as always, thank you,
>>> Jeff
>>>
>>> On Thu, Sep 14, 2017 at 4:53 PM, Jeff Dyke <jeff.dyke at gmail.com> wrote:
>>>
>>> Hi, i have started to notice one thing on restarts.  the RX-QN notices,
>>>
>>> $> grep "(RX" /var/log/suricata/suricata.log
>>> 14/9/2017 -- 16:24:49 - <Notice> - (RX-Q0) Treated: Pkts 788124, Bytes
>>> 297444677, Errors 0
>>> 14/9/2017 -- 16:24:49 - <Notice> - (RX-Q0) Verdict: Accepted 787983,
>>> Dropped 141, Replaced 0
>>> 14/9/2017 -- 16:32:29 - <Notice> - (RX-Q0) Treated: Pkts 5552, Bytes
>>> 1534474, Errors 0
>>> 14/9/2017 -- 16:32:29 - <Notice> - (RX-Q0) Verdict: Accepted 5422,
>>> Dropped 129, Replaced 0
>>> 14/9/2017 -- 16:34:18 - <Notice> - (RX-Q0) Treated: Pkts 1658, Bytes
>>> 438573, Errors 0
>>> 14/9/2017 -- 16:34:18 - <Notice> - (RX-Q0) Verdict: Accepted 1633,
>>> Dropped 24, Replaced 0
>>> 14/9/2017 -- 20:45:51 - <Notice> - (RX-Q0) Treated: Pkts 176445, Bytes
>>> 62762358, Errors 0
>>> 14/9/2017 -- 20:45:51 - <Notice> - (RX-Q0) Verdict: Accepted 176220,
>>> Dropped 223, Replaced 0
>>>
>>> Are these indeed properly dropped packets, hopefully from my rate_filter
>>> rules?  If so, i'm not seeing logs of what has been dropped to confirm i'm
>>> dropping the correct packets, how would i enabled this.  I have drop enabled
>>> in eve-ips.json, but since these are not drop rules i assume they would go
>>> elsewhere???
>>>
>>> Thanks,
>>> Jeff
>>>
>>> On Wed, Sep 13, 2017 at 6:45 PM, Jeff Dyke <jeff.dyke at gmail.com> wrote:
>>>
>>> Thanks Amar.  I could do this in sshd, but those rules are OK by me b/c i
>>> don't allow passwords, nor do i allow root login, you MUST have a valid key,
>>> so the defaults are OK, but you're correct i could reduce them to say 2 (at
>>> home, I accidentally ssh'd with the wrong/default key, then let me pass -i
>>> identity.foo for login).  To me writing the iptables rules is the same as
>>> leaving drops in OSSEC b/c it will learn from its sid's and auth.log and
>>> prevent it from happening again, and do so on my time schedule (currently
>>> 4h, 12h, 1d, 1 week).  I'm actually OK leaving them in OSSEC, but when i get
>>> a single ip trying to login and fail over a long period of time, i'd like to
>>> drop them, and my idea was to do so with suricata.
>>>
>>> While this is not my question, i'm seeing these as event_type: "ssh", not
>>> event_type: "alert", my desire is to make this learn from repeated failed
>>> attempts that are outside the bounds of the alert sids, that i have tried to
>>> override in threshold.config.  again, not my question, i can open a new
>>> thread.
>>>
>>> Appreciate the comments, i hope the response points to what i am trying
>>> to accomplish with suricata, which i've really appreciated since taking the
>>> time to learn how to do most of it correctly.   Its the drops and overrides
>>> i continue to struggle with.
>>>
>>> Jeff
>>>
>>> On Wed, Sep 13, 2017 at 6:02 PM, amar countersnipe.com
>>> <amar at countersnipe.com> wrote:
>>>
>>> Hi Jeff
>>>
>>> Just an initial thought. Most of the services can be controlled via the
>>> service daemons. For example ssh login frequency can be controlled via
>>> sshd_config using
>>>
>>> MaxAuthTries X(number you are happy with) or using MaxStartups
>>>
>>> Alternatively, since you are using iptables, an even better approach
>>> would be to use iptables with something like:
>>>
>>> iptables -A INPUT -p tcp --dport 22 -m state --state NEW -m recent
>>> --update --seconds X --hitcount Y --rttl --name SSH -j DROP
>>>
>>> Once you have eliminated the basic problem then you could get Suricata to
>>> inspect deeper.
>>>
>>> Hope I didn't just miss the requirement in the first place.
>>>
>>>
>>> regards
>>>
>>> Amar Rathore
>>> CounterSnipe - Suricata based IDS/IPS with so much more.
>>>
>>> On September 13, 2017 at 3:57 PM Jeff Dyke <jeff.dyke at gmail.com> wrote:
>>>
>>> I should have stated that i'm successfully attached to NFQUEUE in
>>> inline/IPS mode.  <Info> - NFQ running in standard ACCEPT/DROP mode.
>>>
>>> On Wed, Sep 13, 2017 at 3:53 PM, Jeff Dyke <jeff.dyke at gmail.com> wrote:
>>>
>>> i am running an array of servers on aws (EC2 instances), one server in
>>> both the staging and production environments has SSH open and 2 have 443/80
>>> open (active/passive HAProxy instances)
>>>
>>> I've been using OSSEC with active-response to block malicious ssh
>>> attacks, and while i like the software and the other things that it finds, I
>>> would like to move this type of logic to the edge servers, using suricata.
>>> i'll concentrate on SSH for now, from there i can apply my knowledge or
>>> other protocols.
>>>
>>> If i'm understanding correctly (likely not) i could add a rate_filter
>>> into threshold.conf, or i could add drop rules.  What is the best practice
>>> in this instance.  I know the threshold.config is getting parsed as i see
>>> the warning
>>> [ERRCODE: SC_ERR_EVENT_ENGINE(210)] - signature sid:2019876 has a
>>> threshold set. The signature event var is given precedence over the
>>> threshold.conf one. Bug #425.
>>>
>>> I'm running suricata 4.0.0 RELEASE
>>>
>>> Thanks, for any pointers.  If rate_filter is correct, how do i convert it
>>> to a drop event when threshold is hit?  The docs are great, but i seemed to
>>> have missed this piece.
>>>
>>> Jeff
>>>
>>> my 4 threshold.config entries.
>>> rate_filter gen_id 1, sig_id 2019876, track by_rule, count 3, seconds
>>> 120, new_action drop, timeout 14400
>>> rate_filter gen_id 1, sig_id 2101638, track by_rule, count 3, seconds
>>> 120, new_action drop, timeout 14400
>>> rate_filter gen_id 1, sig_id 2001219, track by_rule, count 3, seconds
>>> 120, new_action drop, timeout 14400
>>> rate_filter gen_id 1, sig_id 2006546, track by_rule, count 3, seconds
>>> 120, new_action drop, timeout 14400
>>> suppress gen_id 1, sig_id 2221002, track by_src, ip 10.0.0.0/16
>>>
>>>
>>>
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>> Conference: https://suricon.net
>>> Trainings: https://suricata-ids.org/training/
>>>
>>>
>>> Kind regards
>>>
>>> Amar Rathore
>>>
>>> CounterSnipe Systems LLC
>>> Tel: +1 617 701 7213
>>> Mobile: +44 (0) 7876 233333
>>> Skype ID: amarrathore
>>> Web: www.countersnipe.com <http://www.countersnipe.com/>
>>>
>>> This message contains confidential information and is intended only for
>>> the individual named. If you are not the named addressee you should not
>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>> immediately by e-mail if you have received this e-mail by mistake and delete
>>> this e-mail from your system.
>>>
>>> E-mail transmission cannot be guaranteed to be secure or error-free as
>>> information could be intercepted, corrupted, lost, destroyed, arrive late or
>>> incomplete, or contain viruses. The sender therefore does not accept
>>> liability for any errors or omissions.
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>> Conference: https://suricon.net
>>> Trainings: https://suricata-ids.org/training/
>>>
>>>
>>> Kind regards
>>>
>>> Amar Rathore
>>>
>>> CounterSnipe Systems LLC
>>> Tel: +1 617 701 7213
>>> Mobile: +44 (0) 7876 233333
>>> Skype ID: amarrathore
>>> Web: www.countersnipe.com <http://www.countersnipe.com/>
>>>
>>> This message contains confidential information and is intended only for
>>> the individual named. If you are not the named addressee you should not
>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>> immediately by e-mail if you have received this e-mail by mistake and delete
>>> this e-mail from your system.
>>>
>>> E-mail transmission cannot be guaranteed to be secure or error-free as
>>> information could be intercepted, corrupted, lost, destroyed, arrive late or
>>> incomplete, or contain viruses. The sender therefore does not accept
>>> liability for any errors or omissions.
>>>
>>> _______________________________________________
>>> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>> Conference: https://suricon.net
>>> Trainings: https://suricata-ids.org/training/
>
>
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>
> Conference: https://suricon.net
> Trainings: https://suricata-ids.org/training/

-- 
Regards,
Peter Manev