[Oisf-users] Negative in the HOME_NET variable

[Peter Manev]

On Fri, Jan 10, 2014 at 3:00 PM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
> On Fri, Jan 10, 2014 at 2:38 PM, Peter Manev <petermanev at gmail.com> wrote:
>> On Fri, Jan 10, 2014 at 3:44 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>> On Fri, Jan 10, 2014 at 2:45 AM, Michael McAndrews <michaelm14 at gmail.com> wrote:
>>>> Hello all,
>>>>
>>>> I'm new to the list so I apologize if this is already out there but
>>>> nothing popped on Google....
>>>>
>>>> I'm running Suricata and encountered an issue when modifying the
>>>> HOME_NET variable. I want to exclude a particular IP address from a CIDR
>>>> defined network. For example, in HOME_NET I have defined 192.168.0.0/16.
>>>> If I want to EXCLUDE the 192.168.14.0 subnet, the documentation I found
>>>> said it would noted as follows:
>>>>
>>>> HOME_NET:       [192.168.0.0/16,!192.168.14.0/24]
>>>>
>>>> If I DO NOT have the negative in my Suricata.yaml file, it loads in seconds:
>>>>
>>>> --------------------------------------------------------------------
>>>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> - 8641
>>>> signatures processed. 704 are IP-only rules, 3688 are inspecting packet
>>>> payload, 4896 inspect application layer, 0 are decoder event only
>>>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> -
>>>> building signature grouping structure, stage 1: adding signatures to
>>>> signature source addresses... complete
>>>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> -
>>>> building signature grouping structure, stage 2: building source address
>>>> list... complete
>>>> Jan  9 20:33:11 IDS_GW suricata: 9/1/2014 -- 20:33:11 - <Info> -
>>>> building signature grouping structure, stage 3: building destination
>>>> address lists... complete
>>>> ---------------------------------------------------------------------
>>>>
>>>> The problem is, when I add the negative to the variable, it takes over
>>>> 40 minutes for Suricata to load and start inspecting traffic.
>>>>
>>>> From the logs, I can tell it hangs after stage 1. Notice the times:
>>>>
>>>> ---------------------------------------------------------------------
>>>> 9/1/2014 -- 20:23:29 - <Info> - building signature grouping structure,
>>>> stage 1: adding signatures to signature source addresses... complete
>>>> 9/1/2014 -- 20:53:39 - <Info> - building signature grouping structure,
>>>> stage 2: building source address list... complete
>>>> 9/1/2014 -- 21:10:09 - <Info> - building signature grouping structure,
>>>> stage 3: building destination address lists... complete
>>>> ---------------------------------------------------------------------
>>>>
>>>> Has anyone else seen this behavior or have a way to remove a subset of
>>>> addresses from a network?
>>>>
>>>
>>> I tried the above by loading a rule that uses the HOME_NET you
>>> specificed, and it loaded instantly.
>>>
>>> Which version of suricata are you using?
>>> How many rules are you loading?
>>>
>>
>>
>> I just run some tests.
>> This is a valid bug with 2.0beta2 :
>>
>> TEST 1)
>> when using in suricata.yaml -
>> HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,!192.168.1.1/24]"
>>
>> 10/1/2014 -- 09:03:57 - <Error> - [ERRCODE:
>> SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp
>> $HOME_NET any -> $EXTERNAL_NET any (msg:"ET WORM W32/Njw0rm CnC
>> Beacon"; flow:established,to_server; content:"lv0njxq80"; depth:9;
>> content:"njxq80"; distance:0;
>> reference:url,www.fireeye.com/blog/technical/malware-research/2013/08/njw0rm-brother-from-the-same-mother.html;
>> reference:md5,4c60493b14c666c56db163203e819272;
>> reference:md5,b0e1d20accd9a2ed29cdacb803e4a89d;
>> classtype:trojan-activity; sid:2017404; rev:3;)" from file
>> /etc/suricata/rules/emerging-worm.rules at line 187
>> 10/1/2014 -- 09:03:57 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] -
>> No rules loaded from /etc/suricata/rules/emerging-worm.rules
>> 10/1/2014 -- 09:03:57 - <Info> - 48 rule files processed. 1602 rules
>> successfully loaded, 12443 rules failed
>> 10/1/2014 -- 09:03:57 - <Info> - 1602 signatures processed. 0 are
>> IP-only rules, 249 are inspecting packet payload, 93 inspect
>> application layer, 72 are decoder event only
>> 10/1/2014 -- 09:03:57 - <Info> - building signature grouping
>> structure, stage 1: preprocessing rules... complete
>> Killed
>>
>> killed ... happens after 40 min.
>>
>> TEST 2)
>> when using
>> HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
>>
>> 10/1/2014 -- 09:02:24 - <Info> - 48 rule files processed. 14045 rules
>> successfully loaded, 0 rules failed
>> 10/1/2014 -- 09:02:24 - <Info> - 14053 signatures processed. 1136 are
>> IP-only rules, 4310 are inspecting packet payload, 10513 inspect
>> application layer, 72 are decoder event only
>> 10/1/2014 -- 09:02:24 - <Info> - building signature grouping
>> structure, stage 1: preprocessing rules... complete
>> 10/1/2014 -- 09:02:24 - <Info> - building signature grouping
>> structure, stage 2: building source address list... complete
>> 10/1/2014 -- 09:02:27 - <Info> - building signature grouping
>> structure, stage 3: building destination address lists... complete
>>
>> everything is fine, Suricata loads in 5 sec with 14K rules
>>
>> I think there are two bugs here:
>> 1) is the time it takes to load the rules
>> 2) the huge amount of rules failing to load (using the same yaml for both cases)
>>
>> @Michael - would you please post 2 bug reports. Please be detailed.
>>
>
> @Peter,
>
> So the delay surfaces if we are loading a huge no of rules with the
> specified HOME_NET?
>

yes.
I tried loading different rulefiles aswell. But it only occures when a
lot of rules are loaded.


-- 
Regards,
Peter Manev