[Oisf-users] Negative in the HOME_NET variable

Sat Jan 11 01:24:40 UTC 2014

On 01/10/2014 07:08 AM, Peter Manev wrote:
> On Fri, Jan 10, 2014 at 3:00 PM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>> On Fri, Jan 10, 2014 at 2:38 PM, Peter Manev <petermanev at gmail.com> wrote:
>>> On Fri, Jan 10, 2014 at 3:44 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
>>>> On Fri, Jan 10, 2014 at 2:45 AM, Michael McAndrews <michaelm14 at gmail.com> wrote:
>>>>> Hello all,
>>>>>
>>>>> I'm new to the list so I apologize if this is already out there but
>>>>> nothing popped on Google....
>>>>>
>>>>> I'm running Suricata and encountered an issue when modifying the
>>>>> HOME_NET variable. I want to exclude a particular IP address from a CIDR
>>>>> defined network. For example, in HOME_NET I have defined 192.168.0.0/16.
>>>>> If I want to EXCLUDE the 192.168.14.0 subnet, the documentation I found
>>>>> said it would noted as follows:
>>>>>
>>>>> HOME_NET:       [192.168.0.0/16,!192.168.14.0/24]
>>>>>
>>>>> If I DO NOT have the negative in my Suricata.yaml file, it loads in seconds:
>>>>>
>>>>> --------------------------------------------------------------------
>>>>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> - 8641
>>>>> signatures processed. 704 are IP-only rules, 3688 are inspecting packet
>>>>> payload, 4896 inspect application layer, 0 are decoder event only
>>>>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> -
>>>>> building signature grouping structure, stage 1: adding signatures to
>>>>> signature source addresses... complete
>>>>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> -
>>>>> building signature grouping structure, stage 2: building source address
>>>>> list... complete
>>>>> Jan  9 20:33:11 IDS_GW suricata: 9/1/2014 -- 20:33:11 - <Info> -
>>>>> building signature grouping structure, stage 3: building destination
>>>>> address lists... complete
>>>>> ---------------------------------------------------------------------
>>>>>
>>>>> The problem is, when I add the negative to the variable, it takes over
>>>>> 40 minutes for Suricata to load and start inspecting traffic.
>>>>>
>>>>> From the logs, I can tell it hangs after stage 1. Notice the times:
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> 9/1/2014 -- 20:23:29 - <Info> - building signature grouping structure,
>>>>> stage 1: adding signatures to signature source addresses... complete
>>>>> 9/1/2014 -- 20:53:39 - <Info> - building signature grouping structure,
>>>>> stage 2: building source address list... complete
>>>>> 9/1/2014 -- 21:10:09 - <Info> - building signature grouping structure,
>>>>> stage 3: building destination address lists... complete
>>>>> ---------------------------------------------------------------------
>>>>>
>>>>> Has anyone else seen this behavior or have a way to remove a subset of
>>>>> addresses from a network?
>>>>>
>>>> I tried the above by loading a rule that uses the HOME_NET you
>>>> specificed, and it loaded instantly.
>>>>
>>>> Which version of suricata are you using?
>>>> How many rules are you loading?
>>>>
>>>
>>> I just run some tests.
>>> This is a valid bug with 2.0beta2 :
>>>
>>> TEST 1)
>>> when using in suricata.yaml -
>>> HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,!192.168.1.1/24]"
>>>
>>> 10/1/2014 -- 09:03:57 - <Error> - [ERRCODE:
>>> SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp
>>> $HOME_NET any -> $EXTERNAL_NET any (msg:"ET WORM W32/Njw0rm CnC
>>> Beacon"; flow:established,to_server; content:"lv0njxq80"; depth:9;
>>> content:"njxq80"; distance:0;
>>> reference:url,www.fireeye.com/blog/technical/malware-research/2013/08/njw0rm-brother-from-the-same-mother.html;
>>> reference:md5,4c60493b14c666c56db163203e819272;
>>> reference:md5,b0e1d20accd9a2ed29cdacb803e4a89d;
>>> classtype:trojan-activity; sid:2017404; rev:3;)" from file
>>> /etc/suricata/rules/emerging-worm.rules at line 187
>>> 10/1/2014 -- 09:03:57 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] -
>>> No rules loaded from /etc/suricata/rules/emerging-worm.rules
>>> 10/1/2014 -- 09:03:57 - <Info> - 48 rule files processed. 1602 rules
>>> successfully loaded, 12443 rules failed
>>> 10/1/2014 -- 09:03:57 - <Info> - 1602 signatures processed. 0 are
>>> IP-only rules, 249 are inspecting packet payload, 93 inspect
>>> application layer, 72 are decoder event only
>>> 10/1/2014 -- 09:03:57 - <Info> - building signature grouping
>>> structure, stage 1: preprocessing rules... complete
>>> Killed
>>>
>>> killed ... happens after 40 min.
>>>
>>> TEST 2)
>>> when using
>>> HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
>>>
>>> 10/1/2014 -- 09:02:24 - <Info> - 48 rule files processed. 14045 rules
>>> successfully loaded, 0 rules failed
>>> 10/1/2014 -- 09:02:24 - <Info> - 14053 signatures processed. 1136 are
>>> IP-only rules, 4310 are inspecting packet payload, 10513 inspect
>>> application layer, 72 are decoder event only
>>> 10/1/2014 -- 09:02:24 - <Info> - building signature grouping
>>> structure, stage 1: preprocessing rules... complete
>>> 10/1/2014 -- 09:02:24 - <Info> - building signature grouping
>>> structure, stage 2: building source address list... complete
>>> 10/1/2014 -- 09:02:27 - <Info> - building signature grouping
>>> structure, stage 3: building destination address lists... complete
>>>
>>> everything is fine, Suricata loads in 5 sec with 14K rules
>>>
>>> I think there are two bugs here:
>>> 1) is the time it takes to load the rules
>>> 2) the huge amount of rules failing to load (using the same yaml for both cases)
>>>
>>> @Michael - would you please post 2 bug reports. Please be detailed.
>>>
>> @Peter,
>>
>> So the delay surfaces if we are loading a huge no of rules with the
>> specified HOME_NET?
>>
> yes.
> I tried loading different rulefiles aswell. But it only occures when a
> lot of rules are loaded.
>
>
Thank you all for testing and confirming. I also have 8900+ rules. I'll
file the appropriate bug reports.

Cheers!