[Oisf-users] Negative in the HOME_NET variable

[Peter Manev]

On Fri, Jan 10, 2014 at 3:44 AM, Anoop Saldanha <anoopsaldanha at gmail.com> wrote:
> On Fri, Jan 10, 2014 at 2:45 AM, Michael McAndrews <michaelm14 at gmail.com> wrote:
>> Hello all,
>>
>> I'm new to the list so I apologize if this is already out there but
>> nothing popped on Google....
>>
>> I'm running Suricata and encountered an issue when modifying the
>> HOME_NET variable. I want to exclude a particular IP address from a CIDR
>> defined network. For example, in HOME_NET I have defined 192.168.0.0/16.
>> If I want to EXCLUDE the 192.168.14.0 subnet, the documentation I found
>> said it would noted as follows:
>>
>> HOME_NET:       [192.168.0.0/16,!192.168.14.0/24]
>>
>> If I DO NOT have the negative in my Suricata.yaml file, it loads in seconds:
>>
>> --------------------------------------------------------------------
>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> - 8641
>> signatures processed. 704 are IP-only rules, 3688 are inspecting packet
>> payload, 4896 inspect application layer, 0 are decoder event only
>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> -
>> building signature grouping structure, stage 1: adding signatures to
>> signature source addresses... complete
>> Jan  9 20:33:04 IDS_GW suricata: 9/1/2014 -- 20:33:04 - <Info> -
>> building signature grouping structure, stage 2: building source address
>> list... complete
>> Jan  9 20:33:11 IDS_GW suricata: 9/1/2014 -- 20:33:11 - <Info> -
>> building signature grouping structure, stage 3: building destination
>> address lists... complete
>> ---------------------------------------------------------------------
>>
>> The problem is, when I add the negative to the variable, it takes over
>> 40 minutes for Suricata to load and start inspecting traffic.
>>
>> From the logs, I can tell it hangs after stage 1. Notice the times:
>>
>> ---------------------------------------------------------------------
>> 9/1/2014 -- 20:23:29 - <Info> - building signature grouping structure,
>> stage 1: adding signatures to signature source addresses... complete
>> 9/1/2014 -- 20:53:39 - <Info> - building signature grouping structure,
>> stage 2: building source address list... complete
>> 9/1/2014 -- 21:10:09 - <Info> - building signature grouping structure,
>> stage 3: building destination address lists... complete
>> ---------------------------------------------------------------------
>>
>> Has anyone else seen this behavior or have a way to remove a subset of
>> addresses from a network?
>>
>
> I tried the above by loading a rule that uses the HOME_NET you
> specificed, and it loaded instantly.
>
> Which version of suricata are you using?
> How many rules are you loading?
>


I just run some tests.
This is a valid bug with 2.0beta2 :

TEST 1)
when using in suricata.yaml -
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12,!192.168.1.1/24]"

10/1/2014 -- 09:03:57 - <Error> - [ERRCODE:
SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp
$HOME_NET any -> $EXTERNAL_NET any (msg:"ET WORM W32/Njw0rm CnC
Beacon"; flow:established,to_server; content:"lv0njxq80"; depth:9;
content:"njxq80"; distance:0;
reference:url,www.fireeye.com/blog/technical/malware-research/2013/08/njw0rm-brother-from-the-same-mother.html;
reference:md5,4c60493b14c666c56db163203e819272;
reference:md5,b0e1d20accd9a2ed29cdacb803e4a89d;
classtype:trojan-activity; sid:2017404; rev:3;)" from file
/etc/suricata/rules/emerging-worm.rules at line 187
10/1/2014 -- 09:03:57 - <Warning> - [ERRCODE: SC_ERR_NO_RULES(42)] -
No rules loaded from /etc/suricata/rules/emerging-worm.rules
10/1/2014 -- 09:03:57 - <Info> - 48 rule files processed. 1602 rules
successfully loaded, 12443 rules failed
10/1/2014 -- 09:03:57 - <Info> - 1602 signatures processed. 0 are
IP-only rules, 249 are inspecting packet payload, 93 inspect
application layer, 72 are decoder event only
10/1/2014 -- 09:03:57 - <Info> - building signature grouping
structure, stage 1: preprocessing rules... complete
Killed

killed ... happens after 40 min.

TEST 2)
when using
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"

10/1/2014 -- 09:02:24 - <Info> - 48 rule files processed. 14045 rules
successfully loaded, 0 rules failed
10/1/2014 -- 09:02:24 - <Info> - 14053 signatures processed. 1136 are
IP-only rules, 4310 are inspecting packet payload, 10513 inspect
application layer, 72 are decoder event only
10/1/2014 -- 09:02:24 - <Info> - building signature grouping
structure, stage 1: preprocessing rules... complete
10/1/2014 -- 09:02:24 - <Info> - building signature grouping
structure, stage 2: building source address list... complete
10/1/2014 -- 09:02:27 - <Info> - building signature grouping
structure, stage 3: building destination address lists... complete

everything is fine, Suricata loads in 5 sec with 14K rules

I think there are two bugs here:
1) is the time it takes to load the rules
2) the huge amount of rules failing to load (using the same yaml for both cases)

@Michael - would you please post 2 bug reports. Please be detailed.

Thanks

-- 
Regards,
Peter Manev