[Discussion] Suricata rule not giving alerts

[Jelte]

Anoop Saldanha schreef op 7/22/2014 1:16 PM:
> On Tue, Jul 22, 2014 at 3:01 PM, Jelte <masterjel5000 at hotmail.com> wrote:
>> Anoop Saldanha schreef op 7/22/2014 4:09 AM:
>>> On Tue, Jul 22, 2014 at 3:51 AM, Jelte <masterjel5000 at hotmail.com> wrote:
>>>> Anoop Saldanha schreef op 7/21/2014 4:32 AM:
>>>>> On Sun, Jul 20, 2014 at 5:50 PM, Jelte <masterjel5000 at hotmail.com> wrote:
>>>>>> Jelte schreef op 7/15/2014 12:08 AM:
>>>>>>> Victor Julien schreef op 7/14/2014 9:27 AM:
>>>>>>>> On 07/14/2014 12:21 AM, Jelte O. wrote:
>>>>>>>>> Hello all,
>>>>>>>>>
>>>>>>>>> I have a rule from the ET rule-set to alert against an attack that is
>>>>>>>>> used to exploit a vulnerability in nginx 1.3.9-1.4.0. In order to
>>>>>>>>> trigger this rule I loaded an exploit module in Metasploit and fired it
>>>>>>>>> on my server.
>>>>>>>>>
>>>>>>>>> The vulnerability:
>>>>>>>>> https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-2028
>>>>>>>>> The rule: http://doc.emergingthreats.net/bin/view/Main/2016918
>>>>>>>>> The Metasploit module:
>>>>>>>>> https://github.com/rapid7/metasploit-framework/blob/master/modules/exploits/linux/http/nginx_chunked_size.rb
>>>>>>>>>
>>>>>>>>> I'll repeat the rule here:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>     alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET
>>>>>>>>>     WEB_SERVER Possible NGINX Overflow CVE-2013-2028 Exploit Specific";
>>>>>>>>>     flow:established,to_server; content:"chunked"; http_header; nocase;
>>>>>>>>>     fast_pattern:only; pcre:"/Transfer-Encoding\x3a[^\r\n]*?chunked/Hi";
>>>>>>>>>     pcre:"/^[\r\n\s]*?[^\r\n]+HTTP\/1\.\d[^\r\n]*?\r?\n((?!(\r?\n\r?\n)).)*?Transfer-Encoding\x3a[^\r\n]*?Chunked((?!(\r?\n\r?\n)).)*?\r?\n\r?\n[\r\n\s]*?(f{6}[8-9a-f][0-9a-f]|[a-f0-9]{9})/si";
>>>>>>>>>     reference:url,www.vnsecurity.net/2013/05/analysis-of-nginx-cve-2013-2028/;
>>>>>>>>>     reference:url,github.com/rapid7/metasploit-framework/blob/master/modules/exploits/linux/http/nginx_chunked_size.rb;
>>>>>>>>>     classtype:attempted-admin; sid:2016918; rev:6;)
>>>>>>>>>
>>>>>>>>> My attack did not generate any alerts. However, as soon as I removed the
>>>>>>>>> "http_header;" and changed "/Hi" to "/i" (in the first pcre) the rule
>>>>>>>>> started generating alerts. From this it seems like the HTTP header is
>>>>>>>>> not complete/not recognized by Suricata. However, when I do an extended
>>>>>>>>> logging on the HTTP traffic, I do see entries like:
>>>>>>>>>
>>>>>>>>> 07/13/14-23:45:27.830342 - - Chunked HTTP/1.1 GET mifpudtilvpjqsjl / - 0
>>>>>>>>> x.x.x.x:40590 -> y.y.y.y:80
>>>>>>>>>
>>>>>>>>> My "customformat" for the http-log contains "%{Transfer-Encoding}i",
>>>>>>>>> which would actually be the "contents of the defined HTTP Request Header
>>>>>>>>> name" according to the documentation (refer to
>>>>>>>>> https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Custom_http_logging).
>>>>>>>>> <https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Custom_http_logging%29.>
>>>>>>>>>
>>>>>>>>> I also created packet dumps of both legitimate web traffic and this
>>>>>>>>> attack and analyzed the streams in Wireshark. In both dumps there are
>>>>>>>>> TCP PDU's which are re-assembled, but in the valid web traffic Wireshark
>>>>>>>>> labels the protocol for some of the fully assembled client-to-server
>>>>>>>>> packets as HTTP while for the attack there are only TCP packets from the
>>>>>>>>> client to the server.
>>>>>>>>>
>>>>>>>>> I am wondering why the HTTP header is not available. I am not sure if
>>>>>>>>> this is caused by Suricata, my OS/network interface or the rule itself.
>>>>>>>>> I hope someone can help me out!
>>>>>>>> As a first step, I'd suggest walking through this page
>>>>>>>> https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Self_Help_Diagrams
>>>>>>>>
>>>>>>> Thanks Will and Victor for your replies! I didn't know of the Self Help
>>>>>>> Diagrams yet, unfortunately, because it could have saved me some time on
>>>>>>> an earlier issue I had with bad TCP checksums. Anyway, I have traffic in
>>>>>>> the stats log and the http log, I already disabled checksum validation
>>>>>>> and the startup log says the rule has successfully been loaded (for
>>>>>>> testing purposes I have only included this single rule). Also, my server
>>>>>>> is not part of a VLAN.
>>>>>>>
>>>>>>> Offloading is enabled for my NIC. If I issue the "ethtool --show-offload
>>>>>>> eth0" command I see that the rx-checksumming, the tx-checksumming,
>>>>>>> tcp-segmentation-offload and the generic-segmentation-offload are on.
>>>>>>> The NIC does not seem to support turning off the rx-checksumming (I get
>>>>>>> "Operation not supported") but I was able to turn off the others. This
>>>>>>> didn't have any effect, though. I also created another packet dump of
>>>>>>> the attack after disabling these settings and compared this one to the
>>>>>>> one I created before turning off the tx-checksumming and segmentation
>>>>>>> offloading and they both matched. I already disabled the checksum
>>>>>>> validation in Suricata to get rid off the invalid checksum errors I had
>>>>>>> in the beginning.
>>>>>>>
>>>>>>> I went to all the self help diagrams but I still couldn't find the
>>>>>>> cause. I did notice that I forgot to mention two important things:
>>>>>>>
>>>>>>> - Snort (on the same system) does generate an alert for that particular
>>>>>>> rule.
>>>>>>> - Suricata does generate alerts for other rules in which I filter on
>>>>>>> content from the http header. For instance, a HTTP request inside
>>>>>>> Firefox generates alerts for a rule that includes 'content:"Mozilla";
>>>>>>> http_header; nocase;'.
>>>>>>>
>>>>>>> It seems like it has to do with the specific exploit that is used in
>>>>>>> Metasploit. Refer to
>>>>>>> https://github.com/rapid7/metasploit-framework/blob/master/modules/exploits/linux/http/nginx_chunked_size.rb
>>>>>>>
>>>>>>> Do you know anything else I can do for debugging? I have included the
>>>>>>> pcap of the attack in the attachment.
>>>>>>>
>>>>> Does changing the "http_header" to "http_raw_header" and 'H' to 'D'
>>>>> make any difference?
>>>>>
>>>> Thanks, but I already tried that and it didn't work. I double checked,
>>>> just to be sure, but even without the pcre's (just the
>>>> flow:established,to_server; content:"chunked"; http_raw_header; nocase;)
>>>> I get no alerts.
>>>>
>>>> Maybe someone is able to reproduce the issue locally? just fire the
>>>> exploit via Metasploit and make sure that you have the rule enabled.
>>>>
>>>> I'm eager to find the solution, so any help will be greatly appreciated!
>>>>
>>> It would be helpful if you could supply the pcap for this?
>>>
>> I already attached it in a previous email (7/15/2014), but here is it
>> again :-)
> Can you test if this patch works for you? -
>
> https://github.com/ironbee/libhtp/pull/78
>

You saved my day, it works!! Thanks so much :-)

If I understand correctly, it was not working before because the code
that is specific for handling chunked transfer encoding packages was
(incorrectly) not executed because the precondition was not satisfied,
the precondition being that the value of the transfer-encoding field
equals "chunked". This was not the case because the value actually was
"Chunked" (with a capital C) and therefore you have changed the
precondition to also include lower-case values of "chunked".

Thx again for your time.

Jelte.