[Oisf-devel] file extraction -- Re: [COMMIT] OISF branch, master, updated. a556338936ad3cd2b0379a6985fb62084368d99e

[Victor Julien]

On 11/29/2011 06:39 PM, Chris Wakelin wrote:
> On 29/11/11 17:28, Victor Julien wrote:
>> On 11/29/2011 06:20 PM, Chris Wakelin wrote:
>>>> Finally there is the filestore keyword. It is the simplest of all: if
>>>> the rule matches, the files will be written to disk.
>>>
>>> I've been trying this with a few pcaps. However, I've found if I've
>>> added "filestore" to some rules, it seems to write files even when they
>>> don't match.
>>
>> What rules have you modified for it?
> 
> The "Scalaxy" ones I submitted to ET (well actually slightly newer
> versions):
> 
> alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"RDG Probable Scalaxy
> exploit kit secondary request"; flow:established,to_server; content:"/";
> http_uri; offset:2; depth:3; content:"?pdf="; http_uri; distance:32;
> within:37; fast_pattern; pcre:"/\/[a-z]\/[0-9a-f]{32}\?pdf=/U";
> filestore; classtype:bad-unknown; sid:378000130; rev:3;)
> 
> alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"RDG Probable Scalaxy
> exploit kit Java or PDF exploit request"; flow:established,to_server;
> content:"/"; http_uri; offset:2; depth:3; urilen:35;
> pcre:"/\/[a-z]\/[0-9a-f]{32}$/U"; filestore; classtype:bad-unknown;
> sid:378000131; rev:3;)
> 
> alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"RDG Scalaxy exploit
> kit binary download request"; flow:established,to_server; content:"/";
> http_uri; offset:2; depth:3; content:"/"; http_uri; within:3; urilen:37;
> pcre:"/\/[a-z]\/[0-9]\/[0-9a-f]{32}$/U"; filestore;
> classtype:trojan-activity; sid:378000133; rev:5;)
> 
> alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"RDG Obfuscated
> Base64 in Javascript - probably Scalaxy exploit kit";
> flow:established,from_server; content:"|3d 22|"; depth:1000;
> content:!"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
> distance:0;content:"|2b 2f 3d 22 3b|"; distance:62; within:67;
> fast_pattern; content:"<<18|7c|"; within:500; content:"<<12|7c|";
> within:13; content:"<<6|7c|"; within:13; filestore;
> classtype:bad-unknown; sid:378000134; rev:2;)
> 
> My "Scalaxy" pcaps do what I expected, but a pcap of a Blackhole exploit
> kit download also gets the file saved:
> 
> 11/17/2011-18:00:51.681852 verdert.in [**] /w.php?f=16&e=1 [**]
> Mozilla/4.0 (Windows 7 6.1) Java/1.6.0_22 [**] <no referer> [**] GET
> [**] HTTP/1.1 [**] 200 [**] 111759 bytes [**] x.x.x.x:n.n.n.n ->
> 194.219.29.152:80
> 
> 11/17/2011-18:00:51.129276  [**] [1:2012169:6] ET TROJAN Potential
> Blackhole Exploit Pack Binary Load Request [**] [Classification:
> Potentially Bad Traffic] [Priority: 2] {TCP} x.x.x.x:n.n.n.n ->
> 194.219.29.152:80
> 11/17/2011-18:00:51.809190  [**] [1:2011582:10] ET POLICY Vulnerable
> Java Version 1.6.x Detected [**] [Classification: Potentially Bad
> Traffic] [Priority: 2] {TCP} x.x.x.x:n.n.n.n -> 194.219.29.152:80
> 
>>
>>> I've got a couple of examples of truncated files as well, even with
>>>
>>> libhtp/default-config/response-body-limit: 0
>>> stream/reassembly/depth: 0
>>>
>>> though it's possible, I suppose, that the pcaps are broken.
>>
>> There are some of those, but it can also be an issue in our code. Things
>> can break on the stream level, in the http parsing level and finally in
>> the file handling itself :)
> 
> I'll have a dig through them with wireshark to check.

I think I know whats going on. The filestore check/set is done
independently. So it acts as a match condition, instead of a post-match
action. This means it will flag the file(s) for storing even if the sig
didn't fully match yet. Didn't run into this as I always tested with
other file matches preceding it.

Will fix it tomorrow.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------