[Oisf-users] File extraction problems (false positives)

[Peter Manev]

On Fri, Feb 21, 2014 at 2:03 PM, Joakim Kunst Forsbakk
<forsbakk at mnemonic.no> wrote:
> Hello all,
>
> I've been trying to get file extraction to work on some Suricata sensors for a while, and I am having some problems I haven't read about in the forum before.
> I have rules to detect and store exe, zip and pdf files based on filemagic. Many of the downloaded files are correctly identified and stored, but most of the stored files are flase positives. The stored files are mostly png, jpeg, gif, html, ascii files and so on.
>
> Some example rules I use:
> alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"FILEMAGIC PDF document"; flow:established,to_client; filemagic:"PDF document"; filestore:to_client,file; classtype:low-severity; sid:1240007; rev:2;)
> alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"FILEMAGIC PE32 executable"; flow:established,to_client; filemagic:"PE32 executable"; filestore:to_client,file; classtype:low-severity; sid:1200443; rev: 1;)
> alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"FILEMAGIC PE for MS Windows"; flow:established,to_client; filemagic:"PE for MS Windows"; filestore:to_client,file; classtype:low-severity; sid:1220012; rev:2;)
> alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"FILEMAGIC Zip archive data"; flow:established,to_client; filemagic:"Zip archive data"; filestore:to_client,file; classtype:low-severity; sid:1230003; rev:2;)
>


Could you just try that:

> I've also tested these, without seeing any big difference:
> alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:"FILECARVING PDF"; flow:to_client,established; file_data; content:"%PDF-"; nocase; filestore; classtype:low-severity; sid:1250001; rev:1;)
> alert tcp $EXTERNAL_NET any -> $HOME_NET any (msg:" FILECARVING EXE"; flow:to_client,established; file_data; content:"MZ"; byte_jump:4,58,relative,little; content:"PE|00 00|"; within:4; distance:-64; filestore; classtype:low-severity; sid:1250002; rev:1;)
>
> Any ideas on how to improve this and get rid of the false positives?
>
> I've tested on sensors in different networks. Most of the sensors are Suricata 1.4.6, but I've also tested 1.4.7. They have 32 cores, 64gb ram, Linux 3.12.0-1.el6.elrepo.x86_64 CentOS.
>
> Excerpt from suricata.yaml:
>
> af-packet:
>   - interface: eth4
>     threads: 16
>     cluster-id: 94
>     cluster-type: cluster_flow
>     defrag: yes
>     use-mmap: yes
>     ring-size: 30000
>     buffer-size: 128536
>   - interface: eth5
>     threads: 16
>     cluster-id: 95
>     cluster-type: cluster_flow
>     defrag: yes
>     use-mmap: yes
>     ring-size: 30000
>     buffer-size: 128536
>
> stream:
>   memcap: 12gb
>   checksum-validation: yes
>   inline: auto
>   reassembly:
>     memcap: 12gb
>     depth: 0
>     toserver-chunk-size: 2560
>     toclient-chunk-size: 2560
>
> libhtp:
>    default-config:
>      personality: IDS
>      request-body-limit: 0
>      response-body-limit: 0
>
> Thanks for any help.
>
> Best regards
> Joakim
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/



-- 
Regards,
Peter Manev