Thanks for the tip. I am looking forward to suricata having this capability with the file_extract options that we mentioned after the last meeting but until then I was going to use packet capture and file carving with nfex or something (<a href="http://blogs.cisco.com/security/network-based-file-carving/#more-13416">http://blogs.cisco.com/security/network-based-file-carving/#more-13416</a>) and then have scripts to run clamav and other stuff against it. Ruminate looks quite good though and I like that they are honest about its shortcomings and its benefits which means you get a more realistic view of where it could fit (though from the sound of it I don't think it is ready for production sized networks though). Still it looks promising although I fear they may end up getting dwarfed by suricata and razorback but for now may do they job and worth experimenting with and I look forward to see where they go with it.<br>
<br><div class="gmail_quote">On 11 April 2011 13:46, Javier Almillategui <span dir="ltr"><<a href="mailto:jalmilla@gmu.edu">jalmilla@gmu.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div dir="ltr">
<div dir="ltr">
<div style="font-family: 'Calibri'; color: rgb(0, 0, 0); font-size: 12pt;">
<div>Hi Kevin,</div>
<div> </div>
<div>I know of Ruminate IDS project that just does what you want to do. it’s
website is <a title="http://ruminate-ids.org/" href="http://ruminate-ids.org/" target="_blank">http://ruminate-ids.org/</a> . I
think file carving has a lot of challenges specially if you want it to scale
properly.</div>
<div style="font-style: normal; display: inline; font-family: 'Calibri'; color: rgb(0, 0, 0); font-size: small; font-weight: normal; text-decoration: none;">
<div style="font: 10pt tahoma;">
<div><font face="Calibri" size="3"></font> </div>
<div><font face="Calibri" size="3">I hope this helps.</font></div>
<div><font face="Calibri" size="3"></font> </div>
<div><font face="Calibri" size="3">best,</font></div>
<div><font face="Calibri" size="3"></font> </div>
<div><font face="Calibri" size="3">Javier</font></div>
<div> </div>
<div><font face="Calibri" size="3"></font> </div>
<div style="background: none repeat scroll 0% 0% rgb(245, 245, 245);">
<div><b>From:</b> <a title="kevross33@googlemail.com" href="mailto:kevross33@googlemail.com" target="_blank">Kevin Ross</a> </div>
<div><b>Sent:</b> Monday, April 11, 2011 7:15 AM</div>
<div><b>To:</b> <a title="victor@inliniac.net" href="mailto:victor@inliniac.net" target="_blank">Victor Julien</a> ; <a title="oisf-users@openinfosecfoundation.org" href="mailto:oisf-users@openinfosecfoundation.org" target="_blank">oisf-users@openinfosecfoundation.org</a>
</div>
<div><b>Subject:</b> Re: [Oisf-users] Suricata File Carving - Malware
Detection</div></div></div>
<div> </div></div>
<div style="font-style: normal; display: inline; font-family: 'Calibri'; color: rgb(0, 0, 0); font-size: small; font-weight: normal; text-decoration: none;"><div><div></div><div class="h5">I
am definitely not suggesting this should be done on the wire or by the main
Suricata processes. What I mean once suricata drops it to disk maybe it could
have a process to analyse and feed the alerts back into suricata to have it
output an alert in unified2. Sure once suricata has file_extract options
creating your own script to run things on the file is easy but suricata could do
it straight out (or at least have it pass off). This way those users who are not
sure of the actual benefits or how to analyse files on disk to detect malware
and stuff don't have to think about it. Examples of what could be done:<br><br>-
Scan file with clamav<br>- Check for suspicious IATs such as ones checking for
debuggers or other stuff and the ability to possible threshold against a score.
That way suricata rather then generate an alert for every executable, suspicious
executables can be alerted on only (which with the right tuning could even help
zero in on unknown malware samples). <br>- Alerting on file type mismatches i.e
server claims to send an image and yet suricata is carving out an exe from the
stream or carving flash out of an excel file like the recent vulnerability (I
guess this one could possibly be handed by suricata itself if a user wanted such
alerts).<br><br>My point is while making up your own scripts to do stuff (run
clamav, jsunpack etc on files) it is nice to have it handle it by default (if
you wanted it) and means more users who are learning or unsure how to take
advantage of file carving or why you would want to could benefit from it.
<br><br>Kev<br><br><br>
<div class="gmail_quote">On 11 April 2011 10:13, Victor Julien <span dir="ltr"><<a href="mailto:victor@inliniac.net" target="_blank">victor@inliniac.net</a>></span> wrote:<br>
<blockquote style="border-left: 1px solid rgb(204, 204, 204); margin: 0px 0px 0px 0.8ex; padding-left: 1ex;" class="gmail_quote">
<div>On 04/09/2011 02:51 AM, Kevin Ross wrote:<br>> Stick with me with
this. This is pescanner from the malware cookbook. I have<br>> modified it
slightly to have more IAT alerts after reading this<br>> <a href="http://www.sans.org/reading_room/whitepapers/malicious/rss/_33649" target="_blank">http://www.sans.org/reading_room/whitepapers/malicious/rss/_33649</a>
as it has<br>> a big list of IATs at the end and their malware uses so I
added them in (in<br>> this case I would say all those IATs look bad in
combination). This was Zeus<br>> with the file carved out a pcap on <a href="http://openpacket.org" target="_blank">openpacket.org</a>. You can see
the<br>> virtustotal report for the MD5 when I searched for it here<br>>
<a href="http://www.virustotal.com/file-scan/report.html?id=2f59173cf3842b3a72ac04404ab045c339cbc6f021f24b977a27441ea881e95b-1295056538" target="_blank">http://www.virustotal.com/file-scan/report.html?id=2f59173cf3842b3a72ac04404ab045c339cbc6f021f24b977a27441ea881e95b-1295056538</a><br>
><br>>
Now what I was thinking is if they file_extract options were put into<br>>
suricata as was mentioned after the last meeting would it be hard to
have<br>> suricata or another tool check IATs, entropy, clamav scan
possibly or<br>> checking the MD5 against virustotal, shadowserver etc to
determine if is is<br>> possibly malicious? Even the IATs for their
possible usage and risk and then<br>> a threshold to then determine if the
file is likely bad. If the file was<br>> possible bad then a preprocessor
style alert could be generated by suricata<br>> with the relevant
information about the file and the possibly malicious file<br>> could be
moved to a malicious folder or something to be stored while if the<br>>
user wants executables or other files that are not detected as anything
or<br>> suspicious could be deleted meaning you have a folder of likely
samples for<br>> stuff entering your network. What do people
think?<br><br></div>This analysis (mostly) requires the full file, right? In
that case I<br>think it makes more sense for Suricata to drop the file to disk
and let<br>a separate process do post
inspection.<br><br>Cheers,<br>Victor<br><br><br>--<br>---------------------------------------------<br>Victor
Julien<br><a href="http://www.inliniac.net/" target="_blank">http://www.inliniac.net/</a><br>PGP: <a href="http://www.inliniac.net/victorjulien.asc" target="_blank">http://www.inliniac.net/victorjulien.asc</a><br>---------------------------------------------<br>
<br>_______________________________________________<br>Oisf-users
mailing list<br><a href="mailto:Oisf-users@openinfosecfoundation.org" target="_blank">Oisf-users@openinfosecfoundation.org</a><br><a href="http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
</blockquote></div><br>
</div></div><p>
</p><hr><div class="im">
_______________________________________________<br>Oisf-users mailing
list<br><a href="mailto:Oisf-users@openinfosecfoundation.org" target="_blank">Oisf-users@openinfosecfoundation.org</a><br><a href="http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
</div></div></div></div></div>
</blockquote></div><br>