<br><br><div class="gmail_quote">On Sun, Jan 6, 2013 at 4:04 AM, Kevin Ross <span dir="ltr"><<a href="mailto:kevross33@googlemail.com" target="_blank">kevross33@googlemail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br><br>I was wondering what people's experience is of accurately (and automatically) extracting files from network traffic (or PCAPs) for then automatically doing further analysis to work out if they are suspicious before submitting to a sandbox like Cuckoobox?<br>
<br>My method right now for getting an accurate file really is just processing suricata metadata files in order to redownload any interesting files which obviously causes issues for stealth as well as not working if it is a download location generated for one time use or is expecting certain things to be right before allowing the download. Suricata's file extraction generally ends up with very small parts of the original files when storing to disk unless I am doing something wrong? <br>
<br>If it can't be done accurately from live network traffic is it possible to get them from PCAPs in an accurate way suitable for at least static anlysis and ideally so it will run on a device with a decent reliability as I do have full packet capture although not for a huge length of time - perhaps a day of traffic. Does anyone else have any solutions for things they have done for this? I know it must be possible as various network malware detection companies (Fireeye, Damballa, HBGary etc) take files from the network although I am unsure how they accomplish this accurately to allow for proper execution.<br>
<br>Thanks for any tips or thoughts.<span class="HOEnZb"><font color="#888888"><br>Kevin<br><br><br>
</font></span><br>_______________________________________________<br>
Suricata IDS Users mailing list: <a href="mailto:oisf-users@openinfosecfoundation.org">oisf-users@openinfosecfoundation.org</a><br>
Site: <a href="http://suricata-ids.org" target="_blank">http://suricata-ids.org</a> | Support: <a href="http://suricata-ids.org/support/" target="_blank">http://suricata-ids.org/support/</a><br>
List: <a href="https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users" target="_blank">https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users</a><br>
OISF: <a href="http://www.openinfosecfoundation.org/" target="_blank">http://www.openinfosecfoundation.org/</a><br></blockquote></div><br>Hi Kevin,<br><br>I think the file extraction is accurate.<br>Have you seen/followed the guide here:<br>
<a href="https://redmine.openinfosecfoundation.org/projects/suricata/wiki/File_Extraction">https://redmine.openinfosecfoundation.org/projects/suricata/wiki/File_Extraction</a><br><br>I think you need to tweak the libhtp settings too:<br>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote"><em>libhtp.default-config.request-body-limit</em> / <em>libhtp.server-config.<config>.request-body-limit</em>
controls how much of the HTTP request body is tracked for inspection by
the http_client_body keyword, but also used to limit file inspection. A
value of 0 means unlimited.<br></blockquote>since the file extraction is done from http...<br><br>aka:<br><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex" class="gmail_quote">
<pre>libhtp:
default-config:
personality: IDS
# Can be specified in kb, mb, gb. Just a number indicates
# it's in bytes.
request-body-limit: 0
response-body-limit: 0</pre></blockquote><br><br>Setting it to unlimited depending on how much traffic you inspect could be mem intensive - but you could surely set it to a smaller value if you will (than 0 - unlimited) - 12Mb.<br>
That would allow you to extract any file that is max size 12 Mb over http.<br><br clear="all">You could generally read through that guide as well, if you would like:<br><a href="https://redmine.openinfosecfoundation.org/projects/suricata/wiki/MD5">https://redmine.openinfosecfoundation.org/projects/suricata/wiki/MD5</a><br>
<br>Thank you<br><br>-- <br><div>Regards,</div>
<div>Peter Manev</div>