<div dir="ltr">Also noteworthy that I use pf_ring for faster capturing</div><div class="gmail_extra"><br><br><div class="gmail_quote">2014/1/13 Nikita Kislitsin <span dir="ltr"><<a href="mailto:kislitsin@group-ib.ru" target="_blank">kislitsin@group-ib.ru</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">The corellation between fast.log and http.log looks poor in my case so far. I got some records in fast.log caused by botnets activity. I wanted to find details on http-requests that were sent to C2-servers. And there's no such records in http.log! Looks like many http-requests are missing in http.log.<div>
<br></div><div>Probably I misconfigured Suricata. It shows such messages:</div><div><font face="courier new, monospace">13/1/2014 -- 22:04:26 - <Info> - Flow emergency mode over, back to normal... unsetting FLOW_EMERGENCY bit (ts.tv_sec: 1389636266, ts.tv_usec:8746) flow_spare_q status(): 36% flows at the queue</font></div>
<div><br></div><div>And in stats.log I see that:</div><div><br></div><div><div><font face="courier new, monospace">capture.kernel_packets | RxPFRp6p11 | 326355778</font></div><div><font face="courier new, monospace">capture.kernel_drops | RxPFRp6p11 | 16148543</font></div>
<div><br></div><div>Looks like Suricata is misconfigured and it misses packets? I got plenty of memory, two good CPU's. The link is 10G, and actual load is 2-3G. I use only botcc.rules now.</div><div><br></div>
<div>Some perfomance-related details from my suricata.yaml:</div><div><br></div><div><font face="courier new, monospace">max-pending-packets: 65000<br></font></div><div><font face="courier new, monospace"><br>
</font></div><div><font face="courier new, monospace">detect-engine:</font></div><div><font face="courier new, monospace"> - profile: medium</font></div><div><font face="courier new, monospace"> - custom-values:</font></div>
<div><font face="courier new, monospace"> toclient-src-groups: 200</font></div><div><font face="courier new, monospace"> toclient-dst-groups: 200</font></div><div><font face="courier new, monospace"> toclient-sp-groups: 200</font></div>
<div><font face="courier new, monospace"> toclient-dp-groups: 300</font></div><div><font face="courier new, monospace"> toserver-src-groups: 200</font></div><div><font face="courier new, monospace"> toserver-dst-groups: 400</font></div>
<div><font face="courier new, monospace"> toserver-sp-groups: 200</font></div><div><font face="courier new, monospace"> toserver-dp-groups: 200</font></div><div><font face="courier new, monospace"> - sgh-mpm-context: auto</font></div>
<div><font face="courier new, monospace"> - inspection-recursion-limit: 3000</font></div><div><font face="courier new, monospace"><br></font></div>What should I change so Suricata would work properly?</div><div><br>
</div><div>Thanks!</div></div><div class="gmail_extra"><div><div class="h5"><br><br><div class="gmail_quote">2014/1/12 Peter Manev <span dir="ltr"><<a href="mailto:petermanev@gmail.com" target="_blank">petermanev@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>On Sat, Jan 11, 2014 at 10:17 PM, Nikita Kislitsin<br>
<<a href="mailto:kislitsin@group-ib.ru" target="_blank">kislitsin@group-ib.ru</a>> wrote:<br>
><br>
> Thanks!<br>
><br>
> I need to search in http requests and write a log that includes all the details about matching sessions - src/dst ip:port, matched rule msg, domain, URI and method of HTTP-request.<br>
><br>
> Looks like Suricata can't do that from the box, right?<br>
><br>
><br>
<br>
<br>
</div>Not right of the box.<br>
It still looks to me that you need to correlate data - but you would<br>
like all the information about that specific session to be written in<br>
one specific log, correct?<br>
<br>
Just to point out entries in the http.log are not directly related to<br>
those in the fast.log(alert). In other words - http.log logs all the<br>
http requests Suriacta sees, regardless of the fact if alerts are<br>
triggered or not.<br>
<br>
Suricata also can log DNS,TLS,Files detailed logs (besides alert and<br>
http) - fyi.<br>
<br>
<br>
<br>
thanks<br>
<span><font color="#888888"><br>
<br>
<br>
--<br>
Regards,<br>
Peter Manev<br>
</font></span></blockquote></div><br><br clear="all"><div><br></div></div></div><div class="im">-- <br><div dir="ltr"><div style="font-size:13px;font-family:arial,sans-serif"><table style="font-size:medium;font-family:Times">
<tbody><tr><td style="text-align:center;padding-right:10px;width:160px"><img src="http://www.group-ib.ru/i/gib_logo.png" width="143" height="59" alt="Group-IB"><br><font style="font-weight:bold;font-size:10px;font-family:sans-serif;line-height:24px">Global Cyber Security Company</font><br>
<a href="http://www.facebook.com/GroupIB" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/facebook.png" width="34" height="34"></a> <a href="http://twitter.com/groupib" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/twitter.png" width="34" height="34"></a> <a href="http://www.linkedin.com/groups/GroupIB-Cybercrime-Cyberterrorism-4390171" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/linkedin.png" width="34" height="34"></a> <a href="http://www.youtube.com/user/GroupIB" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/youtube.png" width="34" height="34"></a></td>
<td><font color="#000000" style="font-family:sans-serif;font-size:12px"><span style="font-size:16px;font-weight:bold;font-family:arial,sans-serif">Nikita Kislitsin</span><br></font><div style="font-family:sans-serif;font-size:12px">
<span style>Head of Botnet Monitoring Project</span><font color="#000000"><br></font></div><font style="font-family:sans-serif;font-size:12px;font-weight:bold" color="#000000">Group-IB</font><br><font face="sans-serif"><span style="font-size:12.222222328186035px">+7 (495) </span><span style="font-size:12px">984-33-64</span><span style="font-size:12.222222328186035px"> ext. 137</span></font><br>
<font face="sans-serif"><span style="font-size:12px"><u></u>+7 (903) 791-65-28<u></u></span></font><br><a href="mailto:kislitsin@group-ib.com" style="font-family:sans-serif;font-size:12px;color:rgb(17,85,204)" target="_blank">kislitsin@group-ib.com</a><br>
<a href="http://www.group-ib.com/" style="font-family:sans-serif;font-size:12px;color:rgb(17,85,204)" target="_blank">www.group-ib.com</a><font face="sans-serif"><span style="font-size:12px"> </span></font></td></tr></tbody></table>
</div><div style="font-size:13px;font-family:arial,sans-serif"><br></div><span style="font-size:13px;font-family:arial,sans-serif"><font><div></div></font></span></div>
</div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)"><table style="font-size:medium;font-family:Times"><tbody><tr>
<td style="text-align:center;padding-right:10px;width:160px"><img src="http://www.group-ib.ru/i/gib_logo.png" width="143" height="59" alt="Group-IB"><br><font style="font-weight:bold;font-size:10px;font-family:sans-serif;line-height:24px">Global Cyber Security Company</font><br>
<a href="http://www.facebook.com/GroupIB" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/facebook.png" width="34" height="34"></a> <a href="http://twitter.com/groupib" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/twitter.png" width="34" height="34"></a> <a href="http://www.linkedin.com/groups/GroupIB-Cybercrime-Cyberterrorism-4390171" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/linkedin.png" width="34" height="34"></a> <a href="http://www.youtube.com/user/GroupIB" style="color:rgb(17,85,204)" target="_blank"><img src="http://www.group-ib.ru/i/youtube.png" width="34" height="34"></a></td>
<td><font color="#000000" style="font-family:sans-serif;font-size:12px"><span style="font-size:16px;font-weight:bold;font-family:arial,sans-serif">Nikita Kislitsin</span><br></font><div style="font-family:sans-serif;font-size:12px">
<span style="color:rgb(0,0,0)">Head of Botnet Monitoring Project</span><font color="#000000"><br></font></div><font style="font-family:sans-serif;font-size:12px;font-weight:bold" color="#000000">Group-IB</font><br><font face="sans-serif"><span style="font-size:12.222222328186035px">+7 (495) </span><span style="font-size:12px">984-33-64</span><span style="font-size:12.222222328186035px"> ext. 137</span></font><br>
<font face="sans-serif"><span style="font-size:12px"><u></u>+7 (903) 791-65-28<u></u></span></font><br><a href="mailto:kislitsin@group-ib.com" style="font-family:sans-serif;font-size:12px;color:rgb(17,85,204)" target="_blank">kislitsin@group-ib.com</a><br>
<a href="http://www.group-ib.com/" style="font-family:sans-serif;font-size:12px;color:rgb(17,85,204)" target="_blank">www.group-ib.com</a><font face="sans-serif"><span style="font-size:12px"> </span></font></td></tr></tbody></table>
</div><div style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)"><br></div><span style="font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)"><font><div></div></font></span></div>
</div>