So it turns out that my CentOS 5.6 server with the default kernel network settings is not optimal for an IDS connected to a high speed network. One of my problems was that the kernel couldn't keep up with the flow of traffic. So I made the following changes to my kernel:<div>
<br></div><div><p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal"><span style="font-family:"Courier New"">sysctl -w
net.core.netdev_max_backlog=10000</span></p>
<p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal"><span style="font-family:"Courier New"">sysctl -w
net.core.rmem_devault=16777216</span></p>
<p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal"><span style="font-family:"Courier New"">sysctl -w
net.core.rmem_max=33554432</span></p>
<p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal"><span style="font-family:"Courier New"">sysctl -w net.ipv4.tcp_mem=’194688
259584 389376’</span></p>
<p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal"><span style="font-family:"Courier New"">sysctl -w net.ipv4.tcp_rmem=’1048576
4194304 33554432’</span></p>
<p class="MsoNormal" style="margin-bottom:0in;margin-bottom:.0001pt;line-height:
normal"><span style="font-family:"Courier New"">sysctl -w
net.ipv4.tcp_no_metrics_save=1</span></p><div><br></div><div>Now when I run tcpdump I get 0 dropped packets after several minutes, and after running Suricata for about 15 minutes my suricata.log drops were down to 3.9%. Much better than the 27% I had been seeing.</div>
<div><br></div><div>Further, looking at the stats.log file my tcp.ssn_memcap_drop number is at 0 for the same run. Unfortunately the tcp.segment_memcap_drop number is still high at 2938343 (out of 14754737 packets)</div>
<div><br></div><div>So even though I've minimized my drops, I'm still uncertain about the metrics listed in my original post.</div><div><br></div><div>Gene</div><br><div class="gmail_quote">On Tue, Aug 9, 2011 at 2:38 PM, Gene Albin <span dir="ltr"><<a href="mailto:gene.albin@gmail.com">gene.albin@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">I'm trying to make sense out of the various packet metrics in the suricata.log and stats.log files. Can anyone shed light on what specifically each of these counters is measuring?<div>
<br></div><div>suricata.log:</div>
<div><div><font face="'courier new', monospace">[4947] 8/8/2011 -- 14:50:31 - (source-pcap.c:561) <Info> (ReceivePcapThreadExitStats) -- (ReceivePcap) <span style="background-color:rgb(255, 255, 0)">Packets 238</span><span style="background-color:rgb(255, 255, 255)">097983</span>, bytes 182382168249</font></div>
</div><div><div><font face="'courier new', monospace">[4947] 8/8/2011 -- 14:50:31 - (source-pcap.c:569) <Info> (ReceivePcapThreadExitStats) -- (ReceivePcap) <span style="background-color:rgb(255, 255, 0)">Pcap Total:539</span>841804 <span style="background-color:rgb(255, 255, 0)">Recv:388</span>969943 <span style="background-color:rgb(255, 255, 0)">Drop:150</span>871861 (27.9%).</font></div>
</div><div><br></div><div>Looking at these two lines from suricata.log it looks like the pcap engine received a total of 238 million packets AND 388 million packets. Also, notice how the difference between 539M and 388M is 150M AND the difference between 388M and 238M is also 150M. I checked another set of suricata.log and stats.log files I have and found that this relationship between <font face="'courier new', monospace">Recv </font>and <font face="'courier new', monospace">Drop</font>, and <font face="'courier new', monospace">Packets </font>and <font face="'courier new', monospace">Drop </font>appears the be the same in that file. </div>
<div><br></div><div>What specifically are each of these metrics measuring and from where are the measurements taken (nic, pcap, suricata)?</div><div>What is the relationship between these numbers? </div><div><br></div><div>
Stats.log:</div><div><font face="'courier new', monospace">decoder.pkts | Decode & Stream | 238097982</font></div><div><font face="'courier new', monospace">tcp.ssn_memcap_drop | Decode & Stream | 299435</font></div>
<div><font face="'courier new', monospace">tcp.segment_memcap_drop | Decode & Stream | 31445861</font></div><div><br></div><div><span>In stats.log the <font face="'courier new', monospace">decoder.pkts</font> line matches up with the </span><span style="font-family:'courier new', monospace">(ReceivePcap) Packets</span><span> line in the suricata.log file. What about these memcap drop lines? They don't seem to match up with the drop counter in suricata.log leading me to believe that these are packets dropped by Suricata and are independent of the ones in the suricata.log file.</span></div>
<div><br></div><div>Sure would appreciate any insight into the differences between these metrics. I'm just a bit confused.</div><div><br></div><div>Thanks,</div><div>-- <br><font color="#888888">Gene Albin<br><a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a><br>
<br>
</font></div>
</blockquote></div><br><br clear="all"><br>-- <br>Gene Albin<br><a href="mailto:gene.albin@gmail.com" target="_blank">gene.albin@gmail.com</a><br><br>
</div>