[Oisf-users] Clarification on dropped packet counters

[Gene Albin]

So it turns out that my CentOS 5.6 server with the default kernel network
settings is not optimal for an IDS connected to a high speed network.  One
of my problems was that the kernel couldn't keep up with the flow of
traffic.  So I made the following changes to my kernel:

sysctl -w net.core.netdev_max_backlog=10000

sysctl -w net.core.rmem_devault=16777216

sysctl -w net.core.rmem_max=33554432

sysctl -w net.ipv4.tcp_mem=’194688 259584 389376’

sysctl -w net.ipv4.tcp_rmem=’1048576 4194304 33554432’

sysctl -w net.ipv4.tcp_no_metrics_save=1

Now when I run tcpdump I get 0 dropped packets after several minutes, and
after running Suricata for about 15 minutes my suricata.log drops were down
to 3.9%. Much better than the 27% I had been seeing.

Further, looking at the stats.log file my tcp.ssn_memcap_drop number is at 0
for the same run.  Unfortunately the tcp.segment_memcap_drop number is still
high at 2938343 (out of 14754737 packets)

So even though I've minimized my drops, I'm still uncertain about the
metrics listed in my original post.

Gene

On Tue, Aug 9, 2011 at 2:38 PM, Gene Albin <gene.albin at gmail.com> wrote:

> I'm trying to make sense out of the various packet metrics in the
> suricata.log and stats.log files.  Can anyone shed light on what
> specifically each of these counters is measuring?
>
> suricata.log:
> [4947] 8/8/2011 -- 14:50:31 - (source-pcap.c:561) <Info>
> (ReceivePcapThreadExitStats) -- (ReceivePcap) Packets 238097983, bytes
> 182382168249
> [4947] 8/8/2011 -- 14:50:31 - (source-pcap.c:569) <Info>
> (ReceivePcapThreadExitStats) -- (ReceivePcap) Pcap Total:539841804
> Recv:388969943 Drop:150871861 (27.9%).
>
> Looking at these two lines from suricata.log it looks like the pcap engine
> received a total of 238 million packets AND 388 million packets.  Also,
> notice how the difference between 539M and 388M is 150M AND the difference
> between 388M and 238M is also 150M.  I checked another set of suricata.log
> and stats.log files I have and found that this relationship between Recv and
> Drop, and Packets and Drop appears the be the same in that file.
>
> What specifically are each of these metrics measuring and from where are
> the measurements taken (nic, pcap, suricata)?
> What is the relationship between these numbers?
>
> Stats.log:
> decoder.pkts              | Decode & Stream   | 238097982
> tcp.ssn_memcap_drop       | Decode & Stream   | 299435
> tcp.segment_memcap_drop   | Decode & Stream   | 31445861
>
> In stats.log the decoder.pkts line matches up with the (ReceivePcap)
> Packets line in the suricata.log file.  What about these memcap drop
> lines?  They don't seem to match up with the drop counter in suricata.log
> leading me to believe that these are packets dropped by Suricata and are
> independent of the ones in the suricata.log file.
>
> Sure would appreciate any insight into the differences between these
> metrics.  I'm just a bit confused.
>
> Thanks,
> --
> Gene Albin
> gene.albin at gmail.com
>
>


-- 
Gene Albin
gene.albin at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20110809/a97727ce/attachment-0002.html>