[Oisf-users] 3.2 - Wildly Dropping Packets

Cloherty, Sean E scloherty at mitre.org
Fri Dec 2 13:21:33 UTC 2016


Thanks for the quick response - 

My startup script, suricata.log and suricata.yaml are attached. One note - the entries on the log are with a modified yaml where I was testing the unix sockets but if you ignore those errors, the output is pretty much identical to what is usually displayed.

OS info is:
CentOS Linux release 7.2.1511 (Core) /  3.10.0-327.36.3.el7.x86_64
The server has 16 cores / 32 threads, 128GB of RAM, has an 10Gb Intel NIC running 5.2.15-k ixgbe drivers.  
Max traffic seen on the interface in the last 4 months has been 1.9 Gb/s, but usually mid-day peaks are around 1.3 Gb/s

Sean.

-----Original Message-----
From: Peter Manev [mailto:petermanev at gmail.com] 
Sent: Friday, December 02, 2016 02:51 AM
To: Cloherty, Sean E <scloherty at mitre.org>
Cc: oisf-users at lists.openinfosecfoundation.org
Subject: Re: [Oisf-users] 3.2 - Wildly Dropping Packets

On Thu, Dec 1, 2016 at 8:51 PM, Cloherty, Sean E <scloherty at mitre.org> wrote:
>
> Thankfully this is a test box, but it has been cooking along with a 
> less than 1% drop rate until I upgraded from 3.1.3 to 3.2
>
>
>
> ----------------------------------------------------------------------
> --------------
>
> Date: 12/1/2016 -- 13:18:53 (uptime: 0d, 04h 29m 24s)
>
> ----------------------------------------------------------------------
> --------------
>
> Counter                                    | TM Name                   | Value
>
> ----------------------------------------------------------------------
> --------------
>
> capture.kernel_packets                     | Total                     | 2926059934
>
> capture.kernel_drops                       | Total                     | 2471792091
>
> decoder.pkts                               | Total                     | 451535597
>
> decoder.bytes                              | Total                     | 273993787357
>
> decoder.ipv4                               | Total                     | 451533977
>
> decoder.ipv6                               | Total                     | 3194
>
> decoder.ethernet                           | Total                     | 451535597
>
> decoder.tcp                                | Total                     | 340732185
>
> decoder.udp                                | Total                     | 109126355
>
> decoder.sctp                               | Total                     | 5
>
> decoder.icmpv4                             | Total                     | 62733
>
> decoder.icmpv6                             | Total                     | 782
>
> decoder.gre                                | Total                     | 425
>
> decoder.teredo                             | Total                     | 2280
>
> decoder.avg_pkt_size                       | Total                     | 606
>
> decoder.max_pkt_size                       | Total                     | 1514
>
> defrag.ipv4.fragments                      | Total                     | 1495
>
> defrag.ipv4.reassembled                    | Total                     | 626
>
> defrag.ipv6.fragments                      | Total                     | 26
>
> tcp.sessions                               | Total                     | 9529307
>
> tcp.pseudo                                 | Total                     | 358711
>
> tcp.syn                                    | Total                     | 4198604
>
> tcp.synack                                 | Total                     | 2568583
>
> tcp.rst                                    | Total                     | 3300939
>
> tcp.reassembly_gap                         | Total                     | 3687801
>
> detect.alert                               | Total                     | 39
>
> detect.nonmpm_list                         | Total                     | 4
>
> app_layer.flow.http                        | Total                     | 435661
>
> app_layer.tx.http                          | Total                     | 1705795
>
> app_layer.tx.smtp                          | Total                     | 5009
>
> app_layer.flow.tls                         | Total                     | 245724
>
> app_layer.flow.ssh                         | Total                     | 835
>
> app_layer.flow.dcerpc_tcp                  | Total                     | 17
>
> app_layer.flow.dns_tcp                     | Total                     | 49
>
> app_layer.tx.dns_tcp                       | Total                     | 98
>
> app_layer.flow.failed_tcp                  | Total                     | 2754586
>
> app_layer.flow.dcerpc_udp                  | Total                     | 4
>
> app_layer.flow.dns_udp                     | Total                     | 265532
>
> app_layer.tx.dns_udp                       | Total                     | 281469
>
> app_layer.flow.failed_udp                  | Total                     | 2327184
>
> flow_mgr.closed_pruned                     | Total                     | 1628718
>
> flow_mgr.new_pruned                        | Total                     | 3996279
>
> flow_mgr.est_pruned                        | Total                     | 6816703
>
> flow.spare                                 | Total                     | 10278
>
> flow.tcp_reuse                             | Total                     | 204468
>
> flow_mgr.flows_checked                     | Total                     | 14525
>
> flow_mgr.flows_notimeout                   | Total                     | 13455
>
> flow_mgr.flows_timeout                     | Total                     | 1070
>
> flow_mgr.flows_timeout_inuse               | Total                     | 171
>
> flow_mgr.flows_removed                     | Total                     | 899
>
> flow_mgr.rows_checked                      | Total                     | 65536
>
> flow_mgr.rows_skipped                      | Total                     | 62883
>
> flow_mgr.rows_empty                        | Total                     | 3
>
> flow_mgr.rows_busy                         | Total                     | 1
>
> flow_mgr.rows_maxlen                       | Total                     | 15
>
> tcp.memuse                                 | Total                     | 66079224
>
> tcp.reassembly_memuse                      | Total                     | 16619040438
>
> dns.memuse                                 | Total                     | 2244149
>
> http.memuse                                | Total                     | 335827011
>
> flow.memuse                                | Total                     | 94227136
>
>


Interesting.
No memcaphits. Is the only change upgrade from 3.1.3 to 3.2? (nothing else?)

I would like to reproduce this.
Can you please share your suricata.log and your suricata.yaml (feel free to do it privately if you would like)?

What is your start command and OS you are running on?

Thank you



--
Regards,
Peter Manev

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: start_suricata.txt
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20161202/95bcb379/attachment-0002.txt>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: suricata.yaml
Type: application/octet-stream
Size: 63722 bytes
Desc: suricata.yaml
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20161202/95bcb379/attachment-0004.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: suricata.log
Type: application/octet-stream
Size: 51131 bytes
Desc: suricata.log
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20161202/95bcb379/attachment-0005.obj>


More information about the Oisf-users mailing list