[Oisf-users] Couple of questions regarding stats.log

[Brandon Ganem]

Hello,
I was reading through:
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Statistics
to help me digest what i'm seeing in stats.log.

Some concerning values i'm seeing (bold). I'm not sure if these values are
something to be concerned about. On exit I don't appear to be dropping
packets.

Counter                   | TM Name                   | Value
-------------------------------------------------------------------
tcp.sessions              | Detect                    | 12565818
*tcp.ssn_memcap_drop       | Detect                    | 7176515*
tcp.pseudo                | Detect                    | 371830
tcp.invalid_checksum      | Detect                    | 0
tcp.no_flow               | Detect                    | 0
tcp.reused_ssn            | Detect                    | 7
tcp.memuse                | Detect                    | 289406976
tcp.syn                   | Detect                    | 19778120
tcp.synack                | Detect                    | 9945438
tcp.rst                   | Detect                    | 2156671
*tcp.segment_memcap_drop   | Detect                    | 47685491*
*tcp.stream_depth_reached  | Detect                    | 621*
tcp.reassembly_memuse     | Detect                    | 6442450854
*tcp.reassembly_gap        | Detect                    | 1080680*
*
*
When I close suricata the packet loss seems fine -
[2381] 7/6/2012 -- 15:44:16 - (source-pfring.c:446) <Info>
(ReceivePfringThreadExitStats) -- (RxPFR1) Pfring Total:814218358
Recv:813945791 Drop:272567 (0.0%).

The box is a dualsocket dualcore with 8GB of ram. I see anywhere from
150mb/s-350mb/s depending on the time of day. I've seen the box reach about
4GB of ram at its max and suricata I've seen reach about 250% cpu (2.5
cores), but it typically stays around 50-150% and 2-3GB of ram. I'm running
a ruleset of ~ 12k rules

Here are what appear to be the relevant lines from suricata.yaml

max-pending-packets: 10000

- file-store:
      enabled: yes       # set to yes to enable
      log-dir: files    # directory to store the files
      force-magic: yes   # force logging magic on all stored files
      force-md5: yes     # force logging of md5 checksums
      #waldo: file.waldo # waldo file to store the file_id across runs

  # output module to log files tracked in a easily parsable json format
  - file-log:
      enabled: yes
      filename: files-json.log
      append: yes
      #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'

      force-magic: yes   # force logging magic on all logged files
      force-md5: yes     # force logging of md5 checksums
pattern-matcher:
  - b2gc:
      search-algo: B2gSearchBNDMq
      hash-size: low
      bf-size: medium
  - b2gm:
      search-algo: B2gSearchBNDMq
      hash-size: low
      bf-size: medium
  - b2g:
      search-algo: B2gSearchBNDMq
      hash-size: low
      bf-size: medium
  - b3g:
      search-algo: B3gSearchBNDMq
      hash-size: low
      bf-size: medium
  - wumanber:
      hash-size: low
      bf-size: medium

# Defrag settings:

defrag:
  max-frags: 65535
  prealloc: yes
  timeout: 60

flow:
  memcap: 3048mb
  hash-size: 65536
  prealloc: 10000
  emergency-recovery: 30
  prune-flows: 5

flow-timeouts:

  default:
    new: 30
    established: 300
    closed: 0
    emergency-new: 10
    emergency-established: 100
    emergency-closed: 0
  tcp:
    new: 60
    established: 3600
    closed: 120
    emergency-new: 10
    emergency-established: 300
    emergency-closed: 20
  udp:
    new: 30
    established: 300
    emergency-new: 10
    emergency-established: 100
  icmp:
    new: 30
    established: 300
    emergency-new: 10
    emergency-established: 100

stream:
  memcap: 3048mb
  checksum-validation: no      # reject wrong csums
  inline: no                    # no inline mode
  reassembly:
    memcap: 1024mb
    depth: 1mb                  # reassemble 1mb into a stream
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560

# Host table:
#
# Host table is used by tagging and per host thresholding subsystems.
#
host:
  hash-size: 4096
  prealloc: 1000
  memcap: 16777216






Thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20120608/4d07aa69/attachment-0002.html>