[Oisf-users] Couple of questions regarding stats.log

[Martin Holste]

Up your memcap settings to 4GB each and see if the numbers improve.
Both memcap drop stats should be zero when everything's right.

On Fri, Jun 8, 2012 at 10:59 AM, Brandon Ganem
<brandonganem+oisf at gmail.com> wrote:
> Hello,
> I was reading through:
> https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Statistics
> to help me digest what i'm seeing in stats.log.
>
> Some concerning values i'm seeing (bold). I'm not sure if these values are
> something to be concerned about. On exit I don't appear to be dropping
> packets.
>
> Counter                   | TM Name                   | Value
> -------------------------------------------------------------------
> tcp.sessions              | Detect                    | 12565818
> tcp.ssn_memcap_drop       | Detect                    | 7176515
> tcp.pseudo                | Detect                    | 371830
> tcp.invalid_checksum      | Detect                    | 0
> tcp.no_flow               | Detect                    | 0
> tcp.reused_ssn            | Detect                    | 7
> tcp.memuse                | Detect                    | 289406976
> tcp.syn                   | Detect                    | 19778120
> tcp.synack                | Detect                    | 9945438
> tcp.rst                   | Detect                    | 2156671
> tcp.segment_memcap_drop   | Detect                    | 47685491
> tcp.stream_depth_reached  | Detect                    | 621
> tcp.reassembly_memuse     | Detect                    | 6442450854
> tcp.reassembly_gap        | Detect                    | 1080680
>
> When I close suricata the packet loss seems fine -
> [2381] 7/6/2012 -- 15:44:16 - (source-pfring.c:446) <Info>
> (ReceivePfringThreadExitStats) -- (RxPFR1) Pfring Total:814218358
> Recv:813945791 Drop:272567 (0.0%).
>
> The box is a dualsocket dualcore with 8GB of ram. I see anywhere from
> 150mb/s-350mb/s depending on the time of day. I've seen the box reach about
> 4GB of ram at its max and suricata I've seen reach about 250% cpu (2.5
> cores), but it typically stays around 50-150% and 2-3GB of ram. I'm running
> a ruleset of ~ 12k rules
>
> Here are what appear to be the relevant lines from suricata.yaml
>
> max-pending-packets: 10000
>
> - file-store:
>       enabled: yes       # set to yes to enable
>       log-dir: files    # directory to store the files
>       force-magic: yes   # force logging magic on all stored files
>       force-md5: yes     # force logging of md5 checksums
>       #waldo: file.waldo # waldo file to store the file_id across runs
>
>   # output module to log files tracked in a easily parsable json format
>   - file-log:
>       enabled: yes
>       filename: files-json.log
>       append: yes
>       #filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
>
>       force-magic: yes   # force logging magic on all logged files
>       force-md5: yes     # force logging of md5 checksums
> pattern-matcher:
>   - b2gc:
>       search-algo: B2gSearchBNDMq
>       hash-size: low
>       bf-size: medium
>   - b2gm:
>       search-algo: B2gSearchBNDMq
>       hash-size: low
>       bf-size: medium
>   - b2g:
>       search-algo: B2gSearchBNDMq
>       hash-size: low
>       bf-size: medium
>   - b3g:
>       search-algo: B3gSearchBNDMq
>       hash-size: low
>       bf-size: medium
>   - wumanber:
>       hash-size: low
>       bf-size: medium
>
> # Defrag settings:
>
> defrag:
>   max-frags: 65535
>   prealloc: yes
>   timeout: 60
>
> flow:
>   memcap: 3048mb
>   hash-size: 65536
>   prealloc: 10000
>   emergency-recovery: 30
>   prune-flows: 5
>
> flow-timeouts:
>
>   default:
>     new: 30
>     established: 300
>     closed: 0
>     emergency-new: 10
>     emergency-established: 100
>     emergency-closed: 0
>   tcp:
>     new: 60
>     established: 3600
>     closed: 120
>     emergency-new: 10
>     emergency-established: 300
>     emergency-closed: 20
>   udp:
>     new: 30
>     established: 300
>     emergency-new: 10
>     emergency-established: 100
>   icmp:
>     new: 30
>     established: 300
>     emergency-new: 10
>     emergency-established: 100
>
> stream:
>   memcap: 3048mb
>   checksum-validation: no      # reject wrong csums
>   inline: no                    # no inline mode
>   reassembly:
>     memcap: 1024mb
>     depth: 1mb                  # reassemble 1mb into a stream
>     toserver-chunk-size: 2560
>     toclient-chunk-size: 2560
>
> # Host table:
> #
> # Host table is used by tagging and per host thresholding subsystems.
> #
> host:
>   hash-size: 4096
>   prealloc: 1000
>   memcap: 16777216
>
>
>
>
>
>
> Thank you!
>
>
>
> _______________________________________________
> Oisf-users mailing list
> Oisf-users at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>