[Oisf-devel] Inspect a memory leak issue for all suricata version.
Peter Manev
petermanev at gmail.com
Fri Jul 25 06:51:51 UTC 2014
On Fri, Jul 25, 2014 at 5:55 AM, nexthop <nexthop at 126.com> wrote:
> Thanks your response.
>
> 1) Hardware:
> Processor: I5 2core； RAM: 4GB, Intel H64-chipset;
> 2) NIC： Intel 82576
> 3）HTTP CPS： 10000 req/s
>
> We found that the resassemble memory usage is large.
> If we force to free the session, the memory issue is gone.
>
> we suspect that there is a corner-case: if the flow-session is terminated
> by an alert, the flow is not free by engine, it need to timeout to free the
> session?
>
> thanks
> George
>
>
my last question that is still unanswered was - how much traffic are
you inspecting?
Yes the flow timeouts play a major role.
I do not see a problem if the memory grows from 300MB to 2GB ...there
are too many dependencies to try to troubleshoot the issue you think
you have with just this information from my point of view.
What makes you think that the an alert terminates a flow session ?
>
>
>
> At 2014-07-25 02:39:53, "Peter Manev" <petermanev at gmail.com> wrote:
>>On Thu, Jun 5, 2014 at 2:20 PM, greatwall <13811880491 at 126.com> wrote:
>>> Hi all:
>>>
>>> I run suricata in Debian(5.0.0) platform. I met an issue that the memory
>>> usage of suricta process is increased from 300MB to 2GB, I had tested the
>>> suricata of 1.4.5 /1.4.6/2.0/2.0.1, there is saome issue in these
>>> version.
>>> my configuration is as following:
>>> ==========================================
>>> %YAML 1.1
>>> ---
>>>
>>> max-pending-packets: 65000
>>> host-mode: auto
>>> pid-file: /var/run/suritaca.pid
>>> action-order:
>>> - pass
>>> - reject
>>> - drop
>>> - alert
>>> default-log-dir: /var/log/suritaca/
>>> outputs:
>>> - fast:
>>> enabled: no
>>> filename: fast.log
>>> append: no
>>> - http-log:
>>> enabled: yes
>>> filename: http.log
>>> append: yes
>>> - stats:
>>> enabled: no
>>> filename: stats.log
>>> interval: 8
>>> nfq:
>>> mode: accept
>>> detect-engine:
>>> - profile: medium
>>> - custom-values:
>>> toclient-src-groups: 200
>>> toclient-dst-groups: 200
>>> toclient-sp-groups: 200
>>> toclient-dp-groups: 300
>>> toserver-src-groups: 200
>>> toserver-dst-groups: 400
>>> toserver-sp-groups: 200
>>> toserver-dp-groups: 250
>>> - sgh-mpm-context: auto
>>> - inspection-recursion-limit: 3000
>>> threading:
>>> set-cpu-affinity: yes
>>> cpu-affinity:
>>> - management-cpu-set:
>>> cpu: [ 0, 1 ]
>>> - receive-cpu-set:
>>> cpu: [ 2, 3 ]
>>> - decode-cpu-set:
>>> cpu: [ 4 ]
>>> mode: "balanced"
>>> - stream-cpu-set:
>>> cpu: [ 5 ]
>>> - detect-cpu-set:
>>> cpu: [ 6, 7 ]
>>> mode: "exclusive"
>>> prio:
>>> low: [ "all" ]
>>> medium: [ 6-7 ]
>>> high: [ "all" ]
>>> default: "medium"
>>> - verdict-cpu-set:
>>> cpu: [ 5 ]
>>> prio:
>>> default: "high"
>>> - reject-cpu-set:
>>> cpu: [ 5 ]
>>> prio:
>>> default: "low"
>>> - output-cpu-set:
>>> cpu: [ 5 ]
>>> prio:
>>> default: "medium"
>>>
>>> detect-thread-ratio: 1.5
>>>
>>> cuda:
>>> - mpm:
>>> packet-buffer-limit: 2400
>>> packet-size-limit: 1500
>>> packet-buffers: 10
>>> batching-timeout: 1
>>> page-locked: enabled
>>> device-id: 0
>>> cuda-streams: 2
>>> mpm-algo: ac
>>> pattern-matcher:
>>> - b2gc:
>>> search-algo: B2gSearchBNDMq
>>> hash-size: low
>>> bf-size: medium
>>> - b2gm:
>>> search-algo: B2gSearchBNDMq
>>> hash-size: low
>>> bf-size: medium
>>> - b2g:
>>> search-algo: B2gSearchBNDMq
>>> hash-size: low
>>> bf-size: medium
>>> - b3g:
>>> search-algo: B3gSearchBNDMq
>>> hash-size: low
>>> bf-size: medium
>>> - wumanber:
>>> hash-size: low
>>> bf-size: medium
>>>
>>>
>>> defrag:
>>> memcap: 32mb
>>> hash-size: 65536
>>> trackers: 65535 # number of defragmented flows to follow
>>> max-frags: 65535 # number of fragments to keep (higher than trackers)
>>> prealloc: yes
>>> timeout: 60
>>>
>>> flow:
>>> memcap: 512mb
>>> hash-size: 102400
>>> prealloc: 400000
>>> emergency-recovery: 30
>>> prune-flows: 5
>>>
>>> vlan:
>>> use-for-tracking: true
>>>
>>> flow-timeouts:
>>> default:
>>> new: 30
>>> established: 300
>>> closed: 0
>>> emergency-new: 10
>>> emergency-established: 100
>>> emergency-closed: 0
>>> tcp:
>>> new: 60
>>> established: 600
>>> closed: 120
>>> emergency-new: 10
>>> emergency-established: 300
>>> emergency-closed: 20
>>> udp:
>>> new: 30
>>> established: 300
>>> emergency-new: 10
>>> emergency-established: 100
>>> icmp:
>>> new: 30
>>> established: 300
>>> emergency-new: 10
>>> emergency-established: 100
>>> stream:
>>> memcap: 1024mb
>>> checksum-validation: yes
>>> inline: auto
>>> prealloc-sessions: 32768
>>> midstream: false
>>> max-synack-queued: 16
>>>
>>> reassembly:
>>> memcap: 64mb
>>> depth: 1mb
>>> toserver-chunk-size: 2560
>>> toclient-chunksize: 2560
>>> randomize-chunk-size: yes
>>>
>>> host:
>>> hash-size: 4096
>>> prealloc: 1000
>>> memcap: 16777216
>>>
>>> logging:
>>> default-log-level: info
>>> default-output-filter:
>>> outputs:
>>> - console:
>>> enabled: no
>>> - file:
>>> enabled: no
>>> filename: /var/log/suritaca/log
>>> # - syslog:
>>> # enabled: no
>>> # facility: local5
>>> # format: "[%i] <%d> -- "
>>>
>>> pfring:
>>> - interface: eth1
>>> threads: 1
>>> cluster-id: 99
>>> cluster-type: cluster-round-robin
>>> ipfw:
>>> default-rule-path: /var/log/suritaca/rules/
>>> rule-files:
>>> - ips.rules
>>> classification-file: /var/log/suritaca/rules/classification.config
>>> reference-config-file: /var/log/suritaca/rules/reference.config
>>> threshold-file: /var/log/suritaca/rules/threshold.config
>>>
>>> vars:
>>> address-groups:
>>> HOME_NET:
>>>
>>> "[192.168.62.245,192.168.62.246,192.168.62.247,192.168.62.248,192.168.62.249,192.168.62.250,192.168.62.251,192.168.62.252,192.168.62.253,192.168.62.254]"
>>> EXTERNAL_NET: "any"
>>> HTTP_SERVERS: "$HOME_NET"
>>> #SMTP_SERVERS: "$HOME_NET"
>>> #SQL_SERVERS: "$HOME_NET"
>>> #DNS_SERVERS: "$HOME_NET"
>>> #TELNET_SERVERS: "$HOME_NET"
>>> #AIM_SERVERS: "$EXTERNAL_NET"
>>> #DNP3_SERVER: "$HOME_NET"
>>> #DNP3_CLIENT: "$HOME_NET"
>>> #MODBUS_CLIENT: "$HOME_NET"
>>> #MODBUS_SERVER: "$HOME_NET"
>>> #ENIP_CLIENT: "$HOME_NET"
>>> #ENIP_SERVER: "$HOME_NET"
>>> port-groups:
>>> HTTP_PORTS: "[80]"
>>> SHELLCODE_PORTS: "!80"
>>> #ORACLE_PORTS: 1521
>>> host-os-policy:
>>> windows: [0.0.0.0/0]
>>> bsd: []
>>> bsd-right: []
>>> old-linux: []
>>> linux: []
>>> old-solaris: []
>>> solaris: []
>>> hpux10: []
>>> hpux11: []
>>> irix: []
>>> macos: []
>>> vista: []
>>> windows2k3: []
>>> asn1-max-frames: 256
>>>
>>> pcre:
>>> match-limit: 3500
>>> match-limit-recursion: 1500
>>>
>>> app-layer:
>>> protocols:
>>> tls:
>>> enabled: no
>>> detection-ports:
>>> toserver: 443
>>>
>>> #no-reassemble: yes
>>> dcerpc:
>>> enabled: no
>>> ftp:
>>> enabled: no
>>> ssh:
>>> enabled: no
>>> smtp:
>>> enabled: no
>>> imap:
>>> enabled: detection-only
>>> msn:
>>> enabled: no
>>> smb:
>>> enabled: no
>>> detection-ports:
>>> toserver: 139
>>> dns:
>>> tcp:
>>> enabled: no
>>> udp:
>>> enabled: no
>>> http:
>>> enabled: yes
>>> memcap: 128mb
>>> #libhtp:
>>> # default-config:
>>> # personality: IDS
>>> # request-body-limit: 0
>>> # response-body-limit: 0
>>> # request-body-minimal-inspect-size: 32kb
>>> # request-body-inspect-window: 4kb
>>> # response-body-minimal-inspect-size: 32kb
>>> # response-body-inspect-window: 4kb
>>> # double-decode-path: no
>>> # double-decode-query: no
>>> profiling:
>>> rules:
>>> enabled: no
>>> filename: rule_perf.log
>>> append: no
>>> sort: avgticks
>>> packets:
>>> enabled: no
>>> filename: packet_stats.log
>>> append: no
>>> csv:
>>> enabled: no
>>> filename: packet_stats.csv
>>> coredump:
>>> max-dump: unlimited
>>> ==========================================
>>>
>>> Could you please help give me a hand?
>>> Thanks
>>>
>>> George
>>>
>>>
>>> 来自网易手机号码邮箱了解更多
>>>
>>> _______________________________________________
>>> Suricata IDS Devel mailing list: oisf-devel at openinfosecfoundation.org
>>> Site: http://suricata-ids.org | Participate:
>>> http://suricata-ids.org/participate/
>>> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>> Redmine: https://redmine.openinfosecfoundation.org/
>>
>>
>>
>>How much traffic are you inspecting?
>>
>>
>>--
>>Regards,
>>Peter Manev
>>_______________________________________________
>>Suricata IDS Devel mailing list: oisf-devel at openinfosecfoundation.org
>>Site: http://suricata-ids.org | Participate:
>> http://suricata-ids.org/participate/
>>List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>Redmine: https://redmine.openinfosecfoundation.org/
>
>
>
--
Regards,
Peter Manev
More information about the Oisf-devel
mailing list