[Oisf-users] Suricata consume more than 50% CPU
Peter Manev
petermanev at gmail.com
Sun Dec 20 15:11:27 UTC 2015
On Sun, Dec 20, 2015 at 2:43 PM, David Touzeau <david at articatech.com> wrote:
>
>
> Hi, all
>
> As you can see the main service consume 52.4% on a Intel Core i7 for about
> less than 10MBS bandwidth.
>
> root 31283 52.4 9.6 455496 773264 ? SNsl 14:16 6:29
> /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile
> /var/run/suricata/suricata.pid --pfring -D
>
> root 31283 65.1 9.6 455496 773264 ? SNsl 14:16 12:06
> /usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile
> /var/run/suricata/suricata.pid --pfring -D
>
> It there any tips to reduce this CPU consumption ?
>
> Configuration:
> ####################################################################################
> %YAML 1.1
> ---
>
> runmode: workers
> host-mode: auto
> pid-file: /var/run/suricata.pid
> default-log-dir: /var/log/suricata/
> unix-command:
> enabled: no
>
> outputs:
>
>
> - fast:
> enabled: no
> filename: fast.log
> append: yes
>
> - eve-log:
> enabled: yes
> type: file
> filename: eve.json
> types:
> - alert
> #- drop
>
>
> - unified2-alert:
> enabled: no
> filename: unified2.alert
> sensor-id: 0
>
> xff:
> enabled: no
> mode: extra-data
> header: X-Forwarded-For
>
> - http-log:
> enabled: no
> filename: http.log
> append: yes
>
>
> - tls-log:
> enabled: no
> filename: tls.log # File to store TLS logs.
> append: yes
> certs-log-dir: certs
>
>
> - dns-log:
> enabled: no
> filename: dns.log
> append: yes
>
> - pcap-info:
> enabled: no
>
> - pcap-log:
> enabled: no
> filename: log.pcap
> limit: 1000mb
> max-files: 2000
>
> mode: normal
> use-stream-depth: no
>
> - alert-debug:
> enabled: no
> filename: alert-debug.log
> append: yes
> filetype: regular
>
> - alert-prelude:
> enabled: no
> profile: suricata
> log-packet-content: no
> log-packet-header: yes
>
> - stats:
> enabled: yes
> filename: stats.log
> interval: 10
>
> - syslog:
> enabled: no
> identity: "suricata"
> facility: local5
>
>
> - drop:
> enabled: no
> filename: drop.log
> append: yes
> filetype: regular
>
> - file-store:
> enabled: no # set to yes to enable
> log-dir: files # directory to store the files
> force-magic: no # force logging magic on all stored files
> force-md5: no # force logging of md5 checksums
>
> - file-log:
> enabled: no
> filename: files-json.log
> append: yes
> filetype: regular
> force-magic: yes
> force-md5: yes
>
> magic-file: /usr/share/file/magic
>
> nfq:
>
>
> nflog:
> - group: 2
> buffer-size: 18432
> - group: default
> qthreshold: 1
> qtimeout: 100
> max-size: 20000
>
>
> af-packet:
> - interface: eth1
> threads: 1
> cluster-id: 99
> cluster-type: cluster_flow
> defrag: yes
> use-mmap: yes
>
> - interface: eth1
> threads: 1
> cluster-id: 98
> cluster-type: cluster_flow
> defrag: yes
>
> - interface: default
>
> legacy:
> uricontent: enabled
>
> detect-engine:
> - profile: medium
> - custom-values:
> toclient-src-groups: 2
> toclient-dst-groups: 2
> toclient-sp-groups: 2
> toclient-dp-groups: 3
> toserver-src-groups: 2
> toserver-dst-groups: 4
> toserver-sp-groups: 2
> toserver-dp-groups: 25
> - sgh-mpm-context: auto
> - inspection-recursion-limit: 3000
>
> threading:
> set-cpu-affinity: yes
>
> cpu-affinity:
> - management-cpu-set:
> cpu: [ "all" ]
>
> - receive-cpu-set:
> cpu: [ 0 ] # include only these cpus in affinity settings
>
> - decode-cpu-set:
> cpu: [ 0, 1 ]
> mode: "balanced"
>
> - stream-cpu-set:
> cpu: [ "0-1" ]
>
> - detect-cpu-set:
> cpu: [ "all" ]
> mode: "exclusive"
> prio:
> low: [ 0 ]
> medium: [ "1-2" ]
> high: [ 3 ]
> default: "medium"
>
> - verdict-cpu-set:
> cpu: [ 0 ]
> prio:
> default: "high"
> - reject-cpu-set:
> cpu: [ 0 ]
> prio:
> default: "low"
> - output-cpu-set:
> cpu: [ "all" ]
> prio:
> default: "medium"
> #
> detect-thread-ratio: 1.5
>
> # Cuda configuration.
> cuda:
> mpm:
> data-buffer-size-min-limit: 0
> data-buffer-size-max-limit: 1500
> cudabuffer-buffer-size: 500mb
> gpu-transfer-size: 50mb
> batching-timeout: 2000
> device-id: 0
> cuda-streams: 2
>
> mpm-algo: ac
>
> pattern-matcher:
> - b2gc:
> search-algo: B2gSearchBNDMq
> hash-size: low
> bf-size: medium
> - b2gm:
> search-algo: B2gSearchBNDMq
> hash-size: low
> bf-size: medium
> - b2g:
> search-algo: B2gSearchBNDMq
> hash-size: low
> bf-size: medium
> - b3g:
> search-algo: B3gSearchBNDMq
> hash-size: low
> bf-size: medium
> - wumanber:
> hash-size: low
> bf-size: medium
>
> # Defrag settings:
>
> defrag:
> memcap: 32mb
> hash-size: 65536
> trackers: 65535 # number of defragmented flows to follow
> max-frags: 65535 # number of fragments to keep (higher than trackers)
> prealloc: yes
> timeout: 60
>
>
> flow:
> memcap: 64mb
> hash-size: 65536
> prealloc: 10000
> emergency-recovery: 30
>
> vlan:
> use-for-tracking: true
>
>
> flow-timeouts:
>
> default:
> new: 30
> established: 300
> closed: 0
> emergency-new: 10
> emergency-established: 100
> emergency-closed: 0
> tcp:
> new: 60
> established: 3600
> closed: 120
> emergency-new: 10
> emergency-established: 300
> emergency-closed: 20
> udp:
> new: 30
> established: 300
> emergency-new: 10
> emergency-established: 100
> icmp:
> new: 30
> established: 300
> emergency-new: 10
> emergency-established: 100
>
> stream:
> memcap: 32mb
> checksum-validation: no # reject wrong csums
> inline: auto # auto will use inline mode in IPS mode, yes
> or no set it statically
> reassembly:
> memcap: 128mb
> depth: 1mb # reassemble 1mb into a stream
> toserver-chunk-size: 2560
> toclient-chunk-size: 2560
> randomize-chunk-size: yes
>
> host:
> hash-size: 4096
> prealloc: 1000
> memcap: 16777216
>
> logging:
>
> default-log-level: notice
> #default-log-format: "[%i] %t - (%f:%l) <%d> (%n) -- "
> default-output-filter:
>
> outputs:
> - console:
> enabled: yes
> - file:
> enabled: yes
> filename: /var/log/suricata.log
> - syslog:
> enabled: yes
> facility: syslog
> format: "[%i] <%d> -- "
>
>
> mpipe:
>
> load-balance: dynamic
> iqueue-packets: 2048
> inputs:
> - interface: xgbe2
> - interface: xgbe3
> - interface: xgbe4
>
>
> stack:
> size128: 0
> size256: 9
> size512: 0
> size1024: 0
> size1664: 7
> size4096: 0
> size10386: 0
> size16384: 0
>
>
> pfring:
>
> - interface: eth0
> threads: 2
> cluster-id: 99
> cluster-type: cluster_flow
>
> - interface: eth1
> threads: 2
> cluster-id: 98
> cluster-type: cluster_flow
>
>
> default-rule-path: /etc/suricata/rules
> rule-files:
> - drop.rules
> - dshield.rules
> - emerging-activex.rules
> - emerging-attack_response.rules
> - emerging-malware.rules
> - emerging-policy.rules
> - emerging-scan.rules
> - emerging-shellcode.rules
> - emerging-trojan.rules
> - emerging-web_client.rules
> - emerging-worm.rules
> - snort.rules
>
> classification-file: /etc/suricata/classification.config
> reference-config-file: /etc/suricata/reference.config
>
> vars:
> address-groups:
> HOME_NET: "[192.168.1.0/24,10.10.1.0/24]"
> EXTERNAL_NET: "!$HOME_NET"
> HTTP_SERVERS: "$HOME_NET"
> SMTP_SERVERS: "$HOME_NET"
> SQL_SERVERS: "$HOME_NET"
> DNS_SERVERS: "$HOME_NET"
> TELNET_SERVERS: "$HOME_NET"
> AIM_SERVERS: "$EXTERNAL_NET"
> DNP3_SERVER: "$HOME_NET"
> DNP3_CLIENT: "$HOME_NET"
> MODBUS_CLIENT: "$HOME_NET"
> MODBUS_SERVER: "$HOME_NET"
> ENIP_CLIENT: "$HOME_NET"
> ENIP_SERVER: "$HOME_NET"
>
> port-groups:
> HTTP_PORTS: "80"
> SHELLCODE_PORTS: "!80"
> ORACLE_PORTS: 1521
> SSH_PORTS: 22
> DNP3_PORTS: 20000
> FILE_DATA_PORTS: "[110,143]"
>
> action-order:
> - pass
> - drop
> - reject
> - alert
>
>
> host-os-policy:
> windows: [0.0.0.0/0]
> bsd: []
> bsd-right: []
> old-linux: []
> linux: [10.0.0.0/8, 192.168.1.100,
> "8762:2352:6241:7245:E000:0000:0000:0000"]
> old-solaris: []
> solaris: ["::1"]
> hpux10: []
> hpux11: []
> irix: []
> macos: []
> vista: []
> windows2k3: []
>
>
> asn1-max-frames: 256
>
> engine-analysis:
> rules-fast-pattern: yes
> rules: yes
>
> pcre:
> match-limit: 3500
> match-limit-recursion: 1500
>
> threshold-file: /etc/suricata/threshold.config
>
> app-layer:
> protocols:
> tls:
> enabled: yes
> detection-ports:
> dp: 443
> dcerpc:
> enabled: yes
> ftp:
> enabled: yes
> ssh:
> enabled: yes
> smtp:
> enabled: yes
> imap:
> enabled: detection-only
> msn:
> enabled: detection-only
> smb:
> enabled: yes
> detection-ports:
> dp: 139
> dns:
>
> tcp:
> enabled: yes
> detection-ports:
> dp: 53
> udp:
> enabled: yes
> detection-ports:
> dp: 53
> http:
> enabled: yes
>
> libhtp:
>
> default-config:
> personality: IDS
> request-body-limit: 3072
> response-body-limit: 3072
> request-body-minimal-inspect-size: 32kb
> request-body-inspect-window: 4kb
> response-body-minimal-inspect-size: 32kb
> response-body-inspect-window: 4kb
> double-decode-path: no
> double-decode-query: no
>
> server-config:
>
>
> profiling:
> rules:
> enabled: yes
> filename: rule_perf.log
> append: yes
> sort: avgticks
> limit: 100
>
> keywords:
> enabled: yes
> filename: keyword_perf.log
> append: yes
>
> packets:
> enabled: yes
> filename: packet_stats.log
> append: yes
>
> csv:
> enabled: no
> filename: packet_stats.csv
>
> locks:
> enabled: no
> filename: lock_stats.log
> append: yes
> coredump:
> max-dump: unlimited
>
> napatech:
> hba: -1
> use-all-streams: yes
> streams: [1, 2, 3]
>
> ############################################################################################################
>
> Stats:
> Date: 12/20/2015 -- 14:16:48
> --------------------------------------------------------------------------
> Num Rule Gid Rev Ticks % Checks Matches
> Max Ticks Avg Ticks Avg Match Avg No Match
> -------- ------------ -------- -------- ------------ ------ --------
> -------- ----------- ----------- ----------- --------------
> 1 2021621 1 6 2472462 0.00 6 0
> 626418 412077.00 0.00 412077.00
> 2 2021529 1 3 2690096101 0.55 9463 0
> 4390290 284275.19 0.00 284275.19
> 3 2018005 1 6 1262809391 0.26 10390 0
> 14480148 121540.85 0.00 121540.85
> 4 2021993 1 2 3446612 0.00 34 0
> 158850 101370.94 0.00 101370.94
> 5 2018637 1 2 12935952 0.00 129 0
> 9942498 100278.70 0.00 100278.70
> 6 24787 1 3 9454741704 1.93 124029 124014
> 74818640 76230.09 0.00 630316113.60
> 7 2021276 1 3 75600 0.00 1 0
> 75600 75600.00 0.00 75600.00
> 8 25043 1 2 78320311 0.02 1043 0
> 7832052 75091.38 0.00 75091.38
> 9 2018457 1 1 789052728 0.16 10603 0
> 9742392 74417.87 0.00 74417.87
> 10 2022078 1 2 5036420 0.00 74 0
> 125892 68059.73 0.00 68059.73
> 11 32413 1 2 10957828 0.00 199 0
> 391374 55064.46 0.00 55064.46
> 12 2018604 1 5 319594 0.00 6 0
> 262260 53265.67 0.00 53265.67
> 13 31371 1 6 188502 0.00 4 0
> 76356 47125.50 0.00 47125.50
> 14 16425 1 17 1408770 0.00 30 30
> 56286 46959.00 46959.00 0.00
> 15 2014376 1 3 229054 0.00 5 0
> 63810 45810.80 0.00 45810.80
> 16 17733 1 12 3675860 0.00 86 52
> 74808 42742.56 49390.81 32574.65
> 17 2012970 1 2 2264024 0.00 56 0
> 89748 40429.00 0.00 40429.00
> 18 24791 1 3 4794438838 0.98 124030 124016
> 101016232 38655.48 0.00 342459917.00
> 19 2012969 1 2 2750828 0.00 73 0
> 239544 37682.58 0.00 37682.58
> 20 32412 1 2 14092239 0.00 374 0
> 151416 37679.78 0.00 37679.78
> 21 23224 1 6 37494 0.00 1 0
> 37494 37494.00 0.00 37494.00
> 22 32387 1 1 70722 0.00 2 0
> 69318 35361.00 0.00 35361.00
> 23 2012981 1 3 70560 0.00 2 0
> 37080 35280.00 0.00 35280.00
> 24 2017816 1 4 4166644 0.00 120 0
> 112896 34722.03 0.00 34722.03
> 25 2020781 1 4 5879307 0.00 175 0
> 249606 33596.04 0.00 33596.04
> 26 2018403 1 8 997676 0.00 30 0
> 46710 33255.87 0.00 33255.87
> 27 30134 1 1 4061564568 0.83 124035 124026
> 28903920 32745.31 0.00 451284952.00
> 28 2018264 1 8 641252 0.00 20 0
> 54720 32062.60 0.00 32062.60
> 29 17394 1 12 507772 0.00 16 16
> 61560 31735.75 31735.75 0.00
> 30 21288 1 8 2745335 0.00 87 87
> 71010 31555.57 31555.57 0.00
> 31 2018121 1 4 943150 0.00 30 0
> 56142 31438.33 0.00 31438.33
> 32 2014090 1 6 250596 0.00 8 0
> 65628 31324.50 0.00 31324.50
> 33 2007650 1 4 45356295 0.01 1455 0
> 4291452 31172.71 0.00 31172.71
> 34 31276 1 2 61704 0.00 2 0
> 31356 30852.00 0.00 30852.00
> 35 15468 1 13 29292 0.00 1 0
> 29292 29292.00 0.00 29292.00
> 36 2018581 1 2 875904 0.00 30 0
> 178812 29196.80 0.00 29196.80
> 37 2020791 1 2 4920368 0.00 175 0
> 225954 28116.39 0.00 28116.39
> 38 2016029 1 3 824358 0.00 30 0
> 36360 27478.60 0.00 27478.60
> 39 2020029 1 2 327394 0.00 12 0
> 47376 27282.83 0.00 27282.83
> 40 2012328 1 5 135298 0.00 5 0
> 33120 27059.60 0.00 27059.60
> 41 31274 1 1 1687170 0.00 63 0
> 155286 26780.48 0.00 26780.48
> 42 2019083 1 2 3530338 0.00 133 0
> 97164 26543.89 0.00 26543.89
> 43 31279 1 1 52524 0.00 2 0
> 26460 26262.00 0.00 26262.00
> 44 2014634 1 1 1757602 0.00 68 0
> 39690 25847.09 0.00 25847.09
> 45 2018295 1 3 900796 0.00 36 0
> 52560 25022.11 0.00 25022.11
> 46 2021245 1 4 747988 0.00 30 0
> 36090 24932.93 0.00 24932.93
> 47 24651 1 4 49284 0.00 2 0
> 24804 24642.00 0.00 24642.00
> 48 2020763 1 2 3023974 0.00 123 0
> 167220 24585.15 0.00 24585.15
> 49 2020800 1 2 3333830 0.00 136 0
> 87246 24513.46 0.00 24513.46
> 50 2020614 1 2 3913592 0.00 160 0
> 83772 24459.95 0.00 24459.95
> 51 2020609 1 4 3111426 0.00 130 0
> 89442 23934.05 0.00 23934.05
> 52 2019141 1 3 568974 0.00 24 0
> 28422 23707.25 0.00 23707.25
> 53 2019602 1 1 3171882 0.00 134 0
> 240822 23670.76 0.00 23670.76
> 54 2003287 1 6 466520 0.00 20 0
> 285516 23326.00 0.00 23326.00
> 55 2016922 1 10 3230312 0.00 139 0
> 91782 23239.65 0.00 23239.65
> 56 2020611 1 3 4594070 0.00 198 0
> 79056 23202.37 0.00 23202.37
> 57 17380 1 15 991624 0.00 43 43
> 59292 23061.02 23061.02 0.00
> 58 2020960 1 2 685418 0.00 30 0
> 30708 22847.27 0.00 22847.27
> 59 2018057 1 3 3583156 0.00 159 0
> 96030 22535.57 0.00 22535.57
> 60 2008782 1 5 2748390 0.00 122 0
> 69048 22527.79 0.00 22527.79
> 61 2020782 1 2 3130320 0.00 139 0
> 88110 22520.29 0.00 22520.29
> 62 2020613 1 3 3356494 0.00 150 0
> 82350 22376.63 0.00 22376.63
> 63 2020769 1 2 2636396 0.00 118 0
> 86958 22342.34 0.00 22342.34
> 64 2020586 1 3 2700166 0.00 122 0
> 90774 22132.51 0.00 22132.51
> 65 2020693 1 1 3049757 0.00 138 0
> 199368 22099.69 0.00 22099.69
> 66 2020799 1 2 3818200 0.00 173 0
> 120798 22070.52 0.00 22070.52
> 67 2006380 1 12 1300862 0.00 59 59
> 33912 22048.51 22048.51 0.00
> 68 2020786 1 2 3212030 0.00 146 0
> 101574 22000.21 0.00 22000.21
> 69 2017915 1 2 3046598 0.00 140 0
> 117576 21761.41 0.00 21761.41
> 70 2018880 1 2 3366284 0.00 155 0
> 94104 21717.96 0.00 21717.96
> 71 2020765 1 2 2808816 0.00 130 0
> 209520 21606.28 0.00 21606.28
> 72 2020784 1 2 2741601 0.00 127 0
> 95958 21587.41 0.00 21587.41
> 73 29189 1 1 1032558 0.00 48 0
> 33894 21511.62 0.00 21511.62
> 74 2020612 1 3 2967752 0.00 138 0
> 89262 21505.45 0.00 21505.45
> 75 2020773 1 2 3074056 0.00 144 0
> 83952 21347.61 0.00 21347.61
> 76 2017263 1 2 127458 0.00 6 0
> 23652 21243.00 0.00 21243.00
> 77 2018638 1 2 2883696 0.00 136 0
> 85752 21203.65 0.00 21203.65
> 78 2020766 1 2 2509209 0.00 119 0
> 211302 21085.79 0.00 21085.79
> 79 2018166 1 3 2357794 0.00 112 0
> 87714 21051.73 0.00 21051.73
> 80 2020795 1 2 2384326 0.00 114 0
> 84744 20915.14 0.00 20915.14
> 81 2020777 1 2 2078802 0.00 100 0
> 78840 20788.02 0.00 20788.02
> 82 2002878 1 8 41562 0.00 2 2
> 22698 20781.00 20781.00 0.00
> 83 2020798 1 2 2462538 0.00 119 0
> 81666 20693.60 0.00 20693.60
> 84 2021520 1 2 123524 0.00 6 0
> 27738 20587.33 0.00 20587.33
> 85 2017191 1 3 20466 0.00 1 0
> 20466 20466.00 0.00 20466.00
> 86 2017707 1 1 3006623 0.00 147 0
> 101628 20453.22 0.00 20453.22
> 87 2020606 1 4 3149168 0.00 154 0
> 199062 20449.14 0.00 20449.14
> 88 32986 1 1 81696 0.00 4 0
> 30438 20424.00 0.00 20424.00
> 89 2020793 1 2 2587716 0.00 127 0
> 221544 20375.72 0.00 20375.72
> 90 2020783 1 2 2678856 0.00 133 0
> 95346 20141.77 0.00 20141.77
> 91 2018153 1 4 1965170 0.00 98 0
> 81612 20052.76 0.00 20052.76
> 92 2020780 1 2 2449289 0.00 123 0
> 94428 19912.92 0.00 19912.92
> 93 2021065 1 2 2663188 0.00 134 0
> 205596 19874.54 0.00 19874.54
> 94 2020764 1 2 2873784 0.00 145 0
> 80622 19819.20 0.00 19819.20
> 95 2020694 1 1 2533778 0.00 128 0
> 89424 19795.14 0.00 19795.14
> 96 32396 1 2 39582 0.00 2 0
> 22158 19791.00 0.00 19791.00
> 97 2020770 1 2 2354850 0.00 119 0
> 95760 19788.66 0.00 19788.66
> 98 2016567 1 6 19674 0.00 1 0
> 19674 19674.00 0.00 19674.00
> 99 2021381 1 7 1075986 0.00 55 4
> 62748 19563.38 59044.50 16466.82
> 100 2020691 1 1 2385889 0.00 123 0
> 96552 19397.47 0.00 19397.47
>
> ############################################################################################################
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 4 & 5 in Barcelona: http://oisfevents.net
Can you please post your suricata.log using pastebin or alike?
Please add "-v" to your start line.
What is the output of -
modinfo pf_ring && cat /proc/net/pf_ring/info
?
Thank you
--
Regards,
Peter Manev
More information about the Oisf-users
mailing list