[Oisf-users] meta-data crashes

[Jeremy A. Grove]

Hello All, 

We use Suricata in a variety of situations with varying amounts of data input. The version that is currently being used is Suricata version 4.1.0-beta1 and we have upgraded a portion to Suricata version 4.1.0. In both versions I am running into an issue with the meta data where it will stop logging entirely or it will only log some of the protocols. This is while Suricata is still running. 

An idea of the configuration: 

- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/dns.json 
pcap-file: false 
types: 
- dns: 
version: 2 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/tls.json 
pcap-file: false 
types: 
- tls: 
extended: yes # enable this for extended logging information 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/files.json 
pcap-file: false 
types: 
- files: 
force-magic: no # force logging magic on all logged files 
force-hash: [md5] # force logging of md5 checksums 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/http.json 
pcap-file: false 
types: 
- http: 
extended: yes # enable this for extended logging information 
# custom allows additional http fields to be included in eve-log 
# the example below adds three additional fields when uncommented 
custom: [accept, accept-charset, accept-encoding, accept-language, 
accept-datetime, authorization, cache-control, cookie, from, 
max-forwards, origin, pragma, proxy-authorization, range, te, via, 
x-requested-with, dnt, x-forwarded-proto, accept-range, age, 
allow, connection, content-encoding, content-language, 
content-length, content-location, content-md5, content-range, 
content-type, date, etags, last-modified, link, location, 
proxy-authenticate, referrer, refresh, retry-after, server, 
set-cookie, trailer, transfer-encoding, upgrade, vary, warning, 
www-authenticate, x-flash-version, x-authenticated-user] 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/ssh.json 
pcap-file: false 
types: 
- ssh 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/smtp.json 
pcap-file: false 
types: 
- smtp: 
extended: yes # enable this for extended logging information 
custom: [received, x-mailer, x-originating-ip, relays, reply-to, bcc] 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/flow.json 
pcap-file: false 
types: 
- flow 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/nfs.json 
pcap-file: false 
types: 
- nfs 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/smb.json 
pcap-file: false 
types: 
- smb 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/tftp.json 
pcap-file: false 
types: 
- tftp 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/ikev2.json 
pcap-file: false 
types: 
- ikev2 
- eve-log: 
enabled: yes 
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis 
filename: /var/log/suricata/flows/current/dhcp.json 
pcap-file: false 
types: 
- dhcp 

I am happy to provide more detail to anyone willing to give an opinion on how this may be addressed but I am not sure exactly what is useful to know. 

Has anyone else seen this behavior? This is a critical piece for us and I will need to switch back to Bro/Zeek until I can get this resolved. 

Regards, 

Jeremy Grove, SSCP 
Security Engineer 
Quadrant Information Security 


Learn more= about our managed SIEM [ https://a.quadrantsec.com/3D%22https://quadrantsec.com/SaganMSSP%22 | people + product ] 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20181129/d1867cdc/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2131 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20181129/d1867cdc/attachment.bin>