[Oisf-users] Suricata 4.0.0 EVE output vs Suricata 3.2.0

[Jason Ish]

Hi Tom,

On 2017-08-02 12:27 PM, Tom Peterson wrote:
> Hi all,
> 
> Thanks for all the hard work on Suricata! I've found it quite easy to 
> build, install, and configure on CentOS which I'm sure took a lot of 
> work and it's just been very user friendly to work with!
> 
> I'm looking at Suricata 4.0.0 and the EVE output that it generates and 
> I'm noticing some differences from Suricata 3.2.0. For example I'm 
> looking at the following rule:
> 
> ----------------------------------------------
> alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"ET TROJAN Windows 
> executable base64 encoded";
>   flow: established,from_server;
>   file_data;
>   content:"TVqQA";
>   pcre:"/^[A-Za-z0-9]{3}(?:[A-Za-z0-9+/]{4}|\s){100}/Rs";
>   reference:md5,49aca228674651cba776be727bdb7e60;
>   classtype:trojan-activity;
>   sid:2018856;
>   rev:10;
> )
> ----------------------------------------------
> 
> In Suricata 4.0.0 I'm noticing that the 'pcap_cnt' and 'payload' values 
> are no longer in the EVE json report.
> 
> My guess is that this has something to do with the 'file_data' keyword 
> in the rule and the new http and metadata logging options in Suricata 
> 4.0.0. I've played with these options a bit but I haven't found a way to 
> get to these two values in the EVE output.
> 
> I'm still investigating this but was wondering if anyone could help me 
> understand this difference.
> 
> I believe the relevant portion of the eve-log config is:
> 
> Suricata 3.2.0:
> 
> ----------------------------------------------
>        types:
>          - alert:
>              payload: yes             # enable dumping payload in Base64
>              # payload-buffer-size: 4kb # max size of payload buffer to 
> output in eve-log
>              # payload-printable: yes   # enable dumping payload in 
> printable (lossy) format
>              # packet: yes              # enable dumping of packet 
> (without stream segments)
>              http: no                # enable dumping of http fields
>              tls: no                 # enable dumping of tls fields
>              ssh: no                 # enable dumping of ssh fields
>              smtp: no                # enable dumping of smtp fields
>              dnp3: no                # enable dumping of DNP3 fields
> ----------------------------------------------
> 
> Suricata 4.0.0:
> 
> ----------------------------------------------
>        types:
>          - alert:
>              payload: yes             # enable dumping payload in Base64
>              # payload-buffer-size: 4kb # max size of payload buffer to 
> output in eve-log
>              # payload-printable: yes   # enable dumping payload in 
> printable (lossy) format
>              # packet: yes              # enable dumping of packet 
> (without stream segments)
>              http-body: no           # enable dumping of http body in Base64
>              http-body-printable: no # enable dumping of http body in 
> printable format
>              metadata: no              # add L7/applayer fields, flowbit 
> and other vars to the alert
> ----------------------------------------------
> 
> Right now I've generated alerts for all of the malware-traffic-analysis 
> pcaps with the ET Open rules on Suricata 3.2.0 and 4.0.0 and going 
> through all of the differences in the EVE output. If anyone is 
> interested I can share my other findings so far.

In all my test cases I have pcap_cnt, and payload. Are you able to 
bundle up a test case that shows this? Rule, pcap, and probably your 
suricata.yaml?

Thanks,
Jason