[Oisf-users] eve.json parser

[Victor Julien]

On 11-02-16 20:30, conf file wrote:
> I am in the process of writing a parser for the eve.json log. So far
> here's what I've accomplished:
> 
> 1) Create PCAPs from the packet field
> 2) Create CEF formatted events to forward off to a SIEM
> 
> All of the above is working as expected and works quite well. What I
> would like to do in addition is attach the payload field to the already
> created PCAP. Since the payload field contains no header information I
> would like to pull the headers off of the packet field and reuse them
> with the payload field. Meaning, I plan to use unpack to pull ~60B off
> of the packet and insert it as a new packet into the PCAP. For
> reference, here's the PCAP format:
> 
> Global PCAP header: 24 bytes (0-24) (magic bytes, PCAP version, snap
> len, link type)
> PCAP header: 16 bytes (25-40) -- timestamp in epoch (4B),
> microseconds(4B), packet len (4B), packet len (4B)
> Ethernet, ip, tcp/udp/L5, data (JSON packet field)
> PCAP header: 16 bytes (25-40) -- timestamp in epoch (4B),
> microseconds(4B), packet len (4B), packet len (4B)
> Ethernet, ip, tcp/udp/L5, data (headers from the packet field + the
> payload field)
> ...
> 
> I have uploaded the script here:http://www.magikman.net/parseLog.txt
> 
> I would really appreciate input from the list. Specifically I am looking
> for ideas / guidance on how best to make use of the payload data. Also,
> what data can I expect to find in the payload field?

Payload is either a single packet's payload (data portion), or a
reassembled stream payload (when stream==1). The latter can be quite
big, as we simply write out whatever payload data for that session we
have in memory. It is ordered. Looks like the max size of this field is
currently 4k.

Especially in IDS mode the 'packet' won't match the 'payload', at least
not for TCP. This is because we inspect data when it's ack'd. So for the
pcap to make sense, you may have to adjust the IP and TCP headers.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------