[Oisf-users] eve.json parser

[conf file]

Victor,

Thank you for your response!

So if I am following you correctly, to make the "flow" look correct in the
PCAP I might have to swap the source / destination information in the
headers? That shouldn't be all that difficult since the headers in each
packet field should be very predictable, correct? I think the more
difficult problem to solve is going to be figuring out where one packet
payload stops and another stops. Are they separated at all? Any
help/information would be appreciated!


Thanks!

On Thu, Feb 11, 2016 at 2:41 PM, Victor Julien <lists at inliniac.net> wrote:

> On 11-02-16 20:30, conf file wrote:
> > I am in the process of writing a parser for the eve.json log. So far
> > here's what I've accomplished:
> >
> > 1) Create PCAPs from the packet field
> > 2) Create CEF formatted events to forward off to a SIEM
> >
> > All of the above is working as expected and works quite well. What I
> > would like to do in addition is attach the payload field to the already
> > created PCAP. Since the payload field contains no header information I
> > would like to pull the headers off of the packet field and reuse them
> > with the payload field. Meaning, I plan to use unpack to pull ~60B off
> > of the packet and insert it as a new packet into the PCAP. For
> > reference, here's the PCAP format:
> >
> > Global PCAP header: 24 bytes (0-24) (magic bytes, PCAP version, snap
> > len, link type)
> > PCAP header: 16 bytes (25-40) -- timestamp in epoch (4B),
> > microseconds(4B), packet len (4B), packet len (4B)
> > Ethernet, ip, tcp/udp/L5, data (JSON packet field)
> > PCAP header: 16 bytes (25-40) -- timestamp in epoch (4B),
> > microseconds(4B), packet len (4B), packet len (4B)
> > Ethernet, ip, tcp/udp/L5, data (headers from the packet field + the
> > payload field)
> > ...
> >
> > I have uploaded the script here:http://www.magikman.net/parseLog.txt
> >
> > I would really appreciate input from the list. Specifically I am looking
> > for ideas / guidance on how best to make use of the payload data. Also,
> > what data can I expect to find in the payload field?
>
> Payload is either a single packet's payload (data portion), or a
> reassembled stream payload (when stream==1). The latter can be quite
> big, as we simply write out whatever payload data for that session we
> have in memory. It is ordered. Looks like the max size of this field is
> currently 4k.
>
> Especially in IDS mode the 'packet' won't match the 'payload', at least
> not for TCP. This is because we inspect data when it's ack'd. So for the
> pcap to make sense, you may have to adjust the IP and TCP headers.
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> Suricata User Conference November 9-11 in Washington, DC:
> http://oisfevents.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20160211/7355ef41/attachment-0002.html>