[Oisf-users] Correlating http transactions and alert logs

[Victor Julien]

On 07/11/2014 11:09 AM, Darren Spruell wrote:
> Request in same vein as
> https://lists.openinfosecfoundation.org/pipermail/oisf-users/2014-January/003281.html
> 
> I'm embarking on a task with Suricata where I'd need to specifically
> correlate alerts to HTTP requests - namely to associate the set of
> alerts that fire in connection with the HTTP transaction under
> inspection. Currently Suricata logging is the only data source
> available to do this for the environment, and I'd prefer to use EVE
> logging.
> 
> If I understand correctly Suricata can generate debug alerting with
> knowledge of the transaction being handled at the moment of alert.
> I've peeked at the debug log output and it is not appropriate for our
> use case. Is it possible to have Suricata generate http logs (maybe
> other service logs too?) that include data about any alerts that were
> generated inspecting that transaction? I want to make sure we have a
> strong correlator on these events, so attempting to match logs up by
> timestamp or similar is potentially impractical or at the least
> awkward and potentially fraught with #fail.
> 
> Alternatively, is it possible to tag the transaction ID at logging
> time into both the alert log and the http log that would allow
> correlation on the two in a datastore?

In my flow-log branch I have also added a simple 'flow id', which is a
tag added to each alert, http, etc record that is connected to a flow.
So this would be a step in the right direction. If you need a tx
specific tag... I guess it would be enough to log the tx number where
available in both http and alert. This is just a simple incrementing
counter that starts at 0 for the first tx. The flow_id+tx_cnt would be
quite unique.

This is all about the json output btw.

Would this be helpful?

> Or is there another way of accomplishing this?

Approx timestamp + 5tulpe is usually good, but it requires a bit of
scripting.

> By the way, absolutely gleeful about this performance:
> 
> 9/7/2014 -- 19:17:33 - <Notice> - Stats for 'bond1':  pkts:
> 18668355600, drop: 50066 (0.00%), invalid chksum: 13028
> 11/7/2014 -- 00:52:46 - <Notice> - Stats for 'bond1':  pkts:
> 11102827636, drop: 41230 (0.00%), invalid chksum: 10322

Nice!

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------