[Oisf-users] Eve logs missing payload and payload_printable for http proto alerts

[Eric Urban]

Hello Peter,

Since the time when I sent that message, we have performed two separate
upgrades.  First to 4.0.6 then to 4.1.2.  I looked back at our log data and
the day when we upgraded to 4.0.6 the cases of alerts for that signature
with no payload went from over 50% down to around 1% or less depending on
the day.  We had roughly 30K alerts for this particular rule over a month
and having around 1% of these cases isn't enough that we notice it anymore.

I looked at the 4.0.6 release notes and saw support issue #2512, which
dealt with truncation in some http_method/user_agent fields, so doesn't
seem to exactly describe this issue but possibly was the same cause?  I
also saw libhtp was upgraded so maybe one of those two changes fixed it.

Thank you for getting back to me and in the unlikely case we should see
this resurface I will try to get a capture.

-- 
Eric Urban
University Information Security | Office of Information Technology |
it.umn.edu
University of Minnesota | umn.edu
eurban at umn.edu


On Thu, Jan 17, 2019 at 6:26 AM Peter Manev <petermanev at gmail.com> wrote:

> On Thu, Nov 8, 2018 at 10:47 PM Eric Urban <eurban at umn.edu> wrote:
> >
> > I am wondering if anyone else has noticed the following behavior or can
> provide advice on what may be the cause of it?
> >
> > Over the last month and a half we have had a large number of alerts
> triggered from HTTP rules that have no payload and no payload_printable
> data present in the logged alert.  Both the fields and values in these
> alerts are absent from the EVE logs.
> >
> > This is happening mostly for Emerging Threats rule 2016683 (
> http://doc.emergingthreats.net/bin/view/Main/2016683) where about 50% of
> the alerts are missing payload/payload_printable data.  That rule has a
> content match in http_client_body so we would expect the traffic triggering
> the alert to have payload.  There are other HTTP rules (e.g. 2019182,
> 2011768) where we see missing payload/payload_printable as well but these
> do not have nearly as high of a percentage of alerts with this behavior.
> >
> > Something else worth noting is that we do have metadata logging enabled,
> and in about 25% of these cases there is HTTP metadata included for these
> alerts that are missing payload/payload_printable data.  I understand the
> metadata does not include payload info, but thought it was worth mentioning
> since other application layer logging is happening fine in some of these
> cases.
> >
> > Also, this behavior looks to have significantly increased after
> upgrading from 3.2.5 to 4.0.5.  I suppose it could be possible the type of
> traffic triggering these alerts is different so may be a red herring, but
> the difference is large enough that I feel it could be a factor.  I noticed
> too that in the 3.2.5 alert data we have that there are many cases where
> payload_printable is not present but payload is there.  In our 4.0.5 alert
> data, I was not able to find such a case.
> >
>
> If this is the case(such a large - 50% diff with that sig)  with 4.1.2
> would it be possible to reproduce it with a pcap ?
>
> Thank you
>
>
>
>
> --
> Regards,
> Peter Manev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20190117/4e3686ca/attachment-0001.html>