[Oisf-users] HTTP Logging Update

[Peter Manev]

On Thu, Jun 5, 2014 at 5:30 PM, Adnan Baykal <abaykal at gmail.com> wrote:
> Here is what I did. I found a top talker - video streaming - and put a bpf
> filter to filter it out (not (host 1.2.3.4)). I am not dropping as many
> packets any more (about 3%-4%).
>
> however, I still see extremely low number of http entries in the http log
> and I dont see anything when I take out the midstream and async entries from
> the yaml file.
>

Do you have VLANs on the mirror port?


>
>
> On Wed, Jun 4, 2014 at 8:14 PM, Adnan Baykal <abaykal at gmail.com> wrote:
>>
>> Mbit
>>
>>
>> On Wed, Jun 4, 2014 at 4:38 PM, Peter Manev <petermanev at gmail.com> wrote:
>>>
>>> On Wed, Jun 4, 2014 at 10:33 PM, Adnan Baykal <abaykal at gmail.com> wrote:
>>> > I do load about 7K rules.  I need to go back to my sensor but it is
>>> > probably
>>> > around 800MB/s
>>> >
>>> >
>>>
>>> Just to confirm - is that 800 Mbit or MByte?
>>>
>>>
>>> > On Wed, Jun 4, 2014 at 4:17 PM, Peter Manev <petermanev at gmail.com>
>>> > wrote:
>>> >>
>>> >> On Wed, Jun 4, 2014 at 10:08 PM, Adnan Baykal <abaykal at gmail.com>
>>> >> wrote:
>>> >> > I have been having no HTTP logging at all on one of my sensors. I
>>> >> > have
>>> >> > posted several questions to this blog. Mind you that this sensor
>>> >> > does
>>> >> > drop
>>> >> > significant amount of data (about 50%) and I do understand that
>>> >> > there
>>> >> > will
>>> >> > be a lot of http traffic missed due to drops but not having any
>>> >> > entry in
>>> >> > the
>>> >> > http.log file was concerning. I thought I would at least see some
>>> >> > entries.
>>> >> >
>>> >> > This morning, I found a setting:
>>> >> >
>>> >> >   midstream: true             # do not allow midstream session
>>> >> > pickups
>>> >> >   async_oneside: true         # do not enable async stream handling
>>> >> >
>>> >> > When above setting is applied to the stream, I get limited HTTP log.
>>> >> > My
>>> >> > question is "can this change in behavior be explained by dropped
>>> >> > packets"?
>>> >> > does this change further support the theory that this box is
>>> >> > significantly
>>> >> > undersized and that the bigger box would operate normally with full
>>> >> > http
>>> >> > traffic?
>>> >> >
>>> >> > I am in the process of upgrading this sensor to a 32GB 20 Core
>>> >> > system
>>> >> > (it is
>>> >> > currently 16GB 8 Core).
>>> >> >
>>> >> > Thanks,
>>> >> >
>>> >> > --Adnan
>>> >> >
>>> >> >
>>> >> > _______________________________________________
>>> >> > Suricata IDS Users mailing list:
>>> >> > oisf-users at openinfosecfoundation.org
>>> >> > Site: http://suricata-ids.org | Support:
>>> >> > http://suricata-ids.org/support/
>>> >> > List:
>>> >> > https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>> >> > OISF: http://www.openinfosecfoundation.org/
>>> >>
>>> >> In general if you have significant % of drops  - you will be missing a
>>> >> lot of logs.
>>> >> How much traffic do you inspect with that set up? (and how many rules
>>> >> do you load?)
>>> >>
>>> >>
>>> >> --
>>> >> Regards,
>>> >> Peter Manev
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Peter Manev
>>
>>
>



-- 
Regards,
Peter Manev