[Oisf-users] JSON output for alerts

[Victor Julien]

On 03/13/2013 05:33 PM, Brian Rectanus wrote:
> On Tue, Mar 12, 2013 at 12:16 PM, Victor Julien <victor at inliniac.net> wrote:
>> On 03/12/2013 02:37 PM, Brian Rectanus wrote:
>>> On Tuesday, March 12, 2013 at 4:06 AM, Victor Julien wrote:
>>>> On 03/11/2013 03:29 PM, Brian Rectanus wrote:
>>>>> On Thu, Mar 7, 2013 at 5:48 AM, Victor Julien <lists at inliniac.net> wrote:
>>>>>> On 03/07/2013 12:46 PM, Aaron Nikula wrote:
>>>>>>> All,
>>>>>>>
>>>>>>> I have been experimenting with the /files/-/json/./log /output and I
>>>>>>> like it a lot. I was wondering if I could also output regular alerts to
>>>>>>> a JSON format? If not, are there any plans to implement that feature?
>>>>>>
>>>>>> I'd like to do that as well. Have been thinking about adding this as an
>>>>>> option to all outputs. Care to open a feature ticket?
>>>>>
>>>>> FYI, take a look at yajl lib for JSON if your looking for a lib.
>>>>> Seems pretty nice and very light. We are using it in ironbee now.
>>>>>
>>>>> http://lloyd.github.com/yajl/
>>>>
>>>> Thanks Brian.
>>>>
>>>> We already use libjansson for the unix socket protocol, so using this
>>>> would require some refactoring. Do you think this yajl will bring big
>>>> benefits over libjansson?
>>>
>>> Depends on what you use it for. Yajl allows for stream based parsing
>>> directly into your own structures. No need to parse everything into a
>>> tree of nodes and then read through that. So, I think yajl is more
>>> efficiently using resources, but at the cost of some extra code
>>> complexity sue to having to write callbacks for each node type. We
>>> needed the streaming parser for potentially large json structures coming
>>> in chunks. Yajl is nice, but I think libjannson is as well. They just
>>> solve different needs. For instance you could build libjannson on top of
>>> yajl.
>>
>> The only performance critical use we have is generating JSON records,
>> not parsing them. The unix socket code that parses JSON records is async
>> and we expect low volume. Otherwise we will be using it to generate
>> output based on alerts, events. So on the output side it is critical
>> that it's fast.
> 
> If libjannson forces you to build a full in-memory json tree, then
> traverse the tree again to write it out, then yajl may be more
> performant as yajl can stream this to the output.

Good point. We'll have to figure this out. Thanks Brian.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------