[Oisf-users] JSON output for alerts

[Brian Rectanus]

On Tue, Mar 12, 2013 at 12:16 PM, Victor Julien <victor at inliniac.net> wrote:
> On 03/12/2013 02:37 PM, Brian Rectanus wrote:
>> On Tuesday, March 12, 2013 at 4:06 AM, Victor Julien wrote:
>>> On 03/11/2013 03:29 PM, Brian Rectanus wrote:
>>>> On Thu, Mar 7, 2013 at 5:48 AM, Victor Julien <lists at inliniac.net> wrote:
>>>>> On 03/07/2013 12:46 PM, Aaron Nikula wrote:
>>>>>> All,
>>>>>>
>>>>>> I have been experimenting with the /files/-/json/./log /output and I
>>>>>> like it a lot. I was wondering if I could also output regular alerts to
>>>>>> a JSON format? If not, are there any plans to implement that feature?
>>>>>
>>>>> I'd like to do that as well. Have been thinking about adding this as an
>>>>> option to all outputs. Care to open a feature ticket?
>>>>
>>>> FYI, take a look at yajl lib for JSON if your looking for a lib.
>>>> Seems pretty nice and very light. We are using it in ironbee now.
>>>>
>>>> http://lloyd.github.com/yajl/
>>>
>>> Thanks Brian.
>>>
>>> We already use libjansson for the unix socket protocol, so using this
>>> would require some refactoring. Do you think this yajl will bring big
>>> benefits over libjansson?
>>
>> Depends on what you use it for. Yajl allows for stream based parsing
>> directly into your own structures. No need to parse everything into a
>> tree of nodes and then read through that. So, I think yajl is more
>> efficiently using resources, but at the cost of some extra code
>> complexity sue to having to write callbacks for each node type. We
>> needed the streaming parser for potentially large json structures coming
>> in chunks. Yajl is nice, but I think libjannson is as well. They just
>> solve different needs. For instance you could build libjannson on top of
>> yajl.
>
> The only performance critical use we have is generating JSON records,
> not parsing them. The unix socket code that parses JSON records is async
> and we expect low volume. Otherwise we will be using it to generate
> output based on alerts, events. So on the output side it is critical
> that it's fast.

If libjannson forces you to build a full in-memory json tree, then
traverse the tree again to write it out, then yajl may be more
performant as yajl can stream this to the output.

-B