[Oisf-users] For those who haven't read this

Wed Jul 21 22:23:20 UTC 2010

It is not something it can do out of the box, but something that I was
looking at for the future.  A better example is offloading detection
to other hardware.  I believe this was a huge issue for the gnort
project as well.

-B

On Wed, Jul 21, 2010 at 3:13 PM, Robert Vineyard
<robert.vineyard at oit.gatech.edu> wrote:
> Interesting, I did not realize that was something that Suricata could do.
>
> Although, it does seem like that would preclude it from running in IPS mode
> since real-time analysis is essential for that kind of setup. However, for
> IDS mode, being able to fork off a separate thread/process to handle
> querying against an IP reputation database would be handy. I'm presently
> looking at ways to post-process the output of <insert favorite IDS here> for
> correlation against those types of data sources, but being able to do it
> natively inside the engine would be a big plus.
>
> I'm not trying to criticize here folks, just wanting to stir up the
> discussion a bit particularly in light of the flame war that seems to be
> starting over snort vs. suricata. I think OISF has the right idea and of
> course an all-star cast of sponsors and developers (I'm especially excited
> about the HTP library's application-layer inspection capabilities). Also I
> wasn't trying to espouse the theory of single-threading vs. multi-threading
> (I'm quite familiar with the implications of both design methodologies), but
> having worked with real-time bandwidth-intensive streaming video
> applications in the past in a parallel processing aspect, keeping everything
> synchronized without having to wait on one or two pieces of the puzzle is
> crucial to achieving real-time performance on these types of applications. I
> suppose I should take a closer look at the threading model implemented by
> Suricata, I was just curious if you guys had a writeup comparable to Marty's
> article on the subject. I can definitely see merit in both approaches, but
> parallel computing is clearly the way of the future now that CPU vendors are
> focusing more on core count than MHz...
>
> Just my 2c :-)
>
> --
> [ Robert Vineyard | RHCE, Security+ ]    [ robert.vineyard at oit.gatech.edu  ]
> [ Information Security Engineer III ]    [ 404.385.6900 | FAX 404.894.4690 ]
> [Finding a needle in a haystack isn't hard when every straw is computerized]
>
> On 7/21/2010 5:27 PM, Brian Rectanus wrote:
>>
>> On Wed, Jul 21, 2010 at 1:00 PM, Matt Jonkman<jonkman at jonkmans.com>
>>  wrote:
>>>
>>> On 7/21/10 2:58 PM, Robert Vineyard wrote:
>>>>
>>>>
>>>> http://vrt-sourcefire.blogspot.com/2010/06/single-threaded-data-processing.html
>>>>
>>>> Here's another link (a writeup by Marty Roesch of Sourcefire) referenced
>>>> from the article you mentioned.
>>>>
>>>> In light of this discussion, does OISF / Suricata have a response to
>>>> Sourcefire's critique of a multi-threaded engine model that uses several
>>>> threads to process the same data simultaneously? It seems to me that the
>>>> most efficient way to do things would be to have a front-end
>>>> load-balancer
>>>> that could distribute the traffic to multiple back-end threads or
>>>> processes
>>>> that would each operate on independent data streams. This is the
>>>> strategy
>>>> employed by Endace and others to accomplish high-throughput IDS
>>>> inspection.
>>>
>>> I'll leave this for Victor and Will to answer in detail, but that's
>>> about where we're going.
>>>
>>> Now nobody is saying that 4 cores gives you a 4-fold performance
>>> increase. It's something less than that. But it's our only way forward.
>>> The processors aren't going to get faster.
>>
>> Besides performance, one thing you cannot do with snort is send the
>> data off to be analyzed externally without blocking until you get a
>> response.  This is important in L7 if you want to send the data off to
>> a WAF module, or maybe send the data to a virus checker, etc.
>>
>> So multi-threaded is not all about performance, but also about being
>> able to do things that snort just cannot do.
>>
>> And comparing<  1yr old code to code as old as snort is a bit
>> unreasonable.
>>
>>>
>>>> On a related note, are there any plans to implement native acceleration
>>>> support for other vendors besides Endace (in particular Napatech /
>>>> nPulse)?
>>>>
>>>
>>> Yes! As Randy mentioned they're building it now. We'll have news about
>>> this very shortly as far as membership and support!
>>>
>>> Matt
>>>
>>>> Thanks!
>>>>
>>>> --
>>>> [ Robert Vineyard | RHCE, Security+ ]    [
>>>> robert.vineyard at oit.gatech.edu  ]
>>>> [ Information Security Engineer III ]    [ 404.385.6900 | FAX
>>>> 404.894.9548 ]
>>>> [Finding a needle in a haystack isn't hard when every straw is
>>>> computerized]
>>>>
>>>>
>>>> On 07/21/2010 07:05 AM, Kevin Ross wrote:
>>>>>
>>>>>
>>>>> http://vrt-sourcefire.blogspot.com/2010/07/innovation-you-keep-using-that-word.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Vrt+(Sourcefire+VRT+-+Vulnerability+Research%2C+Snort+Rules+and+Explosions)
>>>>>
>>>>> <http://vrt-sourcefire.blogspot.com/2010/07/innovation-you-keep-using-that-word.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Vrt+(Sourcefire+VRT+-+Vulnerability+Research%2C+Snort+Rules+and+Explosions)>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Oisf-users mailing list
>>>>> Oisf-users at openinfosecfoundation.org
>>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>>
>>>> _______________________________________________
>>>> Oisf-users mailing list
>>>> Oisf-users at openinfosecfoundation.org
>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>>> --
>>>
>>> ----------------------------------------------------
>>> Matthew Jonkman
>>> Emerging Threats
>>> Open Information Security Foundation (OISF)
>>> Phone 765-429-0398
>>> Fax 312-264-0205
>>> http://www.emergingthreats.net
>>> http://www.openinfosecfoundation.org
>>> ----------------------------------------------------
>>>
>>> PGP: http://www.jonkmans.com/mattjonkman.asc
>>> _______________________________________________
>>> Oisf-users mailing list
>>> Oisf-users at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
>>>
>