[Discussion] Features

[Jeremy]

Thought of another feature I would like to see....

[not sure of the number]
I would like to see a single capture engine have the ability to
perform the analysis of the data (however that may be), and also this
same engine should be able to capture session statistics (flow data),
and full packet captures.  What I mean by this is that currently I run
3 applications on all of my IDS' to get this data.  I use SANCP for
session data, Tcpdump for full packet capture and snort for IDS
functionality.  I believe it would improve overall system performance
to only have one application read the packets from the capture library
such as libpcap instead of several applications having to read this
data.  I know that I ran into a wall with the 32 bit architecture in
that I could only allocate small amounts of Low memory space to each
application, as the kernel can only be allocated 1 GB.  This 1 GB has
to be divided up between with these three applications and well all
other applications.  I recently was told you could actually allocate 2
GB low memory on a 32 bit architecture, but as of today I have not
figured out how.  If you over run the low memory space reserved by the
kernel you end up with oom kills which can be really frustrating as it
will kill off applications based off some really odd/cryptic
algorithm.  Anyways getting back to one application performing all
three tasks, the packet only needs to be copied once and therefore I
could allocate a substantial amount of the low memory reserved by the
kernel to this application.  Now I know not everyone will want to
capture flow data, and full packet data on their IDS boxes, but if
this was a configurable option it may help some of us.  I also noticed
someone else posted about SANCP indexing abilities, this would be nice
to have in the new engine as well to make correlating Session data
(flow data) to full packet captures very fast and easy.  Having the
full packet captures on my IDS boxes has really helped in identifying
false positives and signature tweaking.  The session data (flow data)
has proven to be an outstanding addition to our anomaly detection
capabilities.


--jeremy

On Sat, Oct 18, 2008 at 9:17 PM, th3 m0nq <th3m0nq at gmail.com> wrote:
> <snip>
> As Robert and Martin eluded in an other
> thread, any sort of correlation and IP analysis, scoring,
> severity-bumping, etc, is best done in the analysis or portal engine.
> </snip>
>
> I agree here, but does next generation mean an advanced version of
> snort?  Does next generation mean an evolution in terms of thinking
> about IDS/IPS?  Do we leave things as they are and say that there
> exist NIDs, DIDs, HIDs, and they don't form a cohesive immune system
> for my network?  I would like to see an evolution of such technologies
> be more biological in nature.
>
> <snip>
> I don't think any existing detection engine of an IDS is the proper place.
> But perhaps a parallel engine that collects network stream data and does
> matching on hostile IP's, and then checks (perhaps by packet/session
> timestamp) if the IDS did produce an alert in a recent time window
> (since the session-IP-eval-engine will surely lag behind)
> </snip>
>
> I am with you here as well.  Why is the parallel engine not part of
> the IDS?  Why should next generation IDS mean a better snort?  Why
> can't it mean a better way of looking at network security?
> http://en.wikipedia.org/wiki/Immune_system#Layered_defense  What about
> an all encompassing IDS/IPS _system_?
>
> I am not saying I have the answers, or something similar to an expert
> system is the answer.  I am saying I think the issue should be looked
> at as we dump the current way we do things in a temporary holding
> cell, and think in evolutionary terms.  This makes 4 cents ;-).
>
>
> Adrian
>
>
>
>
>
>
>
>
> On Sat, Oct 18, 2008 at 5:01 PM, Frank Knobbe <frank at knobbe.us> wrote:
>> On Sat, 2008-10-18 at 16:28 -0500, th3 m0nq wrote:
>>> [..] I think there is
>>> room, at minimum, to have packets that do not necessarily trigger a
>>> rule "flagged" in some way in an IDS.  This could be implemented as a
>>> plug-in component of the system.  This may be way outside of the scope
>>> of what you guys are looking at, but that's my 2 cents.
>>
>> Receiving alerts (with session content) on IP's that don't trip existing
>> signatures, but are present in a hostile-IP list, is a good idea (for
>> example, to detect new evasion techniques of existing badware).
>>
>> However, I don't think the sensor is the proper place for that due to
>> the immense IP-matching workload that would have to occur on a
>> per-packet or per-session basis (remember, we're talking at least tens
>> of thousands hostile IP's). As Robert and Martin eluded in an other
>> thread, any sort of correlation and IP analysis, scoring,
>> severity-bumping, etc, is best done in the analysis or portal engine.
>> (We do those things in our shop as well, with back-end scripts and our
>> portal).
>>
>> While you can make decisions on the IP "badness value" for analysis, you
>> would of course miss the content of the IP not receiving an alert. I
>> don't think any existing detection engine of an IDS is the proper place.
>> But perhaps a parallel engine that collects network stream data and does
>> matching on hostile IP's, and then checks (perhaps by packet/session
>> timestamp) if the IDS did produce an alert in a recent time window
>> (since the session-IP-eval-engine will surely lag behind). That check
>> would probably be best done against the IDS/portal database. A simple
>> bpf filter with hostile 20,000+ IP's to capture content will probably
>> not cut it :)
>>
>> -Frank
>>
>>
>> --
>> It is said that the Internet is a public utility. As such, it is best
>> compared to a sewer. A big, fat pipe with a bunch of crap sloshing
>> against your ports.
>>
>>
> _______________________________________________
> Discussion mailing list
> Discussion at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>