[Discussion] Features

Mon Oct 20 17:48:52 UTC 2008

I agree. I know a lot of installs that use rotating tcpdump files which
allows the analyst to go back and see everything that happened related
to an incident. And as long as you have good disks it's a very feasible
thing to do. If you're running inline you can even do the detection from
these files to avoid packet loss at peak traffic flows.

Definitely adding this to the list!

Matt

Jeremy wrote:
> Thought of another feature I would like to see....
> 
> [not sure of the number]
> I would like to see a single capture engine have the ability to
> perform the analysis of the data (however that may be), and also this
> same engine should be able to capture session statistics (flow data),
> and full packet captures.  What I mean by this is that currently I run
> 3 applications on all of my IDS' to get this data.  I use SANCP for
> session data, Tcpdump for full packet capture and snort for IDS
> functionality.  I believe it would improve overall system performance
> to only have one application read the packets from the capture library
> such as libpcap instead of several applications having to read this
> data.  I know that I ran into a wall with the 32 bit architecture in
> that I could only allocate small amounts of Low memory space to each
> application, as the kernel can only be allocated 1 GB.  This 1 GB has
> to be divided up between with these three applications and well all
> other applications.  I recently was told you could actually allocate 2
> GB low memory on a 32 bit architecture, but as of today I have not
> figured out how.  If you over run the low memory space reserved by the
> kernel you end up with oom kills which can be really frustrating as it
> will kill off applications based off some really odd/cryptic
> algorithm.  Anyways getting back to one application performing all
> three tasks, the packet only needs to be copied once and therefore I
> could allocate a substantial amount of the low memory reserved by the
> kernel to this application.  Now I know not everyone will want to
> capture flow data, and full packet data on their IDS boxes, but if
> this was a configurable option it may help some of us.  I also noticed
> someone else posted about SANCP indexing abilities, this would be nice
> to have in the new engine as well to make correlating Session data
> (flow data) to full packet captures very fast and easy.  Having the
> full packet captures on my IDS boxes has really helped in identifying
> false positives and signature tweaking.  The session data (flow data)
> has proven to be an outstanding addition to our anomaly detection
> capabilities.
> 
> 
> --jeremy
> 
> On Sat, Oct 18, 2008 at 9:17 PM, th3 m0nq <th3m0nq at gmail.com> wrote:
>> <snip>
>> As Robert and Martin eluded in an other
>> thread, any sort of correlation and IP analysis, scoring,
>> severity-bumping, etc, is best done in the analysis or portal engine.
>> </snip>
>>
>> I agree here, but does next generation mean an advanced version of
>> snort?  Does next generation mean an evolution in terms of thinking
>> about IDS/IPS?  Do we leave things as they are and say that there
>> exist NIDs, DIDs, HIDs, and they don't form a cohesive immune system
>> for my network?  I would like to see an evolution of such technologies
>> be more biological in nature.
>>
>> <snip>
>> I don't think any existing detection engine of an IDS is the proper place.
>> But perhaps a parallel engine that collects network stream data and does
>> matching on hostile IP's, and then checks (perhaps by packet/session
>> timestamp) if the IDS did produce an alert in a recent time window
>> (since the session-IP-eval-engine will surely lag behind)
>> </snip>
>>
>> I am with you here as well.  Why is the parallel engine not part of
>> the IDS?  Why should next generation IDS mean a better snort?  Why
>> can't it mean a better way of looking at network security?
>> http://en.wikipedia.org/wiki/Immune_system#Layered_defense  What about
>> an all encompassing IDS/IPS _system_?
>>
>> I am not saying I have the answers, or something similar to an expert
>> system is the answer.  I am saying I think the issue should be looked
>> at as we dump the current way we do things in a temporary holding
>> cell, and think in evolutionary terms.  This makes 4 cents ;-).
>>
>>
>> Adrian
>>
>>
>>
>>
>>
>>
>>
>>
>> On Sat, Oct 18, 2008 at 5:01 PM, Frank Knobbe <frank at knobbe.us> wrote:
>>> On Sat, 2008-10-18 at 16:28 -0500, th3 m0nq wrote:
>>>> [..] I think there is
>>>> room, at minimum, to have packets that do not necessarily trigger a
>>>> rule "flagged" in some way in an IDS.  This could be implemented as a
>>>> plug-in component of the system.  This may be way outside of the scope
>>>> of what you guys are looking at, but that's my 2 cents.
>>> Receiving alerts (with session content) on IP's that don't trip existing
>>> signatures, but are present in a hostile-IP list, is a good idea (for
>>> example, to detect new evasion techniques of existing badware).
>>>
>>> However, I don't think the sensor is the proper place for that due to
>>> the immense IP-matching workload that would have to occur on a
>>> per-packet or per-session basis (remember, we're talking at least tens
>>> of thousands hostile IP's). As Robert and Martin eluded in an other
>>> thread, any sort of correlation and IP analysis, scoring,
>>> severity-bumping, etc, is best done in the analysis or portal engine.
>>> (We do those things in our shop as well, with back-end scripts and our
>>> portal).
>>>
>>> While you can make decisions on the IP "badness value" for analysis, you
>>> would of course miss the content of the IP not receiving an alert. I
>>> don't think any existing detection engine of an IDS is the proper place.
>>> But perhaps a parallel engine that collects network stream data and does
>>> matching on hostile IP's, and then checks (perhaps by packet/session
>>> timestamp) if the IDS did produce an alert in a recent time window
>>> (since the session-IP-eval-engine will surely lag behind). That check
>>> would probably be best done against the IDS/portal database. A simple
>>> bpf filter with hostile 20,000+ IP's to capture content will probably
>>> not cut it :)
>>>
>>> -Frank
>>>
>>>
>>> --
>>> It is said that the Internet is a public utility. As such, it is best
>>> compared to a sewer. A big, fat pipe with a bunch of crap sloshing
>>> against your ports.
>>>
>>>
>> _______________________________________________
>> Discussion mailing list
>> Discussion at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>>
> _______________________________________________
> Discussion mailing list
> Discussion at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion

-- 
--------------------------------------------
Matthew Jonkman
Emerging Threats
Phone 765-429-0398
Fax 312-264-0205
http://www.emergingthreats.net
--------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc