[Discussion] Features

Thu Oct 23 00:04:01 UTC 2008

Back to the idea of Spam-assassin scoring:

Once a bad host is identified, then I'm wondering if IP reputation could
maybe using a "halo effect" whereby other IPs by the same provider are given
lower scores.

Say reputation is from 0-100 (with 100 being bad)

So if an IP on hosting provider "Btrivo" contains malware, that IP gets a
reputation score of 50.
Adjacent IPs are given a reputation score of 30.
All of the IPs in "Btrivo's" netblock are given a reputation score of 10.

Two adjacent IP addresses each hosting malware each will get a score of 90
(50+30+10).

If malware goes away, score decreases each day. For each day host is still
up, score increases....

On Mon, Oct 20, 2008 at 1:48 PM, Matt Jonkman <jonkman at jonkmans.com> wrote:

> I agree. I know a lot of installs that use rotating tcpdump files which
> allows the analyst to go back and see everything that happened related
> to an incident. And as long as you have good disks it's a very feasible
> thing to do. If you're running inline you can even do the detection from
> these files to avoid packet loss at peak traffic flows.
>
> Definitely adding this to the list!
>
> Matt
>
> Jeremy wrote:
> > Thought of another feature I would like to see....
> >
> > [not sure of the number]
> > I would like to see a single capture engine have the ability to
> > perform the analysis of the data (however that may be), and also this
> > same engine should be able to capture session statistics (flow data),
> > and full packet captures.  What I mean by this is that currently I run
> > 3 applications on all of my IDS' to get this data.  I use SANCP for
> > session data, Tcpdump for full packet capture and snort for IDS
> > functionality.  I believe it would improve overall system performance
> > to only have one application read the packets from the capture library
> > such as libpcap instead of several applications having to read this
> > data.  I know that I ran into a wall with the 32 bit architecture in
> > that I could only allocate small amounts of Low memory space to each
> > application, as the kernel can only be allocated 1 GB.  This 1 GB has
> > to be divided up between with these three applications and well all
> > other applications.  I recently was told you could actually allocate 2
> > GB low memory on a 32 bit architecture, but as of today I have not
> > figured out how.  If you over run the low memory space reserved by the
> > kernel you end up with oom kills which can be really frustrating as it
> > will kill off applications based off some really odd/cryptic
> > algorithm.  Anyways getting back to one application performing all
> > three tasks, the packet only needs to be copied once and therefore I
> > could allocate a substantial amount of the low memory reserved by the
> > kernel to this application.  Now I know not everyone will want to
> > capture flow data, and full packet data on their IDS boxes, but if
> > this was a configurable option it may help some of us.  I also noticed
> > someone else posted about SANCP indexing abilities, this would be nice
> > to have in the new engine as well to make correlating Session data
> > (flow data) to full packet captures very fast and easy.  Having the
> > full packet captures on my IDS boxes has really helped in identifying
> > false positives and signature tweaking.  The session data (flow data)
> > has proven to be an outstanding addition to our anomaly detection
> > capabilities.
> >
> >
> > --jeremy
> >
> > On Sat, Oct 18, 2008 at 9:17 PM, th3 m0nq <th3m0nq at gmail.com> wrote:
> >> <snip>
> >> As Robert and Martin eluded in an other
> >> thread, any sort of correlation and IP analysis, scoring,
> >> severity-bumping, etc, is best done in the analysis or portal engine.
> >> </snip>
> >>
> >> I agree here, but does next generation mean an advanced version of
> >> snort?  Does next generation mean an evolution in terms of thinking
> >> about IDS/IPS?  Do we leave things as they are and say that there
> >> exist NIDs, DIDs, HIDs, and they don't form a cohesive immune system
> >> for my network?  I would like to see an evolution of such technologies
> >> be more biological in nature.
> >>
> >> <snip>
> >> I don't think any existing detection engine of an IDS is the proper
> place.
> >> But perhaps a parallel engine that collects network stream data and does
> >> matching on hostile IP's, and then checks (perhaps by packet/session
> >> timestamp) if the IDS did produce an alert in a recent time window
> >> (since the session-IP-eval-engine will surely lag behind)
> >> </snip>
> >>
> >> I am with you here as well.  Why is the parallel engine not part of
> >> the IDS?  Why should next generation IDS mean a better snort?  Why
> >> can't it mean a better way of looking at network security?
> >> http://en.wikipedia.org/wiki/Immune_system#Layered_defense  What about
> >> an all encompassing IDS/IPS _system_?
> >>
> >> I am not saying I have the answers, or something similar to an expert
> >> system is the answer.  I am saying I think the issue should be looked
> >> at as we dump the current way we do things in a temporary holding
> >> cell, and think in evolutionary terms.  This makes 4 cents ;-).
> >>
> >>
> >> Adrian
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Sat, Oct 18, 2008 at 5:01 PM, Frank Knobbe <frank at knobbe.us> wrote:
> >>> On Sat, 2008-10-18 at 16:28 -0500, th3 m0nq wrote:
> >>>> [..] I think there is
> >>>> room, at minimum, to have packets that do not necessarily trigger a
> >>>> rule "flagged" in some way in an IDS.  This could be implemented as a
> >>>> plug-in component of the system.  This may be way outside of the scope
> >>>> of what you guys are looking at, but that's my 2 cents.
> >>> Receiving alerts (with session content) on IP's that don't trip
> existing
> >>> signatures, but are present in a hostile-IP list, is a good idea (for
> >>> example, to detect new evasion techniques of existing badware).
> >>>
> >>> However, I don't think the sensor is the proper place for that due to
> >>> the immense IP-matching workload that would have to occur on a
> >>> per-packet or per-session basis (remember, we're talking at least tens
> >>> of thousands hostile IP's). As Robert and Martin eluded in an other
> >>> thread, any sort of correlation and IP analysis, scoring,
> >>> severity-bumping, etc, is best done in the analysis or portal engine.
> >>> (We do those things in our shop as well, with back-end scripts and our
> >>> portal).
> >>>
> >>> While you can make decisions on the IP "badness value" for analysis,
> you
> >>> would of course miss the content of the IP not receiving an alert. I
> >>> don't think any existing detection engine of an IDS is the proper
> place.
> >>> But perhaps a parallel engine that collects network stream data and
> does
> >>> matching on hostile IP's, and then checks (perhaps by packet/session
> >>> timestamp) if the IDS did produce an alert in a recent time window
> >>> (since the session-IP-eval-engine will surely lag behind). That check
> >>> would probably be best done against the IDS/portal database. A simple
> >>> bpf filter with hostile 20,000+ IP's to capture content will probably
> >>> not cut it :)
> >>>
> >>> -Frank
> >>>
> >>>
> >>> --
> >>> It is said that the Internet is a public utility. As such, it is best
> >>> compared to a sewer. A big, fat pipe with a bunch of crap sloshing
> >>> against your ports.
> >>>
> >>>
> >> _______________________________________________
> >> Discussion mailing list
> >> Discussion at openinfosecfoundation.org
> >> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
> >>
> > _______________________________________________
> > Discussion mailing list
> > Discussion at openinfosecfoundation.org
> > http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>
> --
> --------------------------------------------
> Matthew Jonkman
> Emerging Threats
> Phone 765-429-0398
> Fax 312-264-0205
> http://www.emergingthreats.net
> --------------------------------------------
>
> PGP: http://www.jonkmans.com/mattjonkman.asc
>
>
> _______________________________________________
> Discussion mailing list
> Discussion at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/discussion/attachments/20081022/a63b7500/attachment-0002.html>