[Oisf-users] URL reputation?

[Matt Jonkman]

Ya, I've had that on my mind for a while, but I think the scale issues are
core.

We have IP rep now, and shortly DNS rep that can be applied. I think a good
number of url's can be knocked down with good domain rep.

But I also think this is worth exploring. I wonder if there are any
algorithms out there that could take a list of 200k url's and boil them
down to a set of core prequalifying strings minus the domains?

Or masking parameter values that vary in some automated way to get the
least number of matches required?

Matt


On Wed, Jan 30, 2013 at 10:29 AM, Victor Julien <lists at inliniac.net> wrote:

> On 01/30/2013 04:25 PM, Matt wrote:
> > Has anyone discussed URL reputation as a feature?  URL reputation is a
> > common offering for threat intelligence providers.  For instance, Phish
> > Tank publishes an open source feed here:
> > http://www.phishtank.com/developer_info.php.  I can pull that list and
> > turn it into 11k rules, but that doesn't seem optimal.  For larger
> > feeds, it isn't possible at all.  E.g. Symantec's Deep Sight feed has
> > over 150k URLs on it this morning.  I tried turning that into a rule
> > set, but the box ran out of memory trying to load it.
>
> We've been talking about it, just like other forms of reputation like
> dns. I suggest opening a feature ticket...
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Suricata IDS Users mailing list: oisf-users at openinfosecfoundation.org
> Site: http://suricata-ids.org | Support: http://suricata-ids.org/support/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-users
> OISF: http://www.openinfosecfoundation.org/
>



-- 


----------------------------------------------------
Matt Jonkman
Emerging Threats Pro
Open Information Security Foundation (OISF)
Phone 866-504-2523 x110
http://www.emergingthreatspro.com
http://www.openinfosecfoundation.org
----------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20130130/4b19dd7d/attachment.html>