[Oisf-wg-portscan] Hey
Breno Silva
breno.silva at gmail.com
Sun Oct 25 08:22:28 EST 2009
Hi Tomas,
I have a implementation of this in my internal and external network. However
my internal network is very big and my external is AS7738 and AS8167.
In both cases it is working well, almost wiithout false positives and good
rate of attack detection. But in both cases the traffic is highly random
(internal is less random) ... but working well
However i never tested it in a small LAN. If you have one i can send you a
code for testing.
What do you think ?
Thanks
Breno
On Sat, Oct 24, 2009 at 3:15 PM, Tomas L. Byrnes <tomb at byrneit.net> wrote:
> I think the problem with this is that you’re assuming that the PAYLOAD of
> traffic to a given port, especially UDP is highly random in the case of
> normal traffic, and highly self-similar in the case of (D)DOS.
>
>
>
> This is not true, especially for widely used services such as DNS. The vast
> majority of DNS packets are HIGHLY self-similar, especially the ones to/from
> Authoritative Nameservers, which are usually answering queries for the exact
> same RRSETs all the time.
>
>
>
>
>
>
>
> *From:* oisf-wg-portscan-bounces at openinfosecfoundation.org [mailto:
> oisf-wg-portscan-bounces at openinfosecfoundation.org] *On Behalf Of *Breno
> Silva
> *Sent:* Monday, October 12, 2009 8:58 AM
>
> *To:* DDoS and Portscan methods discussion
> *Subject:* Re: [Oisf-wg-portscan] Hey
>
>
>
> Hi Guys
>
>
>
> Good to hear from you.
>
> I´m sending two simples codes for discussion if it can be used
> as a part of future ddos detection engine.
>
> The idea is create something to measure the traffic entropy. Most of
> ddos attacks change (decrease) the entropy of certain traffic.
>
> This is a pseudo-code to implement the idea:
>
>
>
> for_each_packet() {
>
> case udp:
> udp_packet[dest port]->count_bit_1_for_the_packet
> udp_packet[dest port]->store_sddr_daddr_ports_etc
> alfa += apply_the_algorithm_for_the_packet(udp_packet[dest
> port]->count_bit_1_for_the_packet)
> countbit1total[dest port] += udp_packet[dest
> port]->count_bit_1_for_the_packet
>
> case tcp:
> tcp_packet[dest port]->count_bit_1_for_the_packet
> tcp_packet[dest port]->store_sddr_daddr_ports_etc
> apply_the_algorithm_for_the_packet(tcp_packet[dest
> port]->count_bit_1_for_the_packet)
> countbit1total[dest port] += tcp_packet[dest
> port]->count_bit_1_for_the_packet
>
> if(we_have_200_packets_in_this_port)
> {
> beta = apply_the_algorithm_for_the_all_packets(udp_packet[dest
> port]->countbit1total)
>
> if(beta < alfa)
> {
> attack detected
> }
> else {
> normal traffic
> }
> }
> }
>
>
> where
>
>
> apply_the_algorithm_for_the_packet :
>
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
>
> and
>
> apply_the_algorithm_for_the_all_packets :
>
> (PKT_BYTES*8*NUM_PKT_POLL)*(((-(countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))*(log((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))/log(2))
> - (1 -
> ((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))*(log(1-((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))/log(2))))
> + log((PKT_BYTES*8*NUM_PKT_POLL))/log(2);
>
>
> I will try to explain the idea behind the algorithm...
>
> Suppose we have 3 complex strings: X, Y and Z
> So... if we can calculate the complexity for each string using some fomula
> C(x),
>
> for a random/very complex string we have : C(X) + C(Y) + C(Z) < C(XYZ)
>
> in other words .. if you have 3 complex things (C(x) C(y) c(z)) .. and
> concatenate them ... you will have something much more complex (C(XYZ))
>
> make sense ?
>
>
> This is how the algorithm works for ddos detection... measuring a normal
> traffic in a port number .. we will see a lot of random payloads... and
> during an attack.. it will change (if the attacker does not random the
> payload).
>
> So.. for a normal traffic:
>
> Complexity_of(Packet1) + Complexity_of(Packet2) + Complexity_of(PacketN)
> =< All_complexity(Packet1+Packet2+PacketN)
>
> and for a ddos:
>
> Complexity_of(Packet1) + Complexity_of(Packet2) + Complexity_of(PacketN) >
> All_complexity(Packet1+Packet2+PacketN)
>
>
>
>
>
> /* Here we are simulating a normal traffic
> * each bitone represents the distribution of bit 1 in each packet payload
> * and in this case the value of bitone is random
> *
> */
>
> #include <stdio.h>
> #include <math.h>
>
> float NUM_PKT_POLL = 10; // Number of packets to process in each port
> number
> float PKT_BYTES = 32; // payload bytes to count the bit 1
> float countonetotal = 0;
> float THR = 0.3; // I will explain it later
> float bitone = 0;
>
> int main()
> {
> int i;
> float kolmogorov_total = 0;
> float kolmogorov_packet = 0;
>
> bitone = 200;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
>
> countonetotal += 200;
> bitone = 122;
>
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 122;
>
> bitone = 140;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 140;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += bitone;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += bitone;
>
> kolmogorov_total =
> (PKT_BYTES*8*NUM_PKT_POLL)*(((-(countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))*(log((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))/log(2))
> - (1 -
> ((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))*(log(1-((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))/log(2))))
> + log((PKT_BYTES*8*NUM_PKT_POLL))/log(2);
>
> if(kolmogorov_total < kolmogorov_packet)
> printf("ATTACK DETECTED\n");
> else
> printf("NORMAL TRAFFIC\n");
> }
>
> **************************
>
>
>
> ********* CODE ***********
>
>
> /* This is the same code ... but simulating a ddos attack
> */
>
>
> #include <stdio.h>
> #include <math.h>
>
> float NUM_PKT_POLL = 10;
> float PKT_BYTES = 32;
> float countonetotal = 0;
> float THR = 0.3;
> float bitone = 0;
>
> int main()
> {
> int i;
> float kolmogorov_total = 0;
> float kolmogorov_packet = 0;
>
> bitone = 200;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
>
> countonetotal += 200;
> bitone = 122;
>
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 122;
>
> bitone = 140;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 140;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += 150;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += bitone;
>
> bitone = 150;
> kolmogorov_packet +=
> (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2))
> - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2))))
> + log((PKT_BYTES*8))/log(2);
> countonetotal += bitone;
>
> kolmogorov_total =
> (PKT_BYTES*8*NUM_PKT_POLL)*(((-(countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))*(log((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))/log(2))
> - (1 -
> ((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))*(log(1-((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))/log(2))))
> + log((PKT_BYTES*8*NUM_PKT_POLL))/log(2);
>
> if(kolmogorov_total < kolmogorov_packet)
> printf("ATTACK DETECTED\n");
> else
> printf("NORMAL TRAFFIC\n");
> }
>
> On Mon, Oct 12, 2009 at 10:03 AM, Matt Jonkman <jonkman at jonkmans.com>
> wrote:
>
> Yes! We have all the right people here. Shoot us your idea!
>
> Matt
>
>
> Jeff Dickey wrote:
> > I think what Matt was trying to say was "hey, Breno, everybody with a
> > technical interest in OISF is on the list; don't ask to ask - what's the
> > code you've got?"
> >
> > But I'll join the flood anyway :-)
> >
> >
> > On 12/10/09 07:21 , "Matt Jonkman" <jonkman at jonkmans.com> wrote:
> >
> >> There's a large number of people on here, we can't have everyone check
> >> in. :)
> >>
> >> What are you thinking about?
> >>
> >> Matt
> >>
> >> Breno Silva wrote:
> >>> Hey Shyaam!
> >>>
> >>> Good to hear from you!
> >>>
> >>> Lets wait more one day to hear from other guys
> >>>
> >>> cheers
> >>>
> >>> Breno
> >>>
> >>> On Sun, Oct 11, 2009 at 7:57 PM, Shyaam Sundhar <shyaam at gmail.com
> >>> <mailto:shyaam at gmail.com>> wrote:
> >>>
> >>> Everyone is with you brotha!
> >>>
> >>> Sent from my iPhone
> >>>
> >>> On Oct 11, 2009, at 6:35 PM, Breno Silva <breno.silva at gmail.com
> >>> <mailto:breno.silva at gmail.com>> wrote:
> >>>
> >>>> Hey guys,
> >>>>
> >>>> Who is in the list ?
> >>>>
> >>>> I have a simple code to discuss with you
> >>>>
> >>>> Thanks
> >>>>
> >>>> Breno
> >>>> _______________________________________________
> >>>> Oisf-wg-portscan mailing list
> >>>> Oisf-wg-portscan at openinfosecfoundation.org
> >>> <mailto:Oisf-wg-portscan at openinfosecfoundation.org>
> >>>
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
> >>> _______________________________________________
> >>> Oisf-wg-portscan mailing list
> >>> Oisf-wg-portscan at openinfosecfoundation.org
> >>> <mailto:Oisf-wg-portscan at openinfosecfoundation.org>
> >>>
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
> >>>
> >>>
> >>>
> >>>
> ------------------------------------------------------------------------
> >>>
> >>> _______________________________________________
> >>> Oisf-wg-portscan mailing list
> >>> Oisf-wg-portscan at openinfosecfoundation.org
> >>>
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
> >
>
> --
>
> --------------------------------------------
> Matthew Jonkman
> Emerging Threats
> Open Information Security Foundation (OISF)
> Phone 765-429-0398
> Fax 312-264-0205
> http://www.emergingthreats.net
> http://www.openinformationsecurityfoundation.org
> --------------------------------------------
>
> PGP: http://www.jonkmans.com/mattjonkman.asc
>
>
> _______________________________________________
>
> Oisf-wg-portscan mailing list
> Oisf-wg-portscan at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
>
>
>
> _______________________________________________
> Oisf-wg-portscan mailing list
> Oisf-wg-portscan at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openinfosecfoundation.org/pipermail/oisf-wg-portscan/attachments/20091025/1acde2ae/attachment-0001.html
More information about the Oisf-wg-portscan
mailing list