From tomb at byrneit.net  Wed Nov 25 12:07:02 2009
From: tomb at byrneit.net (Tomas L. Byrnes)
Date: Wed, 25 Nov 2009 09:07:02 -0800
Subject: [Oisf-wg-portscan] Hey
In-Reply-To: <65f41b5a0910250622qc15513jb35c7b116464a09a@mail.gmail.com>
References: <C6F91EB6.227A1%jdickey@seven-sigma.com><4AD3292C.8070702@jonkmans.com><65f41b5a0910120857o76860ceam8ee1734fa7019f76@mail.gmail.com><70D072392E56884193E3D2DE09C097A9381E9E@pascal.zaphodb.org>
	<65f41b5a0910250622qc15513jb35c7b116464a09a@mail.gmail.com>
Message-ID: <70D072392E56884193E3D2DE09C097A9382017@pascal.zaphodb.org>

I'm sorry that this fell through the cracks. I've had a hellacious month. I'd be happy to test for you.

 

 

 

From: oisf-wg-portscan-bounces at openinfosecfoundation.org [mailto:oisf-wg-portscan-bounces at openinfosecfoundation.org] On Behalf Of Breno Silva
Sent: Sunday, October 25, 2009 6:22 AM
To: DDoS and Portscan methods discussion
Subject: Re: [Oisf-wg-portscan] Hey

 

Hi Tomas,

I have a implementation of this in my internal and external network. However my internal network is very big and my external is AS7738 and AS8167. 
In both cases it is working well, almost wiithout false positives and good rate of attack detection. But in both cases the traffic is highly random (internal is less random) ... but working well

However i never tested it in a small LAN. If you have one i can send you a code for testing.

What do you think ?

Thanks

Breno

On Sat, Oct 24, 2009 at 3:15 PM, Tomas L. Byrnes <tomb at byrneit.net> wrote:

I think the problem with this is that you're assuming that the PAYLOAD of traffic to a given port, especially UDP is highly random in the case of normal traffic, and highly self-similar in the case of (D)DOS.

 

This is not true, especially for widely used services such as DNS. The vast majority of DNS packets are HIGHLY self-similar, especially the ones to/from Authoritative Nameservers, which are usually answering queries for the exact same RRSETs all the time.

 

 

 

From: oisf-wg-portscan-bounces at openinfosecfoundation.org [mailto:oisf-wg-portscan-bounces at openinfosecfoundation.org] On Behalf Of Breno Silva
Sent: Monday, October 12, 2009 8:58 AM


To: DDoS and Portscan methods discussion
Subject: Re: [Oisf-wg-portscan] Hey

 

Hi Guys 

 

Good to hear from you.

I?m sending two simples codes for discussion if it can be used 
as a part of future ddos detection engine.

The idea is create something to measure the traffic entropy. Most of
ddos attacks change (decrease) the entropy of certain traffic.

This is a pseudo-code to implement the idea:

 

for_each_packet() {

 case udp:
  udp_packet[dest port]->count_bit_1_for_the_packet
  udp_packet[dest port]->store_sddr_daddr_ports_etc
  alfa += apply_the_algorithm_for_the_packet(udp_packet[dest port]->count_bit_1_for_the_packet)
  countbit1total[dest port] += udp_packet[dest port]->count_bit_1_for_the_packet

 case tcp:
  tcp_packet[dest port]->count_bit_1_for_the_packet
  tcp_packet[dest port]->store_sddr_daddr_ports_etc
  apply_the_algorithm_for_the_packet(tcp_packet[dest port]->count_bit_1_for_the_packet)
  countbit1total[dest port] += tcp_packet[dest port]->count_bit_1_for_the_packet
  
  if(we_have_200_packets_in_this_port)
  {
   beta = apply_the_algorithm_for_the_all_packets(udp_packet[dest port]->countbit1total)

   if(beta < alfa)
   {
    attack detected
   }
   else {
    normal traffic
   }
  }
}


where


 apply_the_algorithm_for_the_packet :

(PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);

 and

apply_the_algorithm_for_the_all_packets :

(PKT_BYTES*8*NUM_PKT_POLL)*(((-(countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))*(log((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))/log(2)) - (1 - ((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))*(log(1-((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))/log(2)))) + log((PKT_BYTES*8*NUM_PKT_POLL))/log(2);


I will try to explain the idea behind the algorithm...

Suppose we have 3 complex strings: X, Y and Z
So... if we can calculate the complexity for each string using some fomula C(x),

for a random/very complex string we have :  C(X) + C(Y) + C(Z) < C(XYZ) 

in other words .. if you have 3 complex things (C(x) C(y) c(z)) .. and concatenate them ... you will have something much more complex (C(XYZ))

make sense ?


This is how the algorithm works for ddos detection... measuring a normal traffic in a port number .. we will see a lot of random payloads... and during an attack.. it will change (if the attacker does not random the payload).

So.. for a normal traffic:

Complexity_of(Packet1) +  Complexity_of(Packet2) + Complexity_of(PacketN) =< All_complexity(Packet1+Packet2+PacketN)

and for a ddos:

Complexity_of(Packet1) +  Complexity_of(Packet2) + Complexity_of(PacketN) > All_complexity(Packet1+Packet2+PacketN)

 

 

/* Here we are simulating a normal traffic
 * each bitone represents the distribution of bit 1 in each packet payload
 * and in this case the value of bitone is random
 *
 */

#include <stdio.h>
#include <math.h>

float NUM_PKT_POLL = 10;   // Number of packets to process in each port number
float PKT_BYTES = 32;      // payload bytes to count the bit 1
float countonetotal = 0;
float THR = 0.3;          // I will explain it later
float bitone = 0;

int main()
{
int i;
float kolmogorov_total = 0;
float kolmogorov_packet = 0;

bitone = 200;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);

countonetotal += 200;
bitone = 122;

kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 122;

bitone = 140;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 140;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += bitone;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += bitone;

kolmogorov_total = (PKT_BYTES*8*NUM_PKT_POLL)*(((-(countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))*(log((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))/log(2)) - (1 - ((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))*(log(1-((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))/log(2)))) + log((PKT_BYTES*8*NUM_PKT_POLL))/log(2);

if(kolmogorov_total < kolmogorov_packet)
printf("ATTACK DETECTED\n");
else
printf("NORMAL TRAFFIC\n");
}

**************************

 

********* CODE ***********


/* This is the same code ... but simulating a ddos attack
 */


#include <stdio.h>
#include <math.h>

float NUM_PKT_POLL = 10;
float PKT_BYTES = 32;
float countonetotal = 0;
float THR = 0.3;
float bitone = 0;

int main()
{
int i;
float kolmogorov_total = 0;
float kolmogorov_packet = 0;

bitone = 200;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);

countonetotal += 200;
bitone = 122;

kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 122;

bitone = 140;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 140;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += 150;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += bitone;

bitone = 150;
kolmogorov_packet += (PKT_BYTES*8)*(((-(bitone)/(PKT_BYTES*8))*(log((bitone)/(PKT_BYTES*8))/log(2)) - (1 - ((bitone)/(PKT_BYTES*8)))*(log(1-((bitone)/(PKT_BYTES*8)))/log(2)))) + log((PKT_BYTES*8))/log(2);
countonetotal += bitone;

kolmogorov_total = (PKT_BYTES*8*NUM_PKT_POLL)*(((-(countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))*(log((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL))/log(2)) - (1 - ((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))*(log(1-((countonetotal)/(PKT_BYTES*8*NUM_PKT_POLL)))/log(2)))) + log((PKT_BYTES*8*NUM_PKT_POLL))/log(2);

if(kolmogorov_total < kolmogorov_packet)
printf("ATTACK DETECTED\n");
else
printf("NORMAL TRAFFIC\n");
}

On Mon, Oct 12, 2009 at 10:03 AM, Matt Jonkman <jonkman at jonkmans.com> wrote:

Yes! We have all the right people here. Shoot us your idea!

Matt


Jeff Dickey wrote:
> I think what Matt was trying to say was "hey, Breno, everybody with a
> technical interest in OISF is on the list; don't ask to ask - what's the
> code you've got?"
>
> But I'll join the flood anyway :-)
>
>
> On 12/10/09 07:21 , "Matt Jonkman" <jonkman at jonkmans.com> wrote:
>
>> There's a large number of people on here, we can't have everyone check
>> in. :)
>>
>> What are you thinking about?
>>
>> Matt
>>
>> Breno Silva wrote:
>>> Hey Shyaam!
>>>
>>> Good to hear from you!
>>>
>>> Lets wait more one day to hear from other guys
>>>
>>> cheers
>>>
>>> Breno
>>>
>>> On Sun, Oct 11, 2009 at 7:57 PM, Shyaam Sundhar <shyaam at gmail.com
>>> <mailto:shyaam at gmail.com>> wrote:
>>>
>>>     Everyone is with you brotha!
>>>
>>>     Sent from my iPhone
>>>
>>>     On Oct 11, 2009, at 6:35 PM, Breno Silva <breno.silva at gmail.com
>>>     <mailto:breno.silva at gmail.com>> wrote:
>>>
>>>> Hey guys,
>>>>
>>>> Who is in the list ?
>>>>
>>>> I have a simple code to discuss with you
>>>>
>>>> Thanks
>>>>
>>>> Breno
>>>> _______________________________________________
>>>> Oisf-wg-portscan mailing list
>>>> Oisf-wg-portscan at openinfosecfoundation.org
>>>     <mailto:Oisf-wg-portscan at openinfosecfoundation.org>
>>>     http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
>>>     _______________________________________________
>>>     Oisf-wg-portscan mailing list
>>>     Oisf-wg-portscan at openinfosecfoundation.org
>>>     <mailto:Oisf-wg-portscan at openinfosecfoundation.org>
>>>     http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Oisf-wg-portscan mailing list
>>> Oisf-wg-portscan at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan
>

--

--------------------------------------------
Matthew Jonkman
Emerging Threats
Open Information Security Foundation (OISF)
Phone 765-429-0398
Fax 312-264-0205
http://www.emergingthreats.net <http://www.emergingthreats.net/> 
http://www.openinformationsecurityfoundation.org <http://www.openinformationsecurityfoundation.org/> 
--------------------------------------------

PGP: http://www.jonkmans.com/mattjonkman.asc


_______________________________________________

Oisf-wg-portscan mailing list
Oisf-wg-portscan at openinfosecfoundation.org
http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan

 


_______________________________________________
Oisf-wg-portscan mailing list
Oisf-wg-portscan at openinfosecfoundation.org
http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-wg-portscan

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.openinfosecfoundation.org/pipermail/oisf-wg-portscan/attachments/20091125/74e93a01/attachment-0001.html