[Oisf-devel] Debug HashListTableLookup()

[Yao-Min Chen]

The issue is with the "any" ip address, when handled by 
DetectAddressSetup().

    /* if any, insert 0.0.0.0/0 and ::/0 as well */

Not exactly sure the behavior for x86 but for sparc, the netmask becomes 
0xffffffff. Essentially the full range of IPv4 addresses are added to 
the hash table.

1. IPOnlyAddSignature() calls IPOnlyAddSlash16() to add 0.0.0.0/0 in 64K 
steps (2^32 divided by 2^16)

2. The hash function uses the lower 16 bits which are all 0's, the same 
for each entry to be hashed.

    #define IPONLY_EXTRACT_16(a) ((a)->ip[0] & 0x0000ffff)

3. This is the worst case for the hashed list table as all keys get 
hashed to the same bucket. Towards the end of the 64K steps, each add 
would traverse a linked list of size 64K.

4. I modified the hash function so the upper 16 bits are also used and 
SigTestBidirec03 can run to completion now.

    #define IPONLY_EXTRACT_16(a)\
     ((((a)->ip[0] & 0x0000ffff) + (((a)->ip[0] & 0xffff0000)>>8)) & 
0x0000ffff)

Question: I saw high = 0xffffffff and low = 0x0 in the following 
invocation of IPOnlyAddSlash16(), for adding the address 0.0.0.0/0. Are 
the high and low values correct?

static void IPOnlyAddSlash16(DetectEngineCtx *de_ctx, 
DetectEngineIPOnlyCtx *io_ctx,
HashListTable *ht, DetectAddress *gr, char direction, Signature *s) {
    uint32_t high = ntohl(gr->ip2[0]);
    uint32_t low = ntohl(gr->ip[0]);

Thanks,
Yaomin


On 01/31/10 10:00, Will Metcalf wrote:
> I think this is a problem somewhere in the cidr masking.  I have seen 
> this before on PPC so I'm guessing it is an endian issue, I have not 
> had a chance to look into it any further.  Let me know if you figure 
> it out ;-)....
>
> https://redmine.openinfosecfoundation.org/issues/show/63
>
> Regards,
>
> Will
>
> On Sun, Jan 31, 2010 at 11:23 AM, Yao-Min Chen <Yaomin.Chen at sun.com 
> <mailto:Yaomin.Chen at sun.com>> wrote:
>
>     My unit test SigTestBidirec03 seems to loop forever while
>     traversing the
>     hashed list of signature patterns (I verified this by setting break
>     point and single stepping the run). Any hint on how to check
>     whether the
>     hashed list is properly terminated, and without self-looping?
>
>     Test SigTestBidirec03                                             :
>     ^Cdbx: warning: Interrupt ignored but forwarded to child.
>     t at 1 (l at 1) signal INT (Interrupt) in DetectAddressCmp at line 1392 in
>     file "detect-engine-address.c"
>      1392       if (a->flags & ADDRESS_FLAG_ANY && b->flags &
>     ADDRESS_FLAG_ANY)
>     (dbx) where
>     current thread: t at 1
>     =>[1] DetectAddressCmp(a = <value not available>, b = 0xacc220), line
>     1392 in "detect-engine-address.c"
>      [2] IPOnlyCompareFunc(data1 = 0xa87ce8, len1 = <value not available>,
>     data2 = 0xacc220, len2 = <value not available>), line 190 in
>     "detect-engine-iponly.c"
>      [3] HashListTableLookup(ht = <value not available>, data = 0xacc220,
>     datalen = <value not available>), line 237 in "util-hashlist.c"
>      [4] IPOnlyAddSlash16(de_ctx = 0x4c8240, io_ctx = 0x4ca2d0, ht =
>     0x2b06f8, gr = 0x5e3898, direction = <value not available>, s =
>     0x5e3780), line 81 in "detect-engine-iponly.c"
>      [5] IPOnlyAddSignature(de_ctx = 0x4c8240, io_ctx = 0x4ca2d0, s =
>     0x5e3780), line 475 in "detect-engine-iponly.c"
>      [6] SigAddressPrepareStage2(de_ctx = 0x4c8240), line 1667 in
>     "detect.c"
>      [7] SigGroupBuild(de_ctx = 0x4c8240), line 2797 in "detect.c"
>      [8] UTHMatchPackets(de_ctx = 0x4c8240, p = <value not available>,
>     num_packets = -16948332), line 465 in "util-unittest-helper.c"
>      [9] SigTestBidirec03(), line 1352 in "detect-parse.c"
>      [10] UtRunTests(regex_arg = <value not available>), line 182 in
>     "util-unittest.c"
>      [11] main(argc = <value not available>, argv = <value not
>     available>),
>     line 596 in "suricata.c"
>
>     Thanks,
>     Yaomin
>
>     _______________________________________________
>     Oisf-devel mailing list
>     Oisf-devel at openinfosecfoundation.org
>     <mailto:Oisf-devel at openinfosecfoundation.org>
>     http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Oisf-devel mailing list
> Oisf-devel at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20100204/e292638d/attachment-0002.html>