[Oisf-devel] Debug HashListTableLookup()
Yao-Min Chen
Yaomin.Chen at Sun.COM
Thu Feb 18 09:32:46 UTC 2010
Hi Victor,
I tested your commit in the master. It works for me.
Regards,
Yaomin
On 02/10/10 01:35, Victor Julien wrote:
> Cool Yao-Min! Can you try the attached patch to see if this works as well?
>
> Cheers,
> Victor
>
> Yao-Min Chen wrote:
>
>> I have resolved this. Attached is the diff file. Instead of defining
>> OS_SUNOS like I did, you probably want to use a new #define BIG_ENDIAN,
>> so the patch is applicable to all big-endian processors.
>>
>> Yaomin
>>
>>
>>
>> On 02/04/10 11:12, Yao-Min Chen wrote:
>>
>>> The issue is with the "any" ip address, when handled by
>>> DetectAddressSetup().
>>>
>>> /* if any, insert 0.0.0.0/0 and ::/0 as well */
>>>
>>> Not exactly sure the behavior for x86 but for sparc, the netmask
>>> becomes 0xffffffff. Essentially the full range of IPv4 addresses are
>>> added to the hash table.
>>>
>>> 1. IPOnlyAddSignature() calls IPOnlyAddSlash16() to add 0.0.0.0/0 in
>>> 64K steps (2^32 divided by 2^16)
>>>
>>> 2. The hash function uses the lower 16 bits which are all 0's, the
>>> same for each entry to be hashed.
>>>
>>> #define IPONLY_EXTRACT_16(a) ((a)->ip[0] & 0x0000ffff)
>>>
>>> 3. This is the worst case for the hashed list table as all keys get
>>> hashed to the same bucket. Towards the end of the 64K steps, each add
>>> would traverse a linked list of size 64K.
>>>
>>> 4. I modified the hash function so the upper 16 bits are also used and
>>> SigTestBidirec03 can run to completion now.
>>>
>>> #define IPONLY_EXTRACT_16(a)\
>>> ((((a)->ip[0] & 0x0000ffff) + (((a)->ip[0] & 0xffff0000)>>8)) &
>>> 0x0000ffff)
>>>
>>> Question: I saw high = 0xffffffff and low = 0x0 in the following
>>> invocation of IPOnlyAddSlash16(), for adding the address 0.0.0.0/0.
>>> Are the high and low values correct?
>>>
>>> static void IPOnlyAddSlash16(DetectEngineCtx *de_ctx,
>>> DetectEngineIPOnlyCtx *io_ctx,
>>> HashListTable *ht, DetectAddress *gr, char direction, Signature *s) {
>>> uint32_t high = ntohl(gr->ip2[0]);
>>> uint32_t low = ntohl(gr->ip[0]);
>>>
>>> Thanks,
>>> Yaomin
>>>
>>>
>>> On 01/31/10 10:00, Will Metcalf wrote:
>>>
>>>> I think this is a problem somewhere in the cidr masking. I have seen
>>>> this before on PPC so I'm guessing it is an endian issue, I have not
>>>> had a chance to look into it any further. Let me know if you figure
>>>> it out ;-)....
>>>>
>>>> https://redmine.openinfosecfoundation.org/issues/show/63
>>>>
>>>> Regards,
>>>>
>>>> Will
>>>>
>>>> On Sun, Jan 31, 2010 at 11:23 AM, Yao-Min Chen <Yaomin.Chen at sun.com
>>>> <mailto:Yaomin.Chen at sun.com>> wrote:
>>>>
>>>> My unit test SigTestBidirec03 seems to loop forever while
>>>> traversing the
>>>> hashed list of signature patterns (I verified this by setting break
>>>> point and single stepping the run). Any hint on how to check
>>>> whether the
>>>> hashed list is properly terminated, and without self-looping?
>>>>
>>>> Test SigTestBidirec03 :
>>>> ^Cdbx: warning: Interrupt ignored but forwarded to child.
>>>> t at 1 (l at 1) signal INT (Interrupt) in DetectAddressCmp at line 1392 in
>>>> file "detect-engine-address.c"
>>>> 1392 if (a->flags & ADDRESS_FLAG_ANY && b->flags &
>>>> ADDRESS_FLAG_ANY)
>>>> (dbx) where
>>>> current thread: t at 1
>>>> =>[1] DetectAddressCmp(a = <value not available>, b = 0xacc220), line
>>>> 1392 in "detect-engine-address.c"
>>>> [2] IPOnlyCompareFunc(data1 = 0xa87ce8, len1 = <value not
>>>> available>,
>>>> data2 = 0xacc220, len2 = <value not available>), line 190 in
>>>> "detect-engine-iponly.c"
>>>> [3] HashListTableLookup(ht = <value not available>, data = 0xacc220,
>>>> datalen = <value not available>), line 237 in "util-hashlist.c"
>>>> [4] IPOnlyAddSlash16(de_ctx = 0x4c8240, io_ctx = 0x4ca2d0, ht =
>>>> 0x2b06f8, gr = 0x5e3898, direction = <value not available>, s =
>>>> 0x5e3780), line 81 in "detect-engine-iponly.c"
>>>> [5] IPOnlyAddSignature(de_ctx = 0x4c8240, io_ctx = 0x4ca2d0, s =
>>>> 0x5e3780), line 475 in "detect-engine-iponly.c"
>>>> [6] SigAddressPrepareStage2(de_ctx = 0x4c8240), line 1667 in
>>>> "detect.c"
>>>> [7] SigGroupBuild(de_ctx = 0x4c8240), line 2797 in "detect.c"
>>>> [8] UTHMatchPackets(de_ctx = 0x4c8240, p = <value not available>,
>>>> num_packets = -16948332), line 465 in "util-unittest-helper.c"
>>>> [9] SigTestBidirec03(), line 1352 in "detect-parse.c"
>>>> [10] UtRunTests(regex_arg = <value not available>), line 182 in
>>>> "util-unittest.c"
>>>> [11] main(argc = <value not available>, argv = <value not
>>>> available>),
>>>> line 596 in "suricata.c"
>>>>
>>>> Thanks,
>>>> Yaomin
>>>>
>>>> _______________________________________________
>>>> Oisf-devel mailing list
>>>> Oisf-devel at openinfosecfoundation.org
>>>> <mailto:Oisf-devel at openinfosecfoundation.org>
>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>>
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>> _______________________________________________
>>>> Oisf-devel mailing list
>>>> Oisf-devel at openinfosecfoundation.org
>>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>>
>>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Oisf-devel mailing list
>>> Oisf-devel at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>
>>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Oisf-devel mailing list
>> Oisf-devel at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>
>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Oisf-devel mailing list
> Oisf-devel at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20100218/96bc3d89/attachment-0002.html>
More information about the Oisf-devel
mailing list