[Oisf-devel] Debug HashListTableLookup()
Victor Julien
victor at inliniac.net
Wed Feb 10 09:35:30 UTC 2010
Cool Yao-Min! Can you try the attached patch to see if this works as well?
Cheers,
Victor
Yao-Min Chen wrote:
> I have resolved this. Attached is the diff file. Instead of defining
> OS_SUNOS like I did, you probably want to use a new #define BIG_ENDIAN,
> so the patch is applicable to all big-endian processors.
>
> Yaomin
>
>
>
> On 02/04/10 11:12, Yao-Min Chen wrote:
>> The issue is with the "any" ip address, when handled by
>> DetectAddressSetup().
>>
>> /* if any, insert 0.0.0.0/0 and ::/0 as well */
>>
>> Not exactly sure the behavior for x86 but for sparc, the netmask
>> becomes 0xffffffff. Essentially the full range of IPv4 addresses are
>> added to the hash table.
>>
>> 1. IPOnlyAddSignature() calls IPOnlyAddSlash16() to add 0.0.0.0/0 in
>> 64K steps (2^32 divided by 2^16)
>>
>> 2. The hash function uses the lower 16 bits which are all 0's, the
>> same for each entry to be hashed.
>>
>> #define IPONLY_EXTRACT_16(a) ((a)->ip[0] & 0x0000ffff)
>>
>> 3. This is the worst case for the hashed list table as all keys get
>> hashed to the same bucket. Towards the end of the 64K steps, each add
>> would traverse a linked list of size 64K.
>>
>> 4. I modified the hash function so the upper 16 bits are also used and
>> SigTestBidirec03 can run to completion now.
>>
>> #define IPONLY_EXTRACT_16(a)\
>> ((((a)->ip[0] & 0x0000ffff) + (((a)->ip[0] & 0xffff0000)>>8)) &
>> 0x0000ffff)
>>
>> Question: I saw high = 0xffffffff and low = 0x0 in the following
>> invocation of IPOnlyAddSlash16(), for adding the address 0.0.0.0/0.
>> Are the high and low values correct?
>>
>> static void IPOnlyAddSlash16(DetectEngineCtx *de_ctx,
>> DetectEngineIPOnlyCtx *io_ctx,
>> HashListTable *ht, DetectAddress *gr, char direction, Signature *s) {
>> uint32_t high = ntohl(gr->ip2[0]);
>> uint32_t low = ntohl(gr->ip[0]);
>>
>> Thanks,
>> Yaomin
>>
>>
>> On 01/31/10 10:00, Will Metcalf wrote:
>>> I think this is a problem somewhere in the cidr masking. I have seen
>>> this before on PPC so I'm guessing it is an endian issue, I have not
>>> had a chance to look into it any further. Let me know if you figure
>>> it out ;-)....
>>>
>>> https://redmine.openinfosecfoundation.org/issues/show/63
>>>
>>> Regards,
>>>
>>> Will
>>>
>>> On Sun, Jan 31, 2010 at 11:23 AM, Yao-Min Chen <Yaomin.Chen at sun.com
>>> <mailto:Yaomin.Chen at sun.com>> wrote:
>>>
>>> My unit test SigTestBidirec03 seems to loop forever while
>>> traversing the
>>> hashed list of signature patterns (I verified this by setting break
>>> point and single stepping the run). Any hint on how to check
>>> whether the
>>> hashed list is properly terminated, and without self-looping?
>>>
>>> Test SigTestBidirec03 :
>>> ^Cdbx: warning: Interrupt ignored but forwarded to child.
>>> t at 1 (l at 1) signal INT (Interrupt) in DetectAddressCmp at line 1392 in
>>> file "detect-engine-address.c"
>>> 1392 if (a->flags & ADDRESS_FLAG_ANY && b->flags &
>>> ADDRESS_FLAG_ANY)
>>> (dbx) where
>>> current thread: t at 1
>>> =>[1] DetectAddressCmp(a = <value not available>, b = 0xacc220), line
>>> 1392 in "detect-engine-address.c"
>>> [2] IPOnlyCompareFunc(data1 = 0xa87ce8, len1 = <value not
>>> available>,
>>> data2 = 0xacc220, len2 = <value not available>), line 190 in
>>> "detect-engine-iponly.c"
>>> [3] HashListTableLookup(ht = <value not available>, data = 0xacc220,
>>> datalen = <value not available>), line 237 in "util-hashlist.c"
>>> [4] IPOnlyAddSlash16(de_ctx = 0x4c8240, io_ctx = 0x4ca2d0, ht =
>>> 0x2b06f8, gr = 0x5e3898, direction = <value not available>, s =
>>> 0x5e3780), line 81 in "detect-engine-iponly.c"
>>> [5] IPOnlyAddSignature(de_ctx = 0x4c8240, io_ctx = 0x4ca2d0, s =
>>> 0x5e3780), line 475 in "detect-engine-iponly.c"
>>> [6] SigAddressPrepareStage2(de_ctx = 0x4c8240), line 1667 in
>>> "detect.c"
>>> [7] SigGroupBuild(de_ctx = 0x4c8240), line 2797 in "detect.c"
>>> [8] UTHMatchPackets(de_ctx = 0x4c8240, p = <value not available>,
>>> num_packets = -16948332), line 465 in "util-unittest-helper.c"
>>> [9] SigTestBidirec03(), line 1352 in "detect-parse.c"
>>> [10] UtRunTests(regex_arg = <value not available>), line 182 in
>>> "util-unittest.c"
>>> [11] main(argc = <value not available>, argv = <value not
>>> available>),
>>> line 596 in "suricata.c"
>>>
>>> Thanks,
>>> Yaomin
>>>
>>> _______________________________________________
>>> Oisf-devel mailing list
>>> Oisf-devel at openinfosecfoundation.org
>>> <mailto:Oisf-devel at openinfosecfoundation.org>
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Oisf-devel mailing list
>>> Oisf-devel at openinfosecfoundation.org
>>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Oisf-devel mailing list
>> Oisf-devel at openinfosecfoundation.org
>> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Oisf-devel mailing list
> Oisf-devel at openinfosecfoundation.org
> http://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
--
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Handle-ip-only-matching-correctly-on-big-endian-syst.patch
Type: application/mbox
Size: 3063 bytes
Desc: not available
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20100210/d1e77dc6/attachment.mbox>
More information about the Oisf-devel
mailing list