[Oisf-users] segfault in libc

[Victor Julien]

On 09/16/2013 11:08 AM, Victor Julien wrote:
> On 09/09/2013 10:08 AM, Christophe Vandeplas wrote:
>> @Peter: bug created : #942
>>
>> @Victor: I would prefer to delegate this completely to you, and give
>> you full access on the system. Unfortunately as you said, and as
>> everyone says, this machine is "in production" and not in a network
>> where I can give you access.
>>
>> On Tue, Sep 3, 2013 at 5:41 PM, Victor Julien <lists at inliniac.net> wrote:
>>>> Any pointer what I should do next to help locate the problem?
>>>
>>> This is really strange. The error you get is SEGV, but the bt says it
>>> crashes on a line:
>>>         BUG_ON(data_len > sizeof(data));
>>>
>>> BUG_ON is a wrapper around assert, which should exit/crash with a
>>> different signal. Weird.
>>>
>>> To get at this point there has to be some bug during reassembly. The
>>> data_len variable is too big, that is clear.
>>
>> Perhaps it would be a good idea to also focus on reproducing the bug
>> easily, right now it can take half a day or more before the thing
>> crashes.
>> Could you help me pinpoint what type of traffic was reassembled? I'm
>> sure it must be in the core dump somewhere (I kept 2 core dumps), but
>> I have no idea where to look. With details about the traffic I should
>> be able to put a filter and capture the thing..
> 
> In a bt like this:
> 
> (gdb) bt
> #0 0x00007f0302c1af47 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
> #1 0x00007f0302c1917e in __assert_fail () from
> /lib/x86_64-linux-gnu/libc.so.6
> #2 0x00000000005ac84d in StreamTcpReassembleAppLayer (tv=0x23258570,
> ra_ctx=0x7f02b8001540, ssn=0x7f027c72c9e0,
> stream=0x7f027c72ca30, p=0x25057a0) at stream-tcp-reassemble.c:2979
> #3 0x0000000000000000 in ?? ()
> 
> You can get to the traffic into by going to a frame where you have the
> "p" (Packet) variable. In this case frame 2:
> 
> (gdb) frame 2
> 
> There you can print the entire packet:
> 
> (gdb) print *p
> 
> Or just individual fields:
> 
> (gdp) print p->proto
> (gdp) print p->sp
> (gdp) print p->dp
> 
> Addresses are bit more involved:
> (gdb) print p->src.address.address_un_data32[0]
> (gdb) print p->dst.address.address_un_data32[0]
> 
> This gives you an unsigned int value, you'll have to convert that to the
> dotted quad notation yourself.
> 
> For IPv6 you'll have to use:
> (gdb) print p->src.address.address_un_data32[0]
> (gdb) print p->src.address.address_un_data32[1]
> (gdb) print p->src.address.address_un_data32[2]
> (gdb) print p->src.address.address_un_data32[3]
> 
> To figure out whether it's 4 or 6, print:
> 
> (gdb) print p->src.family
> 
> Here 2 is ipv4, 10 is ipv6.
> 

I've put the above in this wiki page:
https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Debugging#gdb-with-core-files

It has a bit more info, including a perl one liner to get from the int
ip value to the dotted quad ip:

(gdb) print p->src.address.address_un_data32[0]
$2 = 721529024

$ perl -MSocket -e "print inet_ntoa(pack('V',721529024)) . \"\n\";"
192.168.1.43

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------