[Oisf-users] segfault - smtp parser - suricata 1.1

[Christophe Vandeplas]

Hello,

I installed/compiled suricata v1.1 yesterday and had 2 segfaults over
a very short time.

The segfaults I had were: (from dmesg)
[143691.171972] Decode & Stream[26656]: segfault at b6b17000 ip
b760b7aa sp b6b13e48 error 4 in libc-2.13.so[b74f6000+15a000]
[177583.326790] Decode & Stream[517]: segfault at b6c2a002 ip b771e7aa
sp b6c26e48 error 4 in libc-2.13.so[b7609000+15a000]

So the third time I ran suricata I did it in a debugger and it crashed
again a few hours later.
Here is the backtrace:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0xb73c1b70 (LWP 27915)]
0xb7eb67ae in ?? () from /lib/i386-linux-gnu/libc.so.6
(gdb)
(gdb) backtrace
#0  0xb7eb67ae in ?? () from /lib/i386-linux-gnu/libc.so.6
#1  0x081713bb in SMTPGetLine (state=0x9ea81a40) at
/usr/include/bits/string3.h:52
#2  0x081763e8 in SMTPParse (f=0xa98544d0, alstate=0x9ea81a40,
pstate=0xa5c5df88,
    input=0xb73c00dc
"\nxsPHQce/yD3IvMk6ybnKOMq3yzbLtsw1zLXNNc21zjbOts83z7jQOdC60TzRvtI/0sHTRNPG1EnU\r\ny9VO1dHWVdbY11zX4Nhk2OjZbNnx2nba+9uA3AXcit0Q3ZbeHN6i3ynfr+A24L3hROHM4lPi2+Nj\r\n4+vkc+T85YTmDeaW5x/nqegy6LzpRunQ6lvq5etw6/v"...,
input_len=2920, output=0xb73bf734) at app-layer-smtp.c:559
#3  SMTPParseClientRecord (f=0xa98544d0, alstate=0x9ea81a40, pstate=0xa5c5df88,
    input=0xb73c00dc
"\nxsPHQce/yD3IvMk6ybnKOMq3yzbLtsw1zLXNNc21zjbOts83z7jQOdC60TzRvtI/0sHTRNPG1EnU\r\ny9VO1dHWVdbY11zX4Nhk2OjZbNnx2nba+9uA3AXcit0Q3ZbeHN6i3ynfr+A24L3hROHM4lPi2+Nj\r\n4+vkc+T85YTmDeaW5x/nqegy6LzpRunQ6lvq5etw6/v"...,
input_len=2920, output=0xb73bf734) at app-layer-smtp.c:581
#4  0x0815c66e in AppLayerDoParse (f=0xa98544d0,
app_layer_state=0x9ea81a40, parser_state=0xa5c5df88,
    input=0xb73c00dc
"\nxsPHQce/yD3IvMk6ybnKOMq3yzbLtsw1zLXNNc21zjbOts83z7jQOdC60TzRvtI/0sHTRNPG1EnU\r\ny9VO1dHWVdbY11zX4Nhk2OjZbNnx2nba+9uA3AXcit0Q3ZbeHN6i3ynfr+A24L3hROHM4lPi2+Nj\r\n4+vkc+T85YTmDeaW5x/nqegy6LzpRunQ6lvq5etw6/v"...,
input_len=2920, parser_idx=16, proto=3) at app-layer-parser.c:695
#5  0x0815f579 in AppLayerParse (f=0xa98544d0, proto=3 '\003', flags=4 '\004',
    input=0xb73c00dc
"\nxsPHQce/yD3IvMk6ybnKOMq3yzbLtsw1zLXNNc21zjbOts83z7jQOdC60TzRvtI/0sHTRNPG1EnU\r\ny9VO1dHWVdbY11zX4Nhk2OjZbNnx2nba+9uA3AXcit0Q3ZbeHN6i3ynfr+A24L3hROHM4lPi2+Nj\r\n4+vkc+T85YTmDeaW5x/nqegy6LzpRunQ6lvq5etw6/v"...,
input_len=2920) at app-layer-parser.c:908
#6  0x0814bbf3 in StreamTcpReassembleAppLayer (ra_ctx=0xb6201f08,
ssn=0xaf458638, stream=0xaf458678, p=<value optimized out>) at
stream-tcp-reassemble.c:3051
#7  0x0814c551 in StreamTcpReassembleHandleSegmentUpdateACK
(ra_ctx=0xb6201f08, ssn=0xaf458638, stream=0xaf458678, p=0x8b02e58) at
stream-tcp-reassemble.c:3430
#8  0x0814e0d5 in StreamTcpReassembleHandleSegment (tv=0xb399888,
ra_ctx=0xb6201f08, ssn=0xaf458638, stream=0xaf45863c, p=0x8b02e58,
pq=0xb62019a0) at stream-tcp-reassemble.c:3504
#9  0x08147c36 in HandleEstablishedPacketToClient (tv=0xb399888,
p=<value optimized out>, stt=0xb6201998, pq=0xb2d8d50) at
stream-tcp.c:1826
#10 StreamTcpPacketStateEstablished (tv=0xb399888, p=<value optimized
out>, stt=0xb6201998, pq=0xb2d8d50) at stream-tcp.c:1980
#11 StreamTcpPacket (tv=0xb399888, p=<value optimized out>,
stt=0xb6201998, pq=0xb2d8d50) at stream-tcp.c:3560
#12 0x08149691 in StreamTcp (tv=0xb399888, p=0x8b02e58,
data=0xb6201998, pq=0xb2d8d50, postpq=0x0) at stream-tcp.c:3765
#13 0x0812d444 in TmThreadsSlotVarRun (tv=0xb399888, p=0x8b02e58,
slot=0xd14dda0) at tm-threads.c:458
#14 0x0812f8c7 in TmThreadsSlotVar (td=0xb399888) at tm-threads.c:655
#15 0xb7f64e99 in start_thread () from /lib/i386-linux-gnu/libpthread.so.0
#16 0xb7e7173e in clone () from /lib/i386-linux-gnu/libc.so.6
(gdb) i r
eax            0xb73c1fef       -1220796433
ecx            0xffffe06c       -8084
edx            0x98e3d5c0       -1729899072
ebx            0xb7efdff4       -1209016332
esp            0xb73bee48       0xb73bee48
ebp            0xb73bf6a8       0xb73bf6a8
esi            0xb73c00dc       -1220804388
edi            0x4c     76
eip            0xb7eb67ae       0xb7eb67ae
eflags         0x10286  [ PF SF IF RF ]
cs             0x73     115
ss             0x7b     123
ds             0x7b     123
es             0x7b     123
fs             0x0      0
gs             0x33     51


It looks like the SMTP parser is crashing, but as I'm not really that
good with gdb I don't know what else I should do to pinpoint the
cause.
Unfortunately I can't take network captures as I'm working with data
from a production environment.(privacy and confidentiality) (and it
takes a few hours to crash)
Bandwidth was around 10-20 Mbit/s, sniffing a mirror port with
multiple (tagged) VLANs

Could you tell me what I should do (in gdb) to help us understand the
origin of the problem ?

In the meantime I'll keep gdb open with the crashed instance of suricata.
And will try disabling some rules to check if that might be the origin
of the segfault.

Thanks
Christophe