[Oisf-devel] Suricata core dumps with luajit rule and large ruleset?

[Victor Julien]

On 09/20/2012 12:21 AM, Chris Wakelin wrote:
> I've been trying out my little Lua(jit) XORed-binary detector and
> discovered that when I include a large ruleset such as
> emerging-trojans.rules (6.5k) or emerging-malware.rules (4k), Suricata
> segfaults:-
> 
>> Program terminated with signal 11, Segmentation fault.
>> #0  0x00007f26a3eaf580 in ?? () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
>> #0  0x00007f26a3eaf580 in ?? () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
>> No symbol table info available.
>> #1  0x00007f26a3ee2bb0 in lua_tolstring () from /usr/lib/x86_64-linux-gnu/libluajit-5.1.so.2
>> No symbol table info available.
>> #2  0x00000000004ba368 in DetectLuajitThreadInit (data=<optimised out>) at detect-luajit.c:395
>>         _sc_log_err_msg = "[799] 19/9/2012 -- 22:58:45 - (detect-luajit.c:387) <Error> (DetectLuajitThreadInit) -- [ERRCODE: SC_ERR_LUAJIT_ERROR(218)] - ", '\000' <repeats 1921 times>
>>         _sc_log_err_temp = <optimised out>
>>         luajit = <optimised out>
>>         __PRETTY_FUNCTION__ = "DetectLuajitThreadInit"
>>         t = 0x7f269c503740
>>         __FUNCTION__ = "DetectLuajitThreadInit"
>>         status = <optimised out>
>> #3  0x0000000000443f1f in DetectEngineThreadCtxInitKeywords (de_ctx=<optimised out>, det_ctx=<optimised out>) at detect-engine.c:711
>>         item = 0x40ad750
>> #4  0x00000000004455ce in DetectEngineThreadCtxInitKeywords (de_ctx=<optimised out>, det_ctx=<optimised out>) at detect-engine.c:698
>> No locals.
>> #5  DetectEngineThreadCtxInit (tv=0x5ddb280, initdata=0x3fc69d0, data=0x7f26a0cbd628) at detect-engine.c:803
>>         de_ctx = 0x3fc69d0
>>         det_ctx = 0x7f269c007730
>>         __FUNCTION__ = "DetectEngineThreadCtxInit"
>> #6  0x00000000005131c0 in TmThreadsSlotPktAcqLoop (td=0x5ddb280) at tm-threads.c:633
>>         slot_data = 0x0
>>         tv = 0x5ddb280
>>         s = 0x5dc30e0
>>         r = <optimised out>
>>         slot = <optimised out>
>>         __FUNCTION__ = "TmThreadsSlotPktAcqLoop"
>> #7  0x00007f26a33fae9a in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
>> No symbol table info available.
>> #8  0x00007f26a2cad4bd in clone () from /lib/x86_64-linux-gnu/libc.so.6
>> No symbol table info available.
>> #9  0x0000000000000000 in ?? ()
>> No symbol table info available.
> 
> This is using subsets of ET ruleset, plus local my rules. With ~5.5k
> total rules it seems fine; add one of the big ones and it breaks.

Here is works even with emerging-all.rules + the lua rule.

> My luajit rule:
> 
> alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"LUAJIT test - match
> XORed binary"; flowbits:isset,ET.http.javaclient.vulnerable;
> flowbits:isnotset,ET.http.binary; luajit:xor-n-plus4-or-6.lua;
> sid:379000001; rev:2;)
> 
> Victor, I think I sent you the Lua script already? I'll send it to
> anyone else interested if they ask ;-)

Yes, I have it.

> Suricata is latest git - 1.4dev (rev 9a4b612) - and was started with
> "--runmode=single -r <1GB pcap>". I've not tried other runmodes (such as
> live traffic) yet. I'm running Ubuntu 12.04 64-bit.

I fixed the reason it segv'd. But the it crashed in an error message,
and I've unable to reproduce the error condition. Can you send me the
exact config + rules to reproduce?

> (BTW the rule uses a lot of ticks, but isn't the worst by a long way.
> Surprisingly, making it return 0 instantly doesn't seem to speed it up
> much!)

Cool.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------