[Oisf-devel] LuaJIT running out of memory causing Suricata crashes?

[Victor Julien]

On 12/06/2013 05:54 PM, Chris Wakelin wrote:
> On 04/12/13 20:17, Chris Wakelin wrote:
>> On 04/12/13 12:17, Chris Wakelin wrote:
>>> On 04/12/13 12:16, Victor Julien wrote:
>>>> On 12/04/2013 01:13 PM, Victor Julien wrote:
>>>>> On 12/04/2013 12:56 PM, Victor Julien wrote:
>>>>>> On 12/04/2013 12:37 PM, Chris Wakelin wrote:
>>>>>>>> [32578] 4/12/2013 -- 11:18:50 - (detect-luajit.c:281) <Info> (DetectLuajitMatchBuffer) -- failed to run script: not enough memory
>>>>>>> I guess we could add more checks to the Lua scripts to avoid the
>>>>>>> zlib/xor errors, which occur all the time, but I think they're probably
>>>>>>> not significant. Is there a way we could prevent or at least debug the
>>>>>>> "not enough memory" errors which are presumably what causes Suricata to
>>>>>>> crash?
>>>>>>
>>>>>> Maybe you can try inserting a "return 0" on line 282, so:
>>>>>>
>>>>>>     int retval = lua_pcall(tluajit->luastate, 1, 1, 0);
>>>>>>     if (retval != 0) {
>>>>>>         SCLogInfo("failed to run script: %s",
>>>>>> lua_tostring(tluajit->luastate, -1));
>>>>>>         return 0;
>>>>>>     }
>>>>>>
>>>>>
>>>>> Actually, please ignore this. I found that the lua state isn't cleared
>>>>> properly, will send a test patch in a few minutes.
>>>>>
>>>>
>>>> Can you try the attached patch?
>>>>
>>>
>>> OK
>>>
>>>> What kind of process exit code are you getting?
>>>>
>>>
>>> It's running in daemon mode, so difficult to say. I guess I'll try
>>> running it standalone.
>>>
>>> Best Wishes,
>>> Chris
>>>
>>
>> Standalone with the patch crashed with this on STDOUT/STDERR
>>
>> PANIC: unprotected error in call to Lua API (not enough memory)
>>
>> The exit code was 1. The last line of suricata.log was
>>> [24028] 4/12/2013 -- 19:49:47 - (detect-luajit.c:281) <Info> (DetectLuajitMatchBuffer) -- failed to run script: not enough memory
>>
>> Now trying (nearly) git master!
>>
>> Best Wishes,
>> Chris
>>
> 
> I'm still getting crashes on git master - perhaps not so many, e.g.:
> 
>> [9392] 6/12/2013 -- 15:37:07 - (detect-luajit.c:290) <Info> (DetectLuajitMatchBuffer) -- failed to run script: not enough memory
> 
> Is it worth putting in that "return 0;" after "SCLogInfo("failed to run
> script: %s", lua_tostring(tluajit->luastate, -1));" again?

No I actually think that was a bad idea. It would leave the lua stack
"dirty" I believe.

Not really sure how to debug this issue. Are you able to reproduce it on
a pcap? Or does it only happen after some time?

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------