[Oisf-devel] LUA Streaming API

[Martin Dubuc]

I am experimenting with LUA scripting, more specifically the streaming API.
I have tried to follow examples in documentation, but get an error when I
do so.

For instance, if I define the following in my LUA script (as per
documentation at URL
https://suricata.readthedocs.io/en/suricata-4.1.6/lua/lua-functions.html#streaming-data
):

function init (args)
    local needs = {}
    needs["type"] = "streaming"
    needs["protocol"] = "http"
    return needsend

I get the following error:
17/3/2020 -- 10:38:08 - <Notice> - This is Suricata version 4.1.6 RELEASE
17/3/2020 -- 10:38:08 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] -
unsupported data type protocol
17/3/2020 -- 10:38:08 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] -
error parsing signature "alert http any any -> any any (msg:"match http
body lua";content:"GET"; http_method;
luajit:scripts/http-body-content-detection.lua;sid:204; rev:1;)" from file
/var/lib/suricata/rules/http-test.rules at line 17

I looked at the source code, and it does not seem to be possible to match
on the protocol key. And for http, it seems like we need to specify the
exact fields, like http.response_body. However, we cannot specify both
streaming and http.xxx keys as it looks like the engine can only support
one buffer when http traffic is processed.

Now, knowing that, if I change the init function to be:

function init (args)
    local needs = {}
    needs["type"] = "streaming"
    return needsend

I get the following error:
17/3/2020 -- 10:46:01 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] -
unsupported data type type
17/3/2020 -- 10:46:01 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] -
error parsing signature "alert http any any -> any any (msg:"match http
body lua";content:"GET"; http_method;
luajit:scripts/http-body-content-detection.lua;sid:204; rev:1;)" from file
/var/lib/suricata/rules/http-test.rules at line 17

Looking at source code, it looks like it is not so much the type key that
needs to be used, but the stream key. Here is what it should look like:

function init (args)
    local needs = {}
    needs["stream"] = tostring(true)
    return needsend

I think there are a couple of places in the doc that should need to be
fixed to comply with the actual source code. First, in the
https://suricata.readthedocs.io/en/suricata-4.1.6/lua/lua-functions.html#streaming-data
section. And also in the HTTP section, we should stop making reference to
the protocol key, as this key is not recognized. We should instead make
reference to the specific http keys, like http.response_body in the
examples.

While researching this problem, I also encountered a mail on this mailing
list that reported that the statement needs["protocol"] = "tls" encountered
similar problem and documentation was supposed to be updated with
needs["tls"] = tostring(true). However, when I test this statement in a LUA
script, I get the following error:
17/3/2020 -- 10:57:23 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] -
error parsing signature "alert http any any -> any any (msg:"match http
body lua";content:"GET"; http_method;
luajit:scripts/http-body-content-detection.lua;sid:204; rev:1;)" from file
/var/lib/suricata/rules/http-test.rules at line 17
17/3/2020 -- 10:57:23 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] -
couldn't load file: cannot open : No such file or directory
17/3/2020 -- 10:57:23 - <Error> - [ERRCODE: SC_ERR_DETECT_PREPARE(173)] -
setting up thread local detect ctx for keyword "lua" failed
17/3/2020 -- 10:57:23 - <Error> - [ERRCODE: SC_ERR_LUA_ERROR(212)] -
couldn't load file: cannot open : No such file or directory
17/3/2020 -- 10:57:23 - <Error> - [ERRCODE: SC_ERR_DETECT_PREPARE(173)] -
setting up thread local detect ctx for keyword "lua" failed

I can't really explain this error. Is TLS support in LUA broken in 4.1.6?

Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20200317/95738e8b/attachment.html>