[Oisf-devel] RFC: lua scripting flowvar access

[Victor Julien]

On 04/23/2013 12:07 PM, Victor Julien wrote:
> On 04/22/2013 06:05 PM, Victor Julien wrote:
>> On 04/18/2013 06:01 PM, Victor Julien wrote:
>>> Funded by Emerging Threats, I've been working on giving the lua scripts
>>> access to flowvars.
>>>
>>> Currently only "flowvars" are done, "flowints" will be next. Please
>>> review the code at:
>>> https://github.com/inliniac/suricata/tree/dev-lua-flowvar
>>>
>>> Pcre based flowvar capturing is done in a post-match fashion. If the
>>> rule containing the "capture" matches, the var is stored in the flow.
>>>
>>> For lua scripting, this wasn't what the rule writers wanted. In this
>>> case, the flowvars are stored in the flow regardless of a rule match.
>>>
>>> The way a script can start using flowvars is by first registering which
>>> one it needs access to:
>>>
>>> function init (args)
>>>     local needs = {}
>>>     needs["http.request_headers.raw"] = tostring(true)
>>>     needs["flowvar"] = {"cnt"}
>>>     return needs
>>> end
>>>
>>> More than one can be registered, e.g.:
>>>
>>>     needs["flowvar"] = {"cnt", "somevar", "anothervar" }
>>>
>>> The maximum is 15 per script. The order of the vars matters. As Suricata
>>> uses id's internally, to use the vars you have to use id's as well. The
>>> first registered var has id 0, 2nd 1 and so on:
>>>
>>> function match(args)
>>>     a = ScFlowvarGet(0);
>>>     if a then
>>>         print ("We have an A: " .. (a))
>>>         a = tostring(tonumber(a)+1)
>>>         print ("A incremented to: " .. (a))
>>>         ScFlowvarSet(0, a, #a)
>>>     else
>>>         print "Init A to 1"
>>>         a = tostring(1)
>>>         ScFlowvarSet(0, a, #a)
>>>     end
>>>
>>>     print ("A is " .. (a))
>>>     if tonumber(a) == 23 then
>>>         print "Match!"
>>>         return 1
>>>     end
>>>
>>>     return 0
>>> end
>>>
>>> You can also use a var:
>>>
>>> function init (args)
>>>     local needs = {}
>>>     needs["http.request_headers.raw"] = tostring(true)
>>>     needs["flowvar"] = {"blah", "cnt"}
>>>     return needs
>>> end
>>>
>>> local var_cnt = 1
>>>
>>> function match(args)
>>>     a = ScFlowvarGet(var_cnt);
>>>     if a then
>>>         print ("We have an A: " .. (a))
>>>         a = tostring(tonumber(a)+1)
>>>         print ("A incremented to: " .. (a))
>>>         ScFlowvarSet(var_cnt, a, #a)
>>>     else
>>>         print "Init A to 1"
>>>         a = tostring(1)
>>>         ScFlowvarSet(var_cnt, a, #a)
>>>     end
>>>
>>>     print ("A is " .. (a))
>>>     if tonumber(a) == 23 then
>>>         print "Match!"
>>>         return 1
>>>     end
>>>
>>>     return 0
>>> end
>>>
>>> Flowvars are set at the end of the rule's inspection, so after the
>>> script has run.
>>>
>>> When multiple stores are done from the script and/or pcre, the last
>>> match will win. So if order matters, rule priority can be used to
>>> control inspection order.
>>>
>>> Thoughts, comments, and code review highly welcomed.
>>>
>>
>> Updated branch:
>> https://github.com/inliniac/suricata/tree/dev-lua-flowvar-v1.1
>>
>> - Adds flowint support:
>>
>> function init (args)
>>     local needs = {}
>>     needs["http.request_headers"] = tostring(true)
>>     needs["flowint"] = {"cnt"}
>>     return needs
>> end
>>
>> function match(args)
>>     a = ScFlowintGet(0);
>>     if a then
>>         ScFlowintSet(0, a + 1)
>>     else
>>         ScFlowintSet(0, 1)
>>     end
>>
>>     a = ScFlowintGet(0);
>>     if a == 23 then
>>         return 1
>>     end
>>
>>     return 0
>> end
>>
>> return 0
>>
>> Sets are real time, so are done regardless of script match or rule match.
>>
>> - Converts flowvar sets to real time, to fix some var overwrite issues
>> in HTTP header inspection.
>>
> 
> https://github.com/inliniac/suricata/tree/dev-lua-flowvar-v1.2
> 
> Adds in ScFlowintIncr & ScFlowintDecr. From the commit:
> 
> "Add flowint lua functions for incrementing and decrementing flowints.
> 
> First use creates the var and inits to 0. So a call:
> 
>     a = ScFlowintIncr(0)
> 
> Results in a == 1.
> 
> If the var reached UINT_MAX (2^32), it's not further incremented. If the
> var reaches 0 it's not decremented further.
> 
> Calling ScFlowintDecr on a uninitialized var will init it to 0.
> 
> Example script:
> 
>     function init (args)
>         local needs = {}
>         needs["http.request_headers"] = tostring(true)
>         needs["flowint"] = {"cnt_incr"}
>         return needs
>     end
> 
>     function match(args)
>         a = ScFlowintIncr(0);
>         if a == 23 then
>             return 1
>         end
> 
>         return 0
>     end
>     return 0
> 
> This script matches the 23rd time it's invoked on a flow."
> 

This code was just merged into the git master.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------