[Oisf-devel] RFC: lua scripting flowvar access
Vincent Fang
vincent.y.fang at gmail.com
Mon May 20 17:45:48 UTC 2013
One minor issue I'm seeing and I'll be posting a tidbit of the script where
it obtains the string and writes it back into the flowvar, I'm seeing some
kind of binary value being appended to the string when I retrieve the flow
var after it's been set with a string value the first time.
--local a = tostring(args["packet"])
local a = tostring(args["http.response_body"])
--local a = tostring(args["payload"])
local var = ScFlowvarGet(0)
local count = ScFlowintIncr(0)
local file = io.open("luaoutput", "a")
if(file == nil) then
io.write("failed to open luaoutput")
end
file:write("\n[HTTP RESPONSE BODY BEGIN]\n")
file:write(a)
file:write("\n")
if(var == nil) then
file:write("\nvar is nil\n")
var = "mew"
else
file:write("\nvar value is " .. var .. "\n")
end
ScFlowvarSet(0, var, 3)
file:write("\n")
file:write("\nCount is " .. count .. "\n")
file:write("\n[HTTP RESPONSE BODY END]\n")
in my output log I'm seeing this being appended to the var variable.
var value is mew^@
I've tried to use #var instead of hard coding the value 3, and as the flow
continued beyond the count of 2, the number of ^@ increased as well. For
example
var value is mew^@^@^@^@^@^@^@^@^@^@^@
Count is 5
Vince
On Mon, May 20, 2013 at 1:32 PM, Vincent Fang <vincent.y.fang at gmail.com>wrote:
> I apologize for the misinformation. I ran another test again and realized
> that the reason I'm seeing nil so many times is due to the fact many flows
> are occurring not just one. I ran both the ScFlowvarGet(0) and
> ScFlowintIncr(0) and looked for any counts that were 2 instead of 1, and
> saw that the variable was storing the string correctly on the 2nd run of
> the same flow. Everything with the flowvar appears to work as expected.
>
> Vince
>
>
> On Sat, May 18, 2013 at 11:49 AM, Vincent Fang <vincent.y.fang at gmail.com>wrote:
>
>> So in the init function I have this one key line for the flow var:
>>
>> needs["flowvar"] = {"var"}
>>
>> and in the match function, the lines related to the flowvar are:
>>
>> local var = ScFlowvarGet(0)
>> if(var == nil) then
>>
>> io.write("\n var was nil \n")
>> var = "mew"
>>
>> end
>> ScFlowvarSet(0, var, #var)
>>
>> and the result I get is that message var was nil everytime as if the
>> ScFlowvarSet doesn't do anything for a string.
>>
>>
>> Vince
>>
>>
>> On Fri, May 17, 2013 at 2:37 PM, Victor Julien <victor at inliniac.net>wrote:
>>
>>> On 05/17/2013 07:53 PM, Vincent Fang wrote:
>>> > Tested out this feature and was wondering if it's possible for it to
>>> > store a string into the flowvar? I attempted to test that out and kept
>>> > getting nil values from the ScFlowvarGet. The main goal is to find out
>>> > and keep track not only what portion of the flow the lua script is
>>> > examining but which flow it is for the administrator to see with that
>>> > unique string var.
>>>
>>> That should be possible, ya. Can you share (part of) your script?
>>>
>>> Cheers,
>>> Victor
>>>
>>> > On Tue, Apr 23, 2013 at 6:07 AM, Victor Julien <victor at inliniac.net
>>> > <mailto:victor at inliniac.net>> wrote:
>>> >
>>> > On 04/22/2013 06:05 PM, Victor Julien wrote:
>>> > > On 04/18/2013 06:01 PM, Victor Julien wrote:
>>> > >> Funded by Emerging Threats, I've been working on giving the lua
>>> > scripts
>>> > >> access to flowvars.
>>> > >>
>>> > >> Currently only "flowvars" are done, "flowints" will be next.
>>> Please
>>> > >> review the code at:
>>> > >> https://github.com/inliniac/suricata/tree/dev-lua-flowvar
>>> > >>
>>> > >> Pcre based flowvar capturing is done in a post-match fashion.
>>> If the
>>> > >> rule containing the "capture" matches, the var is stored in the
>>> flow.
>>> > >>
>>> > >> For lua scripting, this wasn't what the rule writers wanted. In
>>> this
>>> > >> case, the flowvars are stored in the flow regardless of a rule
>>> match.
>>> > >>
>>> > >> The way a script can start using flowvars is by first
>>> registering
>>> > which
>>> > >> one it needs access to:
>>> > >>
>>> > >> function init (args)
>>> > >> local needs = {}
>>> > >> needs["http.request_headers.raw"] = tostring(true)
>>> > >> needs["flowvar"] = {"cnt"}
>>> > >> return needs
>>> > >> end
>>> > >>
>>> > >> More than one can be registered, e.g.:
>>> > >>
>>> > >> needs["flowvar"] = {"cnt", "somevar", "anothervar" }
>>> > >>
>>> > >> The maximum is 15 per script. The order of the vars matters. As
>>> > Suricata
>>> > >> uses id's internally, to use the vars you have to use id's as
>>> > well. The
>>> > >> first registered var has id 0, 2nd 1 and so on:
>>> > >>
>>> > >> function match(args)
>>> > >> a = ScFlowvarGet(0);
>>> > >> if a then
>>> > >> print ("We have an A: " .. (a))
>>> > >> a = tostring(tonumber(a)+1)
>>> > >> print ("A incremented to: " .. (a))
>>> > >> ScFlowvarSet(0, a, #a)
>>> > >> else
>>> > >> print "Init A to 1"
>>> > >> a = tostring(1)
>>> > >> ScFlowvarSet(0, a, #a)
>>> > >> end
>>> > >>
>>> > >> print ("A is " .. (a))
>>> > >> if tonumber(a) == 23 then
>>> > >> print "Match!"
>>> > >> return 1
>>> > >> end
>>> > >>
>>> > >> return 0
>>> > >> end
>>> > >>
>>> > >> You can also use a var:
>>> > >>
>>> > >> function init (args)
>>> > >> local needs = {}
>>> > >> needs["http.request_headers.raw"] = tostring(true)
>>> > >> needs["flowvar"] = {"blah", "cnt"}
>>> > >> return needs
>>> > >> end
>>> > >>
>>> > >> local var_cnt = 1
>>> > >>
>>> > >> function match(args)
>>> > >> a = ScFlowvarGet(var_cnt);
>>> > >> if a then
>>> > >> print ("We have an A: " .. (a))
>>> > >> a = tostring(tonumber(a)+1)
>>> > >> print ("A incremented to: " .. (a))
>>> > >> ScFlowvarSet(var_cnt, a, #a)
>>> > >> else
>>> > >> print "Init A to 1"
>>> > >> a = tostring(1)
>>> > >> ScFlowvarSet(var_cnt, a, #a)
>>> > >> end
>>> > >>
>>> > >> print ("A is " .. (a))
>>> > >> if tonumber(a) == 23 then
>>> > >> print "Match!"
>>> > >> return 1
>>> > >> end
>>> > >>
>>> > >> return 0
>>> > >> end
>>> > >>
>>> > >> Flowvars are set at the end of the rule's inspection, so after
>>> the
>>> > >> script has run.
>>> > >>
>>> > >> When multiple stores are done from the script and/or pcre, the
>>> last
>>> > >> match will win. So if order matters, rule priority can be used
>>> to
>>> > >> control inspection order.
>>> > >>
>>> > >> Thoughts, comments, and code review highly welcomed.
>>> > >>
>>> > >
>>> > > Updated branch:
>>> > > https://github.com/inliniac/suricata/tree/dev-lua-flowvar-v1.1
>>> > >
>>> > > - Adds flowint support:
>>> > >
>>> > > function init (args)
>>> > > local needs = {}
>>> > > needs["http.request_headers"] = tostring(true)
>>> > > needs["flowint"] = {"cnt"}
>>> > > return needs
>>> > > end
>>> > >
>>> > > function match(args)
>>> > > a = ScFlowintGet(0);
>>> > > if a then
>>> > > ScFlowintSet(0, a + 1)
>>> > > else
>>> > > ScFlowintSet(0, 1)
>>> > > end
>>> > >
>>> > > a = ScFlowintGet(0);
>>> > > if a == 23 then
>>> > > return 1
>>> > > end
>>> > >
>>> > > return 0
>>> > > end
>>> > >
>>> > > return 0
>>> > >
>>> > > Sets are real time, so are done regardless of script match or
>>> rule
>>> > match.
>>> > >
>>> > > - Converts flowvar sets to real time, to fix some var overwrite
>>> issues
>>> > > in HTTP header inspection.
>>> > >
>>> >
>>> > https://github.com/inliniac/suricata/tree/dev-lua-flowvar-v1.2
>>> >
>>> > Adds in ScFlowintIncr & ScFlowintDecr. From the commit:
>>> >
>>> > "Add flowint lua functions for incrementing and decrementing
>>> flowints.
>>> >
>>> > First use creates the var and inits to 0. So a call:
>>> >
>>> > a = ScFlowintIncr(0)
>>> >
>>> > Results in a == 1.
>>> >
>>> > If the var reached UINT_MAX (2^32), it's not further incremented.
>>> If the
>>> > var reaches 0 it's not decremented further.
>>> >
>>> > Calling ScFlowintDecr on a uninitialized var will init it to 0.
>>> >
>>> > Example script:
>>> >
>>> > function init (args)
>>> > local needs = {}
>>> > needs["http.request_headers"] = tostring(true)
>>> > needs["flowint"] = {"cnt_incr"}
>>> > return needs
>>> > end
>>> >
>>> > function match(args)
>>> > a = ScFlowintIncr(0);
>>> > if a == 23 then
>>> > return 1
>>> > end
>>> >
>>> > return 0
>>> > end
>>> > return 0
>>> >
>>> > This script matches the 23rd time it's invoked on a flow."
>>> >
>>> > --
>>> > ---------------------------------------------
>>> > Victor Julien
>>> > http://www.inliniac.net/
>>> > PGP: http://www.inliniac.net/victorjulien.asc
>>> > ---------------------------------------------
>>> >
>>> > _______________________________________________
>>> > Suricata IDS Devel mailing list:
>>> > oisf-devel at openinfosecfoundation.org
>>> > <mailto:oisf-devel at openinfosecfoundation.org>
>>> > Site: http://suricata-ids.org | Participate:
>>> > http://suricata-ids.org/participate/
>>> > List:
>>> >
>>> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
>>> > Redmine: https://redmine.openinfosecfoundation.org/
>>> >
>>> >
>>>
>>>
>>> --
>>> ---------------------------------------------
>>> Victor Julien
>>> http://www.inliniac.net/
>>> PGP: http://www.inliniac.net/victorjulien.asc
>>> ---------------------------------------------
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-devel/attachments/20130520/30db2899/attachment-0002.html>
More information about the Oisf-devel
mailing list