[Oisf-devel] RFC: lua scripting flowvar access
Victor Julien
victor at inliniac.net
Tue Dec 10 14:34:18 UTC 2013
On 05/20/2013 07:45 PM, Vincent Fang wrote:
> One minor issue I'm seeing and I'll be posting a tidbit of the script
> where it obtains the string and writes it back into the flowvar, I'm
> seeing some kind of binary value being appended to the string when I
> retrieve the flow var after it's been set with a string value the first
> time.
I believe this is fixed in the current git master, by commit:
commit 81ee6f5aadeb8e91c97f4b66ecab36f40086c077
Author: Victor Julien <victor at inliniac.net>
Date: Tue Nov 26 18:30:04 2013 +0100
lua: push correct length back through ScFlowvarGet, work around
valgrind warning
Please let me know if you still see this in the recent code.
Cheers,
Victor
>
> --local a = tostring(args["packet"])
> local a = tostring(args["http.response_body"])
> --local a = tostring(args["payload"])
> local var = ScFlowvarGet(0)
> local count = ScFlowintIncr(0)
> local file = io.open("luaoutput", "a")
> if(file == nil) then
> io.write("failed to open luaoutput")
> end
>
> file:write("\n[HTTP RESPONSE BODY BEGIN]\n")
> file:write(a)
> file:write("\n")
> if(var == nil) then
> file:write("\nvar is nil\n")
> var = "mew"
> else
> file:write("\nvar value is " .. var .. "\n")
> end
> ScFlowvarSet(0, var, 3)
> file:write("\n")
> file:write("\nCount is " .. count .. "\n")
> file:write("\n[HTTP RESPONSE BODY END]\n")
>
> in my output log I'm seeing this being appended to the var variable.
>
> var value is mew^@
>
> I've tried to use #var instead of hard coding the value 3, and as the
> flow continued beyond the count of 2, the number of ^@ increased as
> well. For example
>
> var value is mew^@^@^@^@^@^@^@^@^@^@^@
>
>
> Count is 5
>
>
>
> Vince
>
>
> On Mon, May 20, 2013 at 1:32 PM, Vincent Fang <vincent.y.fang at gmail.com
> <mailto:vincent.y.fang at gmail.com>> wrote:
>
> I apologize for the misinformation. I ran another test again and
> realized that the reason I'm seeing nil so many times is due to the
> fact many flows are occurring not just one. I ran both the
> ScFlowvarGet(0) and ScFlowintIncr(0) and looked for any counts that
> were 2 instead of 1, and saw that the variable was storing the
> string correctly on the 2nd run of the same flow. Everything with
> the flowvar appears to work as expected.
>
> Vince
>
>
> On Sat, May 18, 2013 at 11:49 AM, Vincent Fang
> <vincent.y.fang at gmail.com <mailto:vincent.y.fang at gmail.com>> wrote:
>
> So in the init function I have this one key line for the flow var:
>
> needs["flowvar"] = {"var"}
>
> and in the match function, the lines related to the flowvar are:
>
> local var = ScFlowvarGet(0)
> if(var == nil) then
>
> io.write("\n var was nil \n")
> var = "mew"
>
> end
> ScFlowvarSet(0, var, #var)
>
> and the result I get is that message var was nil everytime as if
> the ScFlowvarSet doesn't do anything for a string.
>
>
> Vince
>
>
> On Fri, May 17, 2013 at 2:37 PM, Victor Julien
> <victor at inliniac.net <mailto:victor at inliniac.net>> wrote:
>
> On 05/17/2013 07:53 PM, Vincent Fang wrote:
> > Tested out this feature and was wondering if it's possible
> for it to
> > store a string into the flowvar? I attempted to test that
> out and kept
> > getting nil values from the ScFlowvarGet. The main goal is
> to find out
> > and keep track not only what portion of the flow the lua
> script is
> > examining but which flow it is for the administrator to
> see with that
> > unique string var.
>
> That should be possible, ya. Can you share (part of) your
> script?
>
> Cheers,
> Victor
>
> > On Tue, Apr 23, 2013 at 6:07 AM, Victor Julien
> <victor at inliniac.net <mailto:victor at inliniac.net>
> > <mailto:victor at inliniac.net <mailto:victor at inliniac.net>>>
> wrote:
> >
> > On 04/22/2013 06:05 PM, Victor Julien wrote:
> > > On 04/18/2013 06:01 PM, Victor Julien wrote:
> > >> Funded by Emerging Threats, I've been working on
> giving the lua
> > scripts
> > >> access to flowvars.
> > >>
> > >> Currently only "flowvars" are done, "flowints" will
> be next. Please
> > >> review the code at:
> > >>
> https://github.com/inliniac/suricata/tree/dev-lua-flowvar
> > >>
> > >> Pcre based flowvar capturing is done in a
> post-match fashion. If the
> > >> rule containing the "capture" matches, the var is
> stored in the flow.
> > >>
> > >> For lua scripting, this wasn't what the rule
> writers wanted. In this
> > >> case, the flowvars are stored in the flow
> regardless of a rule match.
> > >>
> > >> The way a script can start using flowvars is by
> first registering
> > which
> > >> one it needs access to:
> > >>
> > >> function init (args)
> > >> local needs = {}
> > >> needs["http.request_headers.raw"] = tostring(true)
> > >> needs["flowvar"] = {"cnt"}
> > >> return needs
> > >> end
> > >>
> > >> More than one can be registered, e.g.:
> > >>
> > >> needs["flowvar"] = {"cnt", "somevar",
> "anothervar" }
> > >>
> > >> The maximum is 15 per script. The order of the vars
> matters. As
> > Suricata
> > >> uses id's internally, to use the vars you have to
> use id's as
> > well. The
> > >> first registered var has id 0, 2nd 1 and so on:
> > >>
> > >> function match(args)
> > >> a = ScFlowvarGet(0);
> > >> if a then
> > >> print ("We have an A: " .. (a))
> > >> a = tostring(tonumber(a)+1)
> > >> print ("A incremented to: " .. (a))
> > >> ScFlowvarSet(0, a, #a)
> > >> else
> > >> print "Init A to 1"
> > >> a = tostring(1)
> > >> ScFlowvarSet(0, a, #a)
> > >> end
> > >>
> > >> print ("A is " .. (a))
> > >> if tonumber(a) == 23 then
> > >> print "Match!"
> > >> return 1
> > >> end
> > >>
> > >> return 0
> > >> end
> > >>
> > >> You can also use a var:
> > >>
> > >> function init (args)
> > >> local needs = {}
> > >> needs["http.request_headers.raw"] = tostring(true)
> > >> needs["flowvar"] = {"blah", "cnt"}
> > >> return needs
> > >> end
> > >>
> > >> local var_cnt = 1
> > >>
> > >> function match(args)
> > >> a = ScFlowvarGet(var_cnt);
> > >> if a then
> > >> print ("We have an A: " .. (a))
> > >> a = tostring(tonumber(a)+1)
> > >> print ("A incremented to: " .. (a))
> > >> ScFlowvarSet(var_cnt, a, #a)
> > >> else
> > >> print "Init A to 1"
> > >> a = tostring(1)
> > >> ScFlowvarSet(var_cnt, a, #a)
> > >> end
> > >>
> > >> print ("A is " .. (a))
> > >> if tonumber(a) == 23 then
> > >> print "Match!"
> > >> return 1
> > >> end
> > >>
> > >> return 0
> > >> end
> > >>
> > >> Flowvars are set at the end of the rule's
> inspection, so after the
> > >> script has run.
> > >>
> > >> When multiple stores are done from the script
> and/or pcre, the last
> > >> match will win. So if order matters, rule priority
> can be used to
> > >> control inspection order.
> > >>
> > >> Thoughts, comments, and code review highly welcomed.
> > >>
> > >
> > > Updated branch:
> > >
> https://github.com/inliniac/suricata/tree/dev-lua-flowvar-v1.1
> > >
> > > - Adds flowint support:
> > >
> > > function init (args)
> > > local needs = {}
> > > needs["http.request_headers"] = tostring(true)
> > > needs["flowint"] = {"cnt"}
> > > return needs
> > > end
> > >
> > > function match(args)
> > > a = ScFlowintGet(0);
> > > if a then
> > > ScFlowintSet(0, a + 1)
> > > else
> > > ScFlowintSet(0, 1)
> > > end
> > >
> > > a = ScFlowintGet(0);
> > > if a == 23 then
> > > return 1
> > > end
> > >
> > > return 0
> > > end
> > >
> > > return 0
> > >
> > > Sets are real time, so are done regardless of script
> match or rule
> > match.
> > >
> > > - Converts flowvar sets to real time, to fix some
> var overwrite issues
> > > in HTTP header inspection.
> > >
> >
> >
> https://github.com/inliniac/suricata/tree/dev-lua-flowvar-v1.2
> >
> > Adds in ScFlowintIncr & ScFlowintDecr. From the commit:
> >
> > "Add flowint lua functions for incrementing and
> decrementing flowints.
> >
> > First use creates the var and inits to 0. So a call:
> >
> > a = ScFlowintIncr(0)
> >
> > Results in a == 1.
> >
> > If the var reached UINT_MAX (2^32), it's not further
> incremented. If the
> > var reaches 0 it's not decremented further.
> >
> > Calling ScFlowintDecr on a uninitialized var will init
> it to 0.
> >
> > Example script:
> >
> > function init (args)
> > local needs = {}
> > needs["http.request_headers"] = tostring(true)
> > needs["flowint"] = {"cnt_incr"}
> > return needs
> > end
> >
> > function match(args)
> > a = ScFlowintIncr(0);
> > if a == 23 then
> > return 1
> > end
> >
> > return 0
> > end
> > return 0
> >
> > This script matches the 23rd time it's invoked on a flow."
> >
> > --
> > ---------------------------------------------
> > Victor Julien
> > http://www.inliniac.net/
> > PGP: http://www.inliniac.net/victorjulien.asc
> > ---------------------------------------------
> >
> > _______________________________________________
> > Suricata IDS Devel mailing list:
> > oisf-devel at openinfosecfoundation.org
> <mailto:oisf-devel at openinfosecfoundation.org>
> > <mailto:oisf-devel at openinfosecfoundation.org
> <mailto:oisf-devel at openinfosecfoundation.org>>
> > Site: http://suricata-ids.org | Participate:
> > http://suricata-ids.org/participate/
> > List:
> >
> https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
> > Redmine: https://redmine.openinfosecfoundation.org/
> >
> >
>
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
>
>
>
--
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------
More information about the Oisf-devel
mailing list