[Oisf-users] Suricata newbie and (Lua?) rule scripting

[James]

Summary question: to what extent is Suricata's rule scripting capability 
intended to allow non-rocket-scientist programmers such as myself to use 
Suricata more as a "firewall enhancement" on an individual server than 
as a larger scale environment intrusion detection system?

Background: a few months ago, I worked up enough ambition to try to 
queue packets out from nftables in order to handle more convoluted 
accept/drop logic than pure nftables provides, but using a definite 
amateur-level scripting language approach (which is all that I can handle).

I would have preferred to use node.js so that I would be able to code 
using ECMAScript but its ecosystem isn't (yet) as mature and 
well-rounded as, for example, Python's or Perl's and I ended up using 
Perl.  I did stumble and got roadblocked at one point but Pierre 
Chifflier helpfully pointed out some details that I was missing and for 
the last month or so I have been happily queuing out IPv4 and IPv6 
packets and successfully dropping the vast majority of DNS and SMTP 
accesses that I don't like... on the first packet, before they hit their 
respective applications.  Awesome.

But one part that I didn't get working yet is handling DNS TCP packets. 
Since I get very few of those that I want to drop it hasn't been a big 
deal, but now I've worked up enough ambition to try to do something 
about that and my meandering ponderings have caused my brain's awareness 
of Suricata to hit critical mass.

So now I'm looking at Suricata as a possible framework to do the heavy 
lifting for me and let me simply (possibly? hopefully?) plug in some 
light-weight scripts to handle my specific traffic-molesting logic.

Current status:
* Suricata installed from main packages on Ubuntu 16.10 and 17.04
* Suricata config'd for minimal rules, with only alerts and drops logged
* Suricata running with "-q 1 -q 2 -q 3 -q 4" and repeat+mark+mask
* nftables running with "mark & 0x1 != 0x1 queue num 1-4 bypass,fanout"
* Suricata is playing nicely with the rest of my nftables rule set
* Suricata resource usage is low (relative to what's available)
* current Suricata alerting/dropping is minimal and seems reasonable

Initial result: "nftables+Suricata as IPS" dial tone test works nicely.

On to Suricata rule scripting!

Currently, I'm using nftables -> queue target -> nfqueue-bindings (for 
Perl) -> Net::Frame and Net::DNS for:

a) SMTP:
* grab IPv4/IPv6 source address
* do forward DNS calls out to RBL providers (e.g., SpamHaus), and
* do reverse DNS calls to get CNAMEs/PTRs, then
* juggle results of previous two steps and decide to accept or drop

b) DNS:
* grab IPv4/IPv6 DNS request packet (UDP so far, TCP falls over on me)
* if recursion requested, or
* if not for one of my domains, or
* if not for "IN" class, then
* drop

And I retrieve and log various details for both (e.g., get informational 
TXT RRs when the DNS RBLs offer unhappy A RRs).

Initial "Hello World" Suricata rule and script equals:

  alert dns any any -> any any (msg:"LUAJIT Hello World"; 
luajit:helloworld.lua;)

and:

  function init (args)
   local needs = {}
   return needs
  end

  function match(args)
   return 1
  end

and indeed "LUAJIT Hello World" shows up in eve.json every time a DNS 
packet floats by.

Now update the match function to:

  function match(args)
   if DnsGetRecursionDesired() == true then
    return 1
   else
    return 0
   end
  end

and... nothing happens?

Wait, I read about that "DnsGetRecursionDesired" on:

  https://suricata.readthedocs.io/en/latest/output/lua-output.html

so it can be used only for Lua output scripts?

Argh.

OK, back the bus up to:

  https://suricata.readthedocs.io/en/latest/rules/rule-lua-scripting.html

and... the only non-HTTP parameters are "packet" and "payload"?

Argh.

OK, how about some Lua run-time/library/community/modules/etc. to peel 
apart "packet" and/or "payload"?

Um... well, sort of, but not really extensive, and nothing immediately 
useful leaps out at me.  In fact, node.js seems to already be better.

Argh.

Soooo... back to the original question, which could now be re-phrased as 
"am I hoping for too much from Suricata's rule scripting capabilities, 
or outright misinterpreting its intended use and I shouldn't even be 
trying this?"

What *is* the intended use, and especially the intended audience, of 
Suricata's rule scripting capabilities?

Maybe a different (or additional) scripting language could offload 
fulfilling my hopes from the Suricata developers and onto the selected 
scripting language's community of run-time resources?

Comments?  Complaints?  Screams of pain?

Thanks.

-- 

  - James