[Oisf-devel] [RFC] library: create a shared library out of Suricata

[Ansis]

On Fri, Apr 17, 2020 at 3:34 AM Victor Julien <lists at inliniac.net> wrote:
>
> Hi Ansis,
>
> On 15-04-2020 06:13, Ansis Atteka wrote:
> > This RFC patch creates libsuricata2.so library that for now merely
> > exports the SuricataMain() function so that suricata daemon could use
> > it. In the future, other functions could be exported as well that
> > would allow other daemons, like Open vSwitch, to call into suricata
> > code base.
> >
> > The long term motiviation behind creating a shared library out of
> > suricata is to:
> > 1. allow IDS integration in packet pipelines that currently don't have
> >    packet acqusition mechanisms in Suricata or where it would be
> >    non-trivial to implement one.
> > 2. achieve better performance by reducing packet copies and context
> >    switches that are necessary to hand off packets to IDS engine.
>
> Not much info here yet, but we're tracking libsuricata here
> https://redmine.openinfosecfoundation.org/issues/2693
>
> Overall I can say that we've started discussing it in a more serious
> way, but also want to do it 'the right way' so just exposing everything
> as it is now is not what we have in mind. We'll want to define a public
> API, for which we can guarantee some level of API stability, etc.

Agree.
>
> Somewhat related: we're experimenting with dynamic plugin support. This
> may be another way to extending suri to work better with other
> tools/frameworks. We hope to get a first preview into suricata 6.

The idea is that suricata would load a plugin with dlopen() call?

>
>
> > With this patch I want to get feedback if decoupling Suricata into a
> > library actually makes sense and if it makes sense to build on top
> > of this patch. In particular I want to start discussion on following
> > topics:
> > 1. Licensing. Open vSwitch is licensed under Apache2 license
> >    whereas Suricata is licensed under GPLv2. Best to my understanding,
> >    if suricata would be converted into a library, then for OVS
> >    to be able to link against such library, Suricata license
> >    would have to be relaxed (e.g. LGPL?). Would something like
> >    this be even possible?
>
> We're not planning to move to a LGPL variant. We might consider GPLv3
> (which is also compatible to APLv2), but that is not a decision we would
> take lightly.
>
>
> > 2. Threading model for Library. At least for OVS use-cases Open vSwitch already
> >    implements its own packet acqusition mechanism (it can get packet via
> >    OVS kernel module Netlink socket, DPDK and possibly in future AF_XDP).
> >    Which means that neither single, autofp or workers mode is appropriate.
> >    One soutlion seems to be to assign ThreadVars variable to OVS threads.
>
> I think you'll be mostly caring about what we call the FlowWorker (this
> runs a pipeline of: flow engine, stream engine, detection engine and
> output engine). This would be reasonably well contained. There are still
> global structures that need to exist to support certain features (e.g.
> for rule thresholding, dataset, and quite a few other things)
>
>
> > 3. Ownership of system resources. The Suricata and OVS should coordinate
> >    on use of per-process system resources to avoid race conditions.
>
> What kind of resources do you have in mind here?

1. Libc logging. In particular ovs and suricata should not be both
attempting to call openlog().
2. Standard Input and Output fds. The suricata library must somehow
know in SCLogMessage() function that the process using the library has
not closed STDOUT or STDERR. Otherwise, functions from suricata
library may write to wrong fds.
3. Signal handlers. Suricata uses signals to get notification when log
file should be rotated or configuration files reloaded. However,
UtilSignalHandlerSetup() function should never override any signal
handlers set by ovs.
4. There are few other like core file config, changing the user under
which process runs...

The 1&2 could be solved by letting OVS to override suricata logging function.

The 3 could be solved by letting the client process that would use the
library to always be responsible for implementing event loops (opposed
to calling functions in suricata that have their own event loops).

All of these of course are solvable.

>
> Regards,
> Victor
>
>
> --
> ---------------------------------------------
> Victor Julien
> http://www.inliniac.net/
> PGP: http://www.inliniac.net/victorjulien.asc
> ---------------------------------------------
>
> _______________________________________________
> Suricata IDS Devel mailing list: oisf-devel at openinfosecfoundation.org
> Site: http://suricata-ids.org | Participate: http://suricata-ids.org/participate/
> List: https://lists.openinfosecfoundation.org/mailman/listinfo/oisf-devel
> Redmine: https://redmine.openinfosecfoundation.org/
>