[Oisf-devel] PR: Hyperscan MPM integration

[Victor Julien]

On 16-03-16 10:11, Victor Julien wrote:
> On 16-03-16 05:30, Viiret, Justin wrote:
>> Late last year, Geoff Langdale gave a talk at the OISF Suricata conference about Intel's Hyperscan pattern matcher and an early integration with Suricata. This integration uses Hyperscan as an MPM algorithm, used for bulk literal scanning.
>>
>> I have just submitted a PR on Github with the code, and we would love to see it accepted upstream for more general use within the Suricata community. 
>>
>> The first PR is here: https://github.com/inliniac/suricata/pull/1941
>> Geoff's slides from the conference: https://openisf.files.wordpress.com/2015/11/oisf-keynote-2015-geoff-langdale.pdf
>>
>> (As Geoff says, there is lots of scope for making more use of Hyperscan for regex matching in Suricata further down the track.)
>>
>> You can get Hyperscan from its official site <https://01.org/hyperscan> and you can find a guide to building it in the "Getting Started" section of the documentation linked from that page.
>>
>> Some small caveats:
>>
>> * Hyperscan will run on x86 processors in 64-bit (Intel 64 Architecture) and 32-bit (IA-32 Architecture) modes. At a minimum, support for Supplemental Streaming SIMD Extensions 3 (SSSE3) is required, which should be available on any modern x86 processor.
>> * Hyperscan has a few dependencies (CMake, Boost, Ragel) and require a C++11-capable C++ compiler.
>> * To link Suricata against Hyperscan you must build Hyperscan with shared libraries, as otherwise the C++ standard library won't be pulled in when the Suricata binary is linked.
>>
>> If anyone would like to test the code and see how it performs, we would love to see some feedback!
> 
> Thanks Justin!
> 
> When trying to run it, Suricata dies at start up with:
> 
> [18965] 16/3/2016 -- 10:06:32 - (util-mpm-hs.c:784) <Error>
> (SCHSInitThreadCtx) -- [ERRCODE: SC_ERR_FATAL(171)] - No scratch space
> prototype
> 
> This is on:
> Ubuntu 15.10, on a Intel Atom C2758
> HyperScan 4.0.1 (from githubs v4.0.1 tag)
> 
> victor at c2758:~/dev/surihs$ ldd src/suricata|grep hs
>         libhs.so.4.0 => /usr/local/lib/libhs.so.4.0 (0x00007f0933338000)
> 
> What am I missing?
> 

I think I found the issue: if rules are loaded but non of them have mpm
patterns, then I get the above. If I pass rules with mpm patterns, it
works. So it appears the case of no mpm patterns isn't handled correctly.

-- 
---------------------------------------------
Victor Julien
http://www.inliniac.net/
PGP: http://www.inliniac.net/victorjulien.asc
---------------------------------------------