[Oisf-users] Hyperscan on RHEL or CentOS

Wed Mar 29 19:55:15 UTC 2017

Hello Justin -

Thank for that info.  I went back and redid the bootstrap stage, and I needed to add the bzip2 libs before the ./b2 would work correctly.  That worked well enough so that when cmake runs the output identifies the Boost version and Build date.  Unfortunately, I am still seeing the errors from cmake -

There were additional errors - which were primarily from missing dependancies, some of which I don't recall seeing.  Installing these got me past errors when running ./b2 and eventually to compile the Hyperscan and subsequently Suricata with HS.  The dependencies were:

bzip2
bzip2-devel
sqlite-devel

Thanks again for the input.  Much appreciated.

Sean

From: Viiret, Justin [mailto:justin.viiret at intel.com]
Sent: Tuesday, March 28, 2017 18:20 PM
To: Cloherty, Sean E <scloherty at mitre.org>; Spransy, Derek <dsprans at emory.edu>; oisf-users at lists.openinfosecfoundation.org
Subject: RE: Hyperscan on RHEL or CentOS

Hi all,

(I work on the Hyperscan project at Intel)

Hyperscan's dependency on Boost is a "header-only" dependency - that is, we don't use any of the small number of Boost libraries that require it to be compiled. You should just be able to copy or symlink the Boost headers into <hyperscan-source>/include/boost without having to go through its build process.

You can find some information in the "Very Quick Start" section of our documentation here:

    http://01org.github.io/hyperscan/dev-reference/getting_started.html#boost-headers

Hope that helps!

Regards,
    Justin

From: Oisf-users [mailto:oisf-users-bounces at lists.openinfosecfoundation.org] On Behalf Of Cloherty, Sean E
Sent: Wednesday, March 29, 2017 6:48 AM
To: Spransy, Derek <dsprans at emory.edu<mailto:dsprans at emory.edu>>; oisf-users at lists.openinfosecfoundation.org<mailto:oisf-users at lists.openinfosecfoundation.org>
Subject: Re: [Oisf-users] Hyperscan on RHEL or CentOS

Further but not there yet.  Boost ended with this -

...failed updating 2 targets...
...skipped 6 targets...
...updated 1126 targets...

Scrolling back I did find these errors but I am not sure if they are relevant or not -

...failed gcc.compile.c++ bin.v2/libs/sync/build/gcc-4.8.5/release/link-static/threading-multi/tss_pthread.o...

 ...skipped <pbin.v2/libs/sync/build/gcc-4.8.5/release/threading-multi>libboost_sync.so.1.63.0 for lack of <pbin.v2/libs/sync/build/gcc-4.8.5/release/threading-multi>tss_pthread.o...

From: Spransy, Derek [mailto:dsprans at emory.edu]
Sent: Tuesday, March 28, 2017 14:24 PM
To: Cloherty, Sean E <scloherty at mitre.org<mailto:scloherty at mitre.org>>; oisf-users at lists.openinfosecfoundation.org<mailto:oisf-users at lists.openinfosecfoundation.org>
Subject: Re: Hyperscan on RHEL or CentOS

cmake -DBUILD_STATIC_AND_SHARED=1 -DBOOST_ROOT=/home/<user>/boost_1_60_0/ ../

- No problem. As I recall I had some trouble at this step as well, but ultimately figured it out.

Two follow up questions if I could -

1.  Does it matter what directory you are in when you invoke git for the Hyperscan package?

- No, just as long as you have the requisite permissions.

2.  Does/Should the boost directory be in a specific users' home directory (like the account that you use to run Suricata), or is it not consequential at all?

- This shouldn't make any difference either, as long as you have permissions. I run pretty much everything out of my home folder when doing this type of work.

From: Spransy, Derek [mailto:dsprans at emory.edu]
Sent: Tuesday, March 28, 2017 12:21 PM
To: Cloherty, Sean E <scloherty at mitre.org<mailto:scloherty at mitre.org>>; oisf-users at lists.openinfosecfoundation.org<mailto:oisf-users at lists.openinfosecfoundation.org>
Subject: Re: Hyperscan on RHEL or CentOS

These are my notes from installing HS and pf_ring support on RHEL 7.

Install with Intel Hyperscan Enabled

Install pre-requisites

sudo yum install cmake gcc-c++ python-devel

Download ragel, unpack, ./configure, make, sudo make install

Download and compile boost headers

Download boost 1.60

tar xvzf boost_1_60_0.tar.gz

cd boost_1_60_0

./bootstrap.sh

./b2

Install Hyperscan

git clone https://github.com/01org/hyperscan
[https://avatars3.githubusercontent.com/u/1635439?v=3&s=400]<https://github.com/01org/hyperscan>

GitHub - 01org/hyperscan: High-performance regular ...<https://github.com/01org/hyperscan>
github.com
README.md Hyperscan. Hyperscan is a high-performance multiple regex matching library. It follows the regular expression syntax of the commonly-used libpcre library ...

cd hyperscan

mkdir build

cd build

cmake -DBUILD_STATIC_AND_SHARED=1 -DBOOST_ROOT=/home/<user>/boost_1_60_0/ ../

make

sudo make install

Compile Suricate with HS and PF_RING support

./configure --prefix=/usr --sysconfdir=/etc --enable-pfring --with-libpfring-includes=/usr/local/include --with-libpfring-libraries=/usr/local/lib --with-libnspr-includes=/usr/include/nspr4/ --with-libnspr-libraries=/usr/include/nspr4/ --with-libcap_ng-libraries=/usr/local/lib --with-libhs-includes=/usr/local/include/hs/ --with-libhs-libraries=/usr/local/lib/

mpm-algo and spm-algo values in suricata.yaml must be set to 'auto' or 'hs'

________________________________

From: Oisf-users <oisf-users-bounces at lists.openinfosecfoundation.org<mailto:oisf-users-bounces at lists.openinfosecfoundation.org>> on behalf of Cloherty, Sean E <scloherty at mitre.org<mailto:scloherty at mitre.org>>
Sent: Tuesday, March 28, 2017 12:15 PM
To: oisf-users at lists.openinfosecfoundation.org<mailto:oisf-users at lists.openinfosecfoundation.org>
Subject: [Oisf-users] Hyperscan on RHEL or CentOS

Has anyone got instructions for installing Hyperscan on RHEL/CentOS?  I've tried a few times now and it seems like I get fairly close, but I've not been able to compile Suricata with Hyperscan.  I know that it is something I am completing incorrectly but have not been able to figure it out.   Are there files or configuration changes that I can check at the end of the install to see if it was completed correctly prior to compiling Suricata?

Thanks.

Sean Cloherty

InfoSec Engineer/Scientist, Lead

MITRE Corporation

office (781) 271-3707

cell      (781) 697-8043

________________________________

This e-mail message (including any attachments) is for the sole use of
the intended recipient(s) and may contain confidential and privileged
information. If the reader of this message is not the intended
recipient, you are hereby notified that any dissemination, distribution
or copying of this message (including any attachments) is strictly
prohibited.

If you have received this message in error, please contact
the sender by reply e-mail message and destroy all copies of the
original message (including attachments).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openinfosecfoundation.org/pipermail/oisf-users/attachments/20170329/0143acd6/attachment-0002.html>