[Oisf-users] OT: A question about ELK and Suricata

[Oliver Humpage]

> On 6 Apr 2017, at 13:46, C. L. Martinez <carlopmart at gmail.com> wrote:
> 
> And my last question: searching over the web to think about how to install and implement this solution, I see a lot of people use Elasticsearch 2.X/Logstash 2.X/Kibana 3.X or 4.X.. Any technical reason for not to use Elasticsearc/Logstash/Kibana 5??

ELK’s been undergoing a lot of change recently, and it can be quite hard work to update the stack due to breaking changes. If you’re starting from scratch, though, you’re probably OK to use the latest. (We also run all the ELK stuff on FreeBSD, even the Java-based bits - it’s not that bad!)

As for remote logging... this is general advice rather than suricata-specific, but we’ve found RabbitMQ to be a very good solution. If a host’s software already has a RabbitMQ plugin (eg pmacct) then it talks directly to the logging cluster’s RabbitMQ servers. If not, we use some very basic logstash instances in the cluster to receive logs and put them straight into RabbitMQ.

Then some more complicated logstash processes (i.e. with all the filters/munging/etc) take messages out of the queue and pump into ElasticSearch.

This all seems to be pretty robust, and also allows for easy changes/upgrades to the logstash/ES instances without losing any log lines.

HTH

Oliver.