syslog-ng + Kibana

[bbzz]

Hi guys.

I've been using stock syslog for years now mostly due to it being in base and not needing extra stuff.
Recently I switched to sysutils/syslog-ng and would like to have this stored in textproc/elasticsearch and visualized in textproc/kibana3.
sysutils/syslog-ng documentation says:

Note the following limitations when using the syslog-ng OSE elasticsearch destination:
[LIST]
[*]This destination is only supported on the Linux platform.[/LIST]

If this could be done on FreeBSD could someone kindly point me to some easy to follow tutorials.

Regards!

 

[pczanik]

Hi,

It is possible to use elasticsearch using the sysutils/syslog-ng port, but only version 1.X is supported and I only tested it once almost a year ago. So, it's not really well tested. You need to build syslog-ng yourself, enable the JAVA option, which needs an active Internet connection, as it downloads many JAR files from Marvel repos. Once that is ready, you will need some hacking about libjvm.so, https://czanik.blogs.balabit.com/2016/03/troubleshooting-java-support-in-syslog-ng/ can give you some hints. And you need to configure syslog-ng, where the documentation can help you.

Good luck!

 

[pczanik]

I did not have time for testing, but compiling succeeded after installing misc/compat9x (it was needed by Gradle, as far as I could see).

 

[vejnovic]

I'm using ELK (textproc/elasticsearch, sysutils/logstash and textproc/kibana43) and works great.

You can try:
https://blog.gufi.org/2016/02/15/elk-first-part/
https://blog.gufi.org/2016/02/23/elk-stack-elasticsearch-logstash-and-kibana-on-freebsd-part-2/
https://blog.gufi.org/2016/03/16/elk-stack-elasticsearch-logstash-and-kibana-on-freebsd-part-3/

 

[bbzz]

I'll have a look at that, awesome.

 

[Rodrick Bouchard]

Hello

I installed ELK onto our FreeBSD system. It works fine but I'm having an issue with the way kibana handles Number fields.

I declared some fields as float using logstash, but kibana recognises them as '?', and not '#'. Making it impossible to make a usable diagram or data analysis. I tried to address this with the folks of elastic.co but all I get is static.

Does any one have an idea?

 

[Oko]

Hello

I installed ELK onto our fBSD system. It works fine but I'm having an issue with the way kibana handles Number fields.

Maybe you could try echofish instead

https://echothrust.github.io/echofish/

which is developed on OpenBSD so should compile fine on FreeBSD.

 

[SirDice]

I installed this for a client and remembered this thread. Client already had a complete centralized logging based on syslog-ng. I didn't use the ElasticSearch addon for Syslog-NG, we have a central repository and all servers use the same syslog-ng package.

In syslog-ng.conf I simply defined a destination:

destination d_logstash { tcp("127.0.0.1" port(9999)); };

And added an additional log line:

log { source(s_central); destination(central); };

Then for logstash.conf you need to create something that will accept the logs:

input {
  tcp {
    mode => server
    port => 9999
    type => "syslog-relay"
  }
}

The rest of the configuration of Logstash, ElasticSearch and Kibana is pretty much standard.