Web Interface Syslog

mayers

mayers

I ask for advice on a log viewing and search interface, example web based using syslogd, I'm not interested in changing configurations but only consulting the logs, extrapolating the reports, querying the data.

Thank
 
From your description, it sounds like you're interested in something simple, and since you're posting to a FreeBSD forum, I assume you want something that will run on FreeBSD. With those constraints, and without any warranty or recommendation for a particular use, you might consider:
  1. GoAccess -- I just learned about this tool. It's MIT licensed, is designed to be run in a Unix terminal or a web browser, and looks pretty light-weight. It bills itself as a tool for looking at web server logs, but you might be able to customize it for the logs produced by the applications logging to your syslogd. There's already a port for it too: sysutils/goaccess.
  2. Graylog Open -- I've heard of this one, but haven't used it either. It's backed by a commercial company, so there may be other features you can buy should you want to get more functionality out of it. It's available as sysutils/graylog.
  3. The ELK Stack -- Of these three, ELK is the only one I've used, and I wouldn't describe it as "simple". It's sponsored by a company, too, and is the product I've found most people think of when they think of an open-source logging solution. That said, I ran it under FreeBSD a number of years ago, and found that it or the OpenJDK we were using leaked memory badly. After running for a few weeks, parts of the stack would consume all the free memory on the host and crash the system. A fellow on the Internet described running ELK on FreeBSD as recently as 2020, so things may have improved. Depending on the volume of log data you need to review, and the complexity of the reports you want to run, you may find ELK is a lot of effort for your needs. According to that blog, it's available as textproc/elasticsearch8, sysutils/logstash8 and textproc/kibana8, which you stitch together to build the ELK stack.
I have yet to see a wholly-commercial vendor support running their log analysis service on FreeBSD, unfortunately.
 
Last edited:
Hi, yes I am looking for a tool for use on simple freebsd, in my opinion GoAccess could be the correct choice. It is not essential that it is a web interface, I can also connect via ssh and run it from the terminal. I was looking for something to use when analyzing a problem. Example I have 20 switches and I send all the logs to a freebsd ok syslogd machine I am very happy with it but if I have to do searches for example by date and time I have to unpack the file of the day search inside the example file with grep or cat. I think I'll try GoAccess thank you very much
 
mayers said:
... I have to unpack the file of the day search inside the example file with grep or cat. ...
Click to expand...

If you only have to search these logs occasionally, and the volume isn't too great, you might consider building command pipelines instead of uncompressing a file, then filtering through it. I use these command-line tools to make ad hoc queries more efficient:
  • zgrep(1) -- it's a tool dedicated to grep'ing through compressed files. I'm not sure if it's appreciably faster, but I find it cleaner for building pipelines, when i already know what I'm searching for.
  • unzip -c and gunzip -c -- If you need to unzip something first to learn what you want grep for, these commands will uncompress the file and print its contents to standard out, rather than to a file. This is faster than writing the uncompressed contents to a file, gives you ability to process the event logs through a pipeline, and leaves fewer uncompressed copies of the log files lying around.
  • net-mgmt/grepcidr -- This is an elegant little tool for that lets you search a file for IP address CIDR blocks.
Else, good luck with GoAccess!
 
Unfortunately GoAccess is not suitable for interpreting system logs. Requires an IP address in each message as a prerequisite. I'm thinking of making myself something in Javascript.
 
It just occurred to me that Nagios is pretty popular among operations teams for generating monitoring graphics. The typical workflow I've seen is to have it email you a daily report, or to display a page of graphs on a second monitor, in order to keep an eye on your systems. When they show something's wrong, they then use command line tools to explore the logs that feed the graphics to troubleshoot a problem.
 
You must log in or register to reply here.
Back
Top