Solved Complete server monitoring tool

CyberCr33p

Active Member

Reaction score: 14
Messages: 181

So doing a deep look I see Munin is based on database/rrdtool too. Icinga and LibreNMS as well.
What about running monitoring tools from a FreeBSD Linode VM. That is where I think I want to go. Cloud based management.
I think it's good idea to separate zabbix in a VM. As most of our FreeBSD servers are in Hetzner datacenter we use a Hetzner Cloud VPS running Debian (they don't support FreeBSD in their cloud) for Zabbix (I use snmp with custom scripts and not zabbix agents). We also use external monitoring from NodePing.com (they have good prices and so far are very reliable). Finally we use Monit to restart services in case they crash and run some custom scripts for disk cleanups if the disk usage is more than 80%.
 

CyberCr33p

Active Member

Reaction score: 14
Messages: 181

My home Zabbix server runs on a 4 core VM, gave it 24GB of RAM (most of it assigned to the database). At the moment I have 12 hosts (including my external VPS) monitored. The database itself is about 30GB at the moment and I have a couple of months worth of data in it.
It doesn't need so much RAM. The VM I host it has 2GB of RAM and 1 core. It monitors at the moment 58 FreeBSD servers. I run every few months a SQL script to clear old data, but the database sometimes is more than 15GB with no optimisations (just the default debian my.cf settings).

In case the SQL script is useful to someone:

Code:
-- intervals in days

SET @history_interval = 7;
SET @trends_interval = 30;

DELETE FROM alerts WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);
DELETE FROM acknowledges WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);
DELETE FROM events WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);

DELETE FROM history WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);
DELETE FROM history_uint WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);
DELETE FROM history_str WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);
DELETE FROM history_text WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);
DELETE FROM history_log WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@history_interval * 24 * 60 * 60);

DELETE FROM trends WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@trends_interval * 24 * 60 * 60);
DELETE FROM trends_uint WHERE (UNIX_TIMESTAMP(NOW()) - clock) > (@trends_interval * 24 * 60 * 60);
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 7,409
Messages: 29,985

It doesn't need so much RAM.
Zabbix itself doesn't need it, that's correct. Most of the memory is assigned to MariaDB in order to improve its performance. I'm running Zabbix server, the frontend and the database on the same machine.
 

Phishfry

Son of Beastie

Reaction score: 1,304
Messages: 3,814

I am back on this again. net-mgmt/netdata does look nice. It is one of the only packages without a full DB.

When I went to install net-mgmt/cacti it needs all kin of stuff: php72, python, perl5, mysql, wayland.
I thought I had found a lightweight rrdtool program. It is not.
 

pyret

Member

Reaction score: 55
Messages: 86

netdata will install its own included web server on each installation. If you work for a Fortune 400 company as I do, that is a non-starter. I believe it is configurable to have a central server though.
 

Phishfry

Son of Beastie

Reaction score: 1,304
Messages: 3,814

I agree it is more of a distributive monitoring with no central node. Only drawing about 1% CPU and 190MB RAM.
I saw settings to use Prometheus as the backend in the netdata.conf.

Do any offer Bhyve (with no libvirt) monitoring? I setup netdata on all 4 of my bhyve VM's to test it out.
 

Phishfry

Son of Beastie

Reaction score: 1,304
Messages: 3,814

I found a major drawback with Netdata. I knew it used distributed monitoring but what I don't care for is the use of cookies.
It does not remember your netdata monitored computers. You must visit each one to repopulate the list.
This is because I clear cookies on every browser exit.

Found this is the excellent documentation:
How my browser is identified?

Your browser gets a random GUID the first time it contacts the registry. This acts as a PERSONID. This is you. Every time the browser talks to the registry it gives this id back to the registry.

  • Where is this PERSONID saved?
    a cookie (cookies are synced among your PCs, so cookies are good).
  • What happens if I lose this PERSONID?
    nothing, just access all your netdata dashboards once again.
    If you need a backup, the netdata dashboards could have a special menu item to give you the list.
Here is the dropdown with my VM's I am monitoring.
6101
 

quicktrick

Member


Messages: 21

Hello! How do you get Netdata working on FreeBSD? I've installed it via pkg, the process is working, but I cannot access its WebUI from another host. I tried to change its binding address in config, I disabled all the unnecessary plugins, I have 19999 port open... no success.

I have a few Linux servers with Netdata working in docker there, and I have no problem with monitoring them.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 7,409
Messages: 29,985

Never used Netdata but have you looked at its log file for clues?
 

Phishfry

Son of Beastie

Reaction score: 1,304
Messages: 3,814

You must add your IP to the netdata.conf in place of 127.0.0.1
 

quicktrick

Member


Messages: 21

I did that, though I added my IP as a second IP, like
Code:
bind to = 127.0.0.1 10.0.0.30
And I saw in the log that the web server was listening on the two IPs. And sockstat -l4 showed the same. And the port was opened in PF.

Thanks anyway, I'll try it again.

Edit:
I got it working somehow. But it works badly on my system (11.2 on ESXi 5.1). I was able to connect to WebUI, but after some short time WebUI stopped to work. Restarting Netdata didn't help. I removed the db and disabled all the plugins except "freebsd". I cleaned cache in the browser too. After starting Netdata I was able to connect to WebUI again and again for only some short time.

I've removed Netdata, it doesn't work on my system.
 
Top