bhyve Best way to monitor VMs

patpro

patpro

Hello,

I'm using munin to monitor 2 FreeBSD bare metal servers. Since I've started to use BHyve on one of them, I'm wondering what would be the best way to monitor the VMs. I'm currently using munin-node on 3 of them but may be there is something more efficient than just install munin-node on every VM.
Any thoughts on that topic?
 
Hello,
Prometheus seems really complicated for just a handful of hosts (I could as well use my already installed Splunk instance)
 
patpro said:
I'm wondering what would be the best way to monitor the VMs.
Click to expand...
There are basically two schools of thought. One is to treat a VM like any other physical server, installing the monitoring on the machine itself. The other is to monitor from the host and write some code that monitors the VMs from that point of view. The easiest to deal with is just to treat a VM the same way you would treat a physical machine.
 
Setting up a monitor using timescaledb/netdata/prometheus/grafana is straithforward. I have statistics each 20 seconden.
I think rrdtool defaults to statistics each 15 minutes. And rrdtool need some configuration, setup of a webserver to create regulary graphs.
 
Thank you all for your replies. As nothing specific to BHyve seems to exist, I'm going to stick with munin.
 
I have been researching munin contenders and I think nagios, cacti and zabbix offer similar features.

Zabbix requires mysql and that to me is too much. rrdtools database is nice and lean.

I tend to judge software by the amount and size of dependencies.
I am installing a collector on my Internet facing router and want minimal overhead.

What does disappoint me is the size of munin-node. It is large and needs many Perl packages (Python too?).
Code: 
pkg install munin-node
Number of packages to be installed: 95
The process will require 241 MiB more space.
That is not lightweight for a collector.

munin-master is the server and lighttpd is the web server.
Code: 
pkg install munin-master lighttpd
Number of packages to be installed: 98
The process will require 334 MiB more space.


While Nagios server component weighs in at 126 MiB I believe it has an embedded web server.
Code: 
pkg install nagios
Number of packages to be installed: 18
The process will require 126 MiB more space.
This includes mysql too. I really don't like mySQL or mariaDB. I want lean.
Nagios has various collectors. This part was hard to judge as they have multiple collectors.


Cacti looks nice and it uses rrdtools. Too big for me.
Code: 
pkg install cacti
Number of packages to be installed: 82
The process will require 545 MiB more space.

The collector:
Code: 
pkg install cacti88-spine
Number of packages to be installed: 19
The process will require 191 MiB more space.
 
Alain De Vos said:
It makes also a difference if you want to collect a statistic every 15minutes or every 15seconds.
Click to expand...
Difference regarding database choice or overall storage size needed for 15 sec monitoring?
I am happy with 15min.

I can accept the fact that munin-node needs perl scripts to work but why python on the collector.
Perl and Python do the same thing right? That part irks me.
 
So to expand on this thread about monitoring software.
Nagios is lighter in size. But you must download the plugins wherein Munin ships with a base set.

But after much prodding by Alain De Vos I decided to see how large the buzzy sounding "Prometheus/Grafana/node exporter" combination weighed in.
Code: 
pkg install prometheus node_exporter grafana

New packages to be INSTALLED:
    grafana: 1.9.1
    node_exporter: 1.2.2
    prometheus: 2.30.0

Number of packages to be installed: 3

The process will require 158 MiB more space.
31 MiB to be downloaded.
That seems light enough. No dependencies at all.
Prometheus includes a web server. Good and easy.

What about client collectors?
 
I'd like to chime in here: I am also a big fan of Zabbix. Sure it might take some time to get familiar with it but that's the case with most "serious software".
I wouldn't want to miss it. It works extremely well.
 
Zabbix needs mySQL and I don't want that or mariaDB.

I like the approach of Nagios which seems to use an internal DAT file for database.
Still have not tried it or Icinga yet.

I have Prometheus/Grafana/Node_Exporter setup but I have a bias against it.
It seems like the cockpit of a jet fighter and not a long term monitoring tool.
The combination is very configurable but it is not what I want.
The fact that grafana8 won't work on SeaMonkey was the dagger.
I originally installed Grafana 1.9 but it did not include an rc.d startup script.
That was when I discovered the multitude of versions of Grafana.
What a trainwreck.

I really enjoy rrdtools and a webserver I can manage. lighttpd.
The search continues....
 
Last edited:
I am probably going to stick with Munin.

Cacti uses rrdtools but also needs an SQL database.
I guess they are only using rrdtools for the graphing.

Here is what Munin offers that is pretty unmatched.
The setup script you run on munin nodes really is nice.
For that alone the bulk of perl scripts is worth it. From network connections to firewall it has you covered.
The fact that it detects smartmontools and uses it is the best feature.
I can put up with its primordial look.
 
Alain De Vos said:
Setting up a monitor using timescaledb/netdata/prometheus/grafana is straithforward. I have statistics each 20 seconden.
I think rrdtool defaults to statistics each 15 minutes. And rrdtool need some configuration, setup of a webserver to create regulary graphs.
Click to expand...
Well I think this shows the divide. You want 20 sec. polling for stats while I am happy with 15 min.

I use Munin as a forensic tool to investigate patterns after the fact.
With 20 seconds you are using Prometheus for near live stats. More of an IDS.

I like a separate web server as well. Having Grafana and Prometheus web servers was weird
(punching internal holes for 9090, 9100 and 3000).
The disk usage was impressive. It is not large using Node_Exporter for collector.
 
SirDice said:
I'm a big Zabbix fan. So I use that. Scales really well too. Both require some effort to set up though.
Click to expand...
I plan to create a Zabbix template. I was looking at a new feature in pull reqeuest https://github.com/churchers/vm-bhyve/pull/402 of vm-bhyve project, which is JSON format for vm-list and then it will be easy to parse the output and it will be possible to create LLDs for guests, switch, storage.

Example:
Code: 
# ./vm switch list -j
{
   "switches": [
      {
         "name": "public",
         "type": "standard",
         "iface": "vm-public",
         "address": "-",
         "private": "no",
         "mtu": "-",
         "vlan": "-",
         "ports": "em0"
      }
   ]
}

# ./vm datastore list -j
{
   "datastores": [
      {
         "name": "default",
         "type": "zfs",
         "path": "/vms",
         "dataset": "zroot/vms"
      }
   ]
}

# ./vm list -j
{
   "machines": [
      {
         "name": "debian12",
         "datastore": "default",
         "loader": "grub",
         "cpu": "1",
         "memory": "512M",
         "vnc": "-",
         "auto": "No",
         "state": "Stopped"
      },{
         "name": "zbx3-debian",
         "datastore": "default",
         "loader": "grub",
         "cpu": "1",
         "memory": "512M",
         "vnc": "-",
         "auto": "No",
         "state": "Running (8912)"
      }
   ]
}
 
patpro said:
Hello,

I'm using munin to monitor 2 FreeBSD bare metal servers. Since I've started to use BHyve on one of them, I'm wondering what would be the best way to monitor the VMs. I'm currently using munin-node on 3 of them but may be there is something more efficient than just install munin-node on every VM.
Any thoughts on that topic?
Click to expand...
Consider a combination of host-level monitoring using munin-node on the host server for system-level metrics and implementing a centralized solution like Prometheus with exporters for VM-specific metrics. This approach balances efficiency and granularity in monitoring FreeBSD VMs running on BHyve.
 
SmEjDiL said:
I plan to create a Zabbix template. I was looking at a new feature in pull reqeuest https://github.com/churchers/vm-bhyve/pull/402 of vm-bhyve project, which is JSON format for vm-list and then it will be easy to parse the output and it will be possible to create LLDs for guests, switch, storage.

Example:
Code: 
# ./vm switch list -j
{
   "switches": [
      {
         "name": "public",
         "type": "standard",
         "iface": "vm-public",
         "address": "-",
         "private": "no",
         "mtu": "-",
         "vlan": "-",
         "ports": "em0"
      }
   ]
}

# ./vm datastore list -j
{
   "datastores": [
      {
         "name": "default",
         "type": "zfs",
         "path": "/vms",
         "dataset": "zroot/vms"
      }
   ]
}

# ./vm list -j
{
   "machines": [
      {
         "name": "debian12",
         "datastore": "default",
         "loader": "grub",
         "cpu": "1",
         "memory": "512M",
         "vnc": "-",
         "auto": "No",
         "state": "Stopped"
      },{
         "name": "zbx3-debian",
         "datastore": "default",
         "loader": "grub",
         "cpu": "1",
         "memory": "512M",
         "vnc": "-",
         "auto": "No",
         "state": "Running (8912)"
      }
   ]
}
Click to expand...
Here is a development template for Zabbix 7.0 usable with vm-bhyve with JSON support from https://github.com/runhyve/vm-bhyve/tree/json-lists on pull request pending
github.com

zabbix-ansible/templates/FreeBSD_bhyve_by_Zabbix_agent-7.0.yaml at main · smejdil/zabbix-ansible

First steps with Ansible and Zabbix. Contribute to smejdil/zabbix-ansible development by creating an account on GitHub.
github.com github.com
 

Attachments

  • FreeBSD_bhyve_by_Zabbix_agent-7.0.xml
    42.4 KB · Views: 39
You must log in or register to reply here.
Back
Top