ZFS Health and Status Monitoring

Oko

Daemon

Reaction score: 773
Messages: 1,620

What is recommended way to monitor ZFS Health and Status? I am shamelessly using knowledge transfer from FreeNAS to FreeBSD to configure S.M.A.R.T daemon.
However I don't see ZFSd on 10.0.3 TrueOS. However running

Code:
zpool status -x
seems rather impractical. Is there a collectd plug in for zpool status -x or something which reports to SNMP daemon either built in or net-snmp?
 
Last edited:
OP
OP
Oko

Oko

Daemon

Reaction score: 773
Messages: 1,620

SirDice said:
Err.. So what's bsnmpd(1) doing then?
Thanks for bringing bsnmpd to my attention. That brain fart with "FreeBSD has no SNMP native daemon" is the second stupidest thing I wrote on this forum :r . Could you please address the ZFS monitoring part of my question?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 8,075
Messages: 31,686

Oko said:
Could you please address the ZFS monitoring part of my question?
I would if I could ;)
 
OP
OP
Oko

Oko

Daemon

Reaction score: 773
Messages: 1,620

getopt said:
Have a look on /etc/periodic/daily/404.status-zfs
I already saw it! Reading TrueOS and FreeNAS configuration files carefully. These people know what they are doing.
 

bhughes

Member

Reaction score: 8
Messages: 23

Have a look at devd(8) and /etc/devd.conf (or possibly /etc/devd/zfs.conf), which shows how you can react to various ZFS events. I have yet to find any comprehensive documentation about all available ZFS events, but the examples in the devd.conf files are a good start.
 

ralphbsz

Daemon

Reaction score: 1,275
Messages: 2,045

I run zpool scrub ... once a night, and capture the output in a logfile. If the output looks unusual, or if the error code from the command is non-zero, and e-mail is sent to the admin.

And my monitoring tool (100 lines of pedestrian, boring python) runs zpool status, looks for all the pools I expect, and makes sure they are shown as "online" with no extra messages. If yes, it gives ZFS a green light; otherwise it gets a yellow light, and the knowledgeable user has to look in the log file.

All very simply, yet surprisingly sturdy. No SNMP integration, just a bunch of boring cron jobs and short scripts.
 

nakal

Active Member

Reaction score: 37
Messages: 245

You should rather watch if there is something that doesn't equal "0" in the tabular output of zpool status (but drive status is also important of course!). This is usually an early warning about problems. I run scrub about twice a month, it is not needed to do it every day, in my opinion. This zpool table is also updated during normal accesses. When you have checksums on, ZFS will notice errors at least as early as S.M.A.R.T. does. ZFS also won't lie to you, S.M.A.R.T. does sometimes, and sometimes it is just broken.
 

Raviraj

New Member


Messages: 2

What is recommended way to monitor ZFS Health and Status? I am shamelessly using knowledge transfer from FreeNAS to FreeBSD to configure S.M.A.R.T daemon.
However I don't see ZFSd on 10.0.3 TrueOS. However running

Code:
zpool status -x
seems rather impractical. Is there a collectd plug in for zpool status -x or something which reports to SNMP daemon even though unlike OpenBSD FreeBSD doesn't have a built in SNMP?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 8,075
Messages: 31,686

If anybody is interested, I wrote a couple of scripts and templates for Zabbix to monitor ZFS. Pools and filesystems are detected using Zabbix' LLD. It's nowhere near finished but it does keep track of availability (status of RAID for example) and capacity. I'm planning to add some of the zpool iostat parameters but I haven't gotten around to it yet.
 

Sylhouette

Active Member

Reaction score: 29
Messages: 200

I would like these scripts and templates for ZFS and Zabbix. I am sure more people want them!
 

virgil

New Member

Reaction score: 5
Messages: 6

If anybody is interested, I wrote a couple of scripts and templates for Zabbix to monitor ZFS. Pools and filesystems are detected using Zabbix' LLD. It's nowhere near finished but it does keep track of availability (status of RAID for example) and capacity. I'm planning to add some of the zpool iostat parameters but I haven't gotten around to it yet.
Is it still possible to get a look at your Zabbix-ZFS scripts & templates?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 8,075
Messages: 31,686

This isn't what I was working on, I used it as a basis. But it should get you going:
Save this as /usr/local/etc/zabbix22/zabbix_agentd.conf.d/userparam_zfs.sh
Code:
UserParameter=vfs.zpool.discovery,/usr/local/bin/sudo /sbin/zpool list -H -o name | perl -e 'while(<>){chomp;push(@P,qq(\t\t{"{#POOLNAME}":).qq("$_"}));};print qq({\n\t"data":[\n);print join(",\n",@P)."\n";print qq(\t]\n}\n);'
UserParameter=vfs.zfs.discovery,/usr/local/bin/sudo /sbin/zfs list -H -o name | perl -e 'while(<>){chomp;push(@P,qq(\t\t{"{#FSNAME}":).qq("$_"}));};print qq({\n\t"data":[\n);print join(",\n",@P)."\n";print qq(\t]\n}\n);'
UserParameter=vfs.zfs.get[*],/usr/local/bin/sudo /sbin/zfs get -o value -Hp $2 $1 | sed -e 's/x//'
UserParameter=vfs.zpool.get[*],/usr/local/bin/sudo /sbin/zpool get -Hp -o value $2 $1 | sed -e 's/x//'
Save this as /usr/local/etc/sudoers.d/zabbix:
Code:
Defaults:zabbix !requiretty
zabbix  ALL=(root)  NOPASSWD:   /sbin/zpool
zabbix  ALL=(root)  NOPASSWD:   /sbin/zfs
And import the attached XML template and assign the "Template App ZFS" to a host with ZFS.
(Rename the file to .xml. The forum wouldn't let me save it as XML)
 

Attachments

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 8,075
Messages: 31,686

Short explanation, the template only has discovery rules. It will dynamically find all pools and filesystems. It will then, for each pool or filesystem, fetch some of the interesting values like compression ratio, dedup ratio, usage, etc. It will also, for each pool and filesystem, create graphs for usage and compression ratio. There's also a trigger that will alert if a pool is in a degraded state. There's no trigger for >80% full yet but it should be relatively easy to add one.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 8,075
Messages: 31,686

I'm not sure if the commands work if you run them on the zabbix user account. It's possible it doesn't have enough access privileges to get the information.
 

Eric A. Borisch

Well-Known Member

Reaction score: 255
Messages: 441

You shouldn't need to elevate permissions for the zfs/zpool get/list commands. I run extremely similar commands from the [FONT=Courier New]zabbix[/FONT] user directly. (I think I used awk instead of perl to make the discovery response.)
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 8,075
Messages: 31,686

Yeah, already confirmed that with the new scripts/templates. It works just fine without sudo(8).
 
Top