ZFS RAID notifications

6502

6502

Is there tool or zpool option for sending email if some disk in RAID array fails? I know it can be done with cron and shell scripts but I guess ZFS infrastructure may include ready for use option.
 
I don't think there's anything out-of-the-box in FreeBSD base system (yet) - there's zfsd(8), but the man page says it simply logs the messages about events to syslog (as compared to e.g. Fault Manager in illumos where it has snmp/smtp notifications).
 
/etc/periodic/daily/404.status-zfs
/etc/rc.conf
daily_status_zfs_enable="yes"
Click to expand...

All notifications are received via the daily run output email. The problem is that you will need to monitor those e-mails or set some logic/triger in your e-mail client to catch and rise alarm/task when the content of the e-mail is matching some fail.

For example the status of the graid received in periodic output mail look like this:

Code: 
Checking status of graid(8) devices:
   Name   Status  Components
raid/r0  OPTIMAL  ada1 (ACTIVE (ACTIVE))
                  ada0 (ACTIVE (ACTIVE))

So the e-mail client need to trigger when the status is changed from OPTIMAL to DEGRADED otherwise during the year you will stop read those e-mails and you will miss the moment when a fail occurs.

Code: 
Checking status of graid(8) devices:
   Name    Status  Components
raid/r0  DEGRADED  ada0 (ACTIVE (ACTIVE))

I personally have a mail rule to move all those e-mails in sub folders in my mailbox and organize them for each server and another rule that is looking for keywords in the e-mail to rise a task.
 
If your mail delivery agent obeys $HOME/.forward directives (most do) you can use procmail(1). See also procmailex(5).

Edit: this is what I put in my $HOME/.forward file about 25 years ago: "|IFS=' '&&p=/usr/local/bin/procmail&&test -f $p&&exec $p -Yf-||exit 75"
 
6502 said:
I found that there is similar daemon on Linux - zed:
medium.com

Configuring ZFS Event Daemon (ZED) on Ubuntu 16.04

Summary
medium.com medium.com
Click to expand...
These go to devd on FreeBSD. See zpool-events(8).

When I'm doing
# zpool offline build ada1p9.elip2
devd reports this (and a lot more):
Processing event '!system=ZFS subsystem=ZFS type=resource.fs.zfs.statechange version=0 class=resource.fs.zfs.statechange pool=build [etc.etc.etc.]

So, this is not really a "ready-for-use" option, it is rather an event infrastructure for almost any requirement. Sending e-mail from devd should be simple. Filtering for the appropriate interesting events might be a bit more difficult.
 
yuripv79 said:
I don't think there's anything out-of-the-box in FreeBSD base system (yet) - there's zfsd(8), but the man page says it simply logs the messages about events to syslog (as compared to e.g. Fault Manager in illumos where it has snmp/smtp notifications).
Click to expand...
That's cool, I didn't know it existed. A few years ago I started a prototype "ZFS watcher", which was a small script that ran once an hour and sent an e-mail whenever the zpool status was "not perfect". It worked for a while, but it had hard-coded into it what the perfect zpool status should be (zpool A shall have two disks, and their names shall be A_1 and A_2, zpool B shall have one disk ...). I stopped using it when the configuration changed, because I was too lazy.

In my case, zfsd would just work, since I don't have hot spares. So any message from it would indicate that something has gone wrong; I could more easily write an hourly script that looks in syslog for any messages from zfsd, and sends them by e-mail (probably with a copy of "zpool status" output).
 
You must log in or register to reply here.
Back
Top