ZFS mirror: Dying boot drive - what is the right way to act?

[Snurg]

My test PC has been complaining the last twenty minutes with messages like this on the console:

(da0:...)  ... SYNCHRONIZE CACHE ... 
... CAM STATUS: SCSI Status Error ...
... HARDWARE FAILURE ... Peripheral device write fault ...

It's a 2-drive mirrored ZFS installation on da0 and da1.
Can I just remove da0, boot from da1 and use the zfs pool with one drive until a replacement drive has arrived?

 

[usdmatt]

You can probably just offline the failed disk so ZFS won't use it, then physically swap the drive when you get the replacement.

Now:

zpool offline pool da0

When disk has been swapped. (Assuming new disk gets da0, which it should do):

zpool replace pool da0

 

[kpa]

With mirrors you can even detach the faulty disk and continue with a single disk pool until you get the replacement:

zpool detach pool da0

Then later when you have the replacement disk:

zpool attach pool daX

I wrote daX because the new disk might become da1 with the other working disk being da0. Take good note which one is which.

 

[Snurg]

zpool attach pool daX

I just read the zpool man page attach section.
You can later decide to add even more mirror disks.
This is really cool.

BTW, I was lazy and just copied the beginning of the still-working disk to the new mirror disk (exactly same size, already zeroed) using dd and after that just zeroed the swap and zfs partitions.
gpart show then told me that the partition table is corrupt. But gpart restore fixed that. I guess I did not a clean job

 

[usdmatt]

The GPT table is stored at the beginning and end of the disk, so it was probably complaining about the end GPT table being corrupt as you only copied the beginning of the disk.
I would of just partitioned the disk using gpart and not bothered zeroing it myself, but your method seems to have worked sufficiently...

 

[Snurg]

The GPT table is stored at the beginning and end of the disk...

Thank you, that explains it very well

I would of just partitioned the disk using gpart and not bothered zeroing it myself, but your method seems to have worked sufficiently...

Actually the problem for me was the question how to correctly set up also the grub thing and the /boot partition.
These were set up by the FreeBSD installer and I don't know the parameters used.
So I just copied them with dd.
Will test later whether the replacement drive will boot fine.

P.S.: This is just one thing I love about FreeBSD...
You probably know that Linux has also a software mirror disk system (md).
The gross thing there is that only the data partitions are redundant.
But the boot (grub) and root things are being set up only on the first drive!
If the boot drive fails then a Linux mirrored system won't let you boot!