ZFS Failed drive in encrypted ZROOT

[drobison]

Hiya,

I have an issue where a server running 11.3-RELEASE has an encrypted ZFS software mirror, set up through the installer. It's ada0/ada1 and ada1 has gone south so the mirror is degraded.

Can anyone point me to a fairly detailed how-to for replacing the encrypted, failed drive? I don't have access to this machine, and the person who will do this is trustworthy but not what I'd describe as a UNIX admin. I don't have a test system which I can easily use to replicate this situation.

I've found a few threads and blogs which talk about doing this, but not in much detail. The main issue which jumps out is that they all seem to mention the encryption.key file.

In the past, I've noticed that some of our servers have this file while some don't. I've searched and poked around svn.freebsd.org but have yet to find out whether some installations require this file while others don't, or if this was a change in the source tree at some point where this file was added or deprecated.

Any hints are appreciated.

 

[T-Daemon]

a server running 11.3-RELEASE

FreeBSD 11.x is EOL, not supported anymore.

I don't have a test system which I can easily use to replicate this situation.

You could practice in a VM, for example a VirtualBox VM. Create a exact hardware and system setup, remove one disk, add another disk, partition new disk gpart(8), geli(8) init and attach, zpool-replace(8). Best is repeat the exercise a few time, to settle the process. After practice you can instruct the person who has access to the machine better.

Assuming GPT partition scheme, bad disk removed, new disk installed, name ada1, BIOS+UEFI. Adapt if necessary to your system:

gpart create -s gpt ada1
gpart add -t efi -s 260m -l efiboot1 ada1
gpart add -t freebsd-boot -s 512k -l gptboot1 ada1
gpart add -t freebsd-swap -s 2g -l swap1 ada1
gpart add -t freebsd-zfs -a 4k -l zfs1 ada1

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1

newfs_msdos /dev/ada1p1
mount_msdosfs /dev/ada1p1 /mnt
mkdir -p /mnt/efi/boot
mkdir /mnt/efi/freebsd
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
cp /boot/loader.efi /mnt/efi/freebsd
umount /mnt

In this step the geli initialization passphrase of the old disks is needed.
geli init -bg -l 256 -s 4096 ada1p4
geli attach ada1p4

Encrypt swap, if swap is encrypted:
geli onetime ada1p3

geli status

zpool replace zroot ada1p4.eli ada1p4.eli
zpool status
(resilvering)

If you are in doubt about applying the guide, post the systems gpart show and geli status outputs.

 

[drobison]

FreeBSD 11.x is EOL, not supported anymore.


You could practice in a VM, for example a VirtualBox VM. Create a exact hardware and system setup, remove one disk, add another disk, partition new disk gpart(8), geli(8) init and attach, zpool-replace(8). Best is repeat the exercise a few time, to settle the process. After practice you can instruct the person who has access to the machine better.

Assuming GPT partition scheme, bad disk removed, new disk installed, name ada1, BIOS+UEFI. Adapt if necessary to your system:

gpart create -s gpt ada1
gpart add -t efi -s 260m -l efiboot1 ada1
gpart add -t freebsd-boot -s 512k -l gptboot1 ada1
gpart add -t freebsd-swap -s 2g -l swap1 ada1
gpart add -t freebsd-zfs -a 4k -l zfs1 ada1

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada1

newfs_msdos /dev/ada1p1
mount_msdosfs /dev/ada1p1 /mnt
mkdir -p /mnt/efi/boot
mkdir /mnt/efi/freebsd
cp /boot/loader.efi /mnt/efi/boot/bootx64.efi
cp /boot/loader.efi /mnt/efi/freebsd
umount /mnt

In this step the geli initialization passphrase of the old disks is needed.
geli init -bg -l 256 -s 4096 ada1p4
geli attach ada1p4

Encrypt swap, if swap is encrypted:
geli onetime ada1p3

geli status

zpool replace zroot ada1p4.eli ada1p4.eli
zpool status
(resilvering)

If you are in doubt about applying the guide, post the systems gpart show and geli status outputs.

Thanks, I will try this.

And I'm well aware that 11.3R is well out of support, but banks move at a glacial pace, and this is a server at a bank.

I'll check back when I know if this worked.