Solved Server won't boot after replacing zroot disks

mariourk

mariourk

Hi,

I just replaced both disks of a zroot mirror (da2 and da10). As far as I can tell I did everything right. But evidently something went wrong, since the server won't boot. All I get is a black screen, with a blinking cursor. On the bright side, this is an old decommissioned server, I'm just using to mess around and try out some new stuff. Mainly getting the hang of bhyve. I also still have the original disks. So, worst case scenario, nothing bad happens anyway. But I still like to fix this, in case I come to face this scenario in a more dire and serious situation.

Here is what I did:
First, I added a new disk in the empty HDD-slot. FreeBSD (11.2) named it da11. Then I ran the following commands:
Code: 
gpart show da2 (get the layout of the original disk)
gpart destroy -F da11
gpart create -s GPT da11
gpart add -b 40 -s 200M -t efi da11
gpart add -b 409640 -s 512k -t freebsd-boot da11
gpart add -b 410664 -s 2G -t freebsd-swap da11
gpart add -t freebsd-zfs da11
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da11
dd if=/dev/da2p1 of=/dev/da11p1
gpart show da11 (all seems fine)
zpool attach zroot /dev/da2p4 /dev/da11p4
zpool detach zroot /dev/da2p4

I then removed da2 and added the second HDD to this slot.FreeBSD named it da2. Then I ran the following commands:
Code: 
gpart show da10 (get the layout of the original disk)
gpart destroy -F /dev/da2
gpart create -s GPT da2
gpart add -b 40 -s 200M -t efi da2
gpart add -b 409640 -s 512k -t freebsd-boot da2
gpart add -b 410664 -s 2G -t freebsd-swap da2
gpart add -t freebsd-zfs da2
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da2
dd if=/dev/da10p1 of=/dev/da2p1
gpart show da2 (all seems fine)
zpool attach zroot /dev/da10p4 /dev/da2p4
zpool detach zroot /dev/da10p4

zpool status -v showed that everything was fine, so I decided to reboot. And this resulted in an unbootable server. So, my question is, is this fixable? I so, how? And, what went wrong?

Kind regards,
Marinus ten Napel
 
mariourk said:
And this resulted in an unbootable server.
Click to expand...
Define unbootable? Does it boot at all (or at least try to)? Is is stuck on the loader(8) prompt? Stuck in single user mode due to errors in /etc/fstab?

Scratch that, I already see where you went wrong. You created an efi boot partition and proceeded to write the gptzfsboot(8) code to it. That's not going to work, you're mixing up different boot types. For a traditional BIOS boot you need to write the gptzfsboot(8) code to a freebsd-boot partition. For an UEFI boot you create the efi partition and dd(1) the contents of /boot/boot1.efifat to it.

You can create a disk layout that works for both BIOS and UEFI (like you did) but you must write the correct code to the correct partition.
 
Aha! The -i 1 daX part, should have been -i 2 daX, right?

Well, at least now I know what went wrong.

Followup question. I simply used dd to write the original efi partition to the new efi partition. Is this the right way to do it? Or at least a valid way? If not, how should I do it properly?
 
mariourk said:
The -i 1 daX part, should have been -i 2 daX, right?
Click to expand...
In your case, yes.

mariourk said:
I simply used dd to write the original efi partition to the new efi partition. Is this the right way to do it?
Click to expand...
I don't know what the original content of that source disk was. So to be safe use /boot/boot1.efifat as a source. The boot1.efifat is basically a disk image which contains everything so you can simply dd(1) it to the right (efi) partition.

Easily fixed by booting the install media and using the shell to write the correct boot code in the right place.
 
Check how is your computer set too boot. If it setup to boot with UEFI without compatibility support module(CSM/ legacy / BIOS) then you don't need freebsd-boot partition at all. Here's several examples of different layouts

GPT - UEFI layout
-----
efi (200MB) - contains /boot/boot1.efifat created with dd if=/boot/boot1.efifat of=/dev/da#p1 freebsd-swap
freebsd-zfs or freebsd-ufs

GPT - BIOS layout
-----
freebsd-boot (512K) - smaller than 545kB (contains bootstrap code like /boot/gptboot or /boot/gptzfsboot
freebsd-swap
freebsd-zfs or freebsd-ufs

GPT - UEFI+BIOS layout (support both types)
----
efi (200MB) - contains /boot/boot1.efifat created with dd if=/boot/boot1.efifat of=/dev/da#p1 freebsd-boot (512K) - contains bootstrap code like /boot/gptboot or /boot/gptzfsboot
freebsd-swap
freebsd-zfs or freebsd-ufs

Edit:
if you create your partition manually make sure that they are properly alignment of 4k.
 
Last edited:
I booted the server with a livecd and used dd to copy the freebsd-boot partition from one of the original HDD's, to the two new HDD's. After this the server booted without errors and everything seems to work fine.

You guys have been most helpfull :)
 
You must log in or register to reply here.
Back
Top