Boot problem

mefizto · Apr 3, 2021

Greetings all,

I have a UEFI partition with /boot/loader.efi installed on /dev/da0. The machine finds the "FreeBSD/amd64 EFI loader. Revision 1.1" and sets a "Load Path: \EFI\BOOT\BOOTX64.EFI."

According to gptzfsboot(8):

2.loader.efi searches partitions of type freebsd-ufs and
freebsd-zfs for loader.efi. The search begins with partitions
on the device from which loader.efi was loaded, and continues
with other available partitions. If both freebsd-ufs and
freebsd-zfs partitions exist on the same device the
freebsd-zfs partition is preferred.

Despite /dev/nvd0 containing /freebsd-zfs:

Code:

# gpart show nvd0
=>    40    4883970088    nvd0    GPT    (233G)
    40    41169608            -free-    (201M)
41169608    8388608    2        freebsd-swap    (4.0G)
8388608    47959654    3        freebsf-zfs        (229G)

the machine reports: "No bootable path found" and reboots.

Is a different /dev not considered a "partition"? Any ides?

Kindest regards,

M

zirias@ · Apr 3, 2021

I can't make any sense of that. gptzfsboot is for legacy boot, not for UEFI. Then you write you have it on da0, and THEN, you show a partition table on nvd0, which has neither a freebsd-boot nor an efi partition. Sorry, I'm confused here… please clarify.

mefizto · Apr 3, 2021

Hi Zirias,

you are correct. I have been trying to make it work with both UEFI and BIOS, so I did confused the bootloaders with the one I am trying to use now. I corrected the original post.

The /dev/da0 contains the UEFI bootloader, that is the /boot/loader.efi written into /EFI/BOOT/BOOTX64.EFI.

The /dev/nvd0 contains the OS without the bootloader. I showed the GPT partition table of the /dev/nvd0 as a proof that the freebsd-zfs that the loader.efi is alleged to find does indeed exist.

Does it make sense now?

Kindest regards,

M

tingo · Apr 3, 2021

Indeed, the man page specifies "gptzfsboot tries to find all ZFS pools that are composed of BIOS-visible hard disks or partitions on them.", so it should find your zfs if it operates as described.
You should try to interrupt the loader, and check out manually if you can see the freebsd-zfs partition from the loader. Also see if the vfs.zfs.boot.primary_vdev and vfs.zfs.boot.primary_pool are set correctly. It also supports a status command, which might be helpful.

_martin · Apr 3, 2021

mefizto said:
you are correct. I have been trying to make it work with both UEFI and BIOS

Side note: you can use one disk to achieve this, e.g.

Code:

node01:(~)# gpart show ada0
=>       40  976773088  ada0  GPT  (466G)
         40     408832     1  efi  (200M)
     408872       1024     2  freebsd-boot  (512K)
     409896  968884224     3  freebsd-zfs  (462G)
  969294120    7479008     4  freebsd-swap  (3.6G)
node01:(~)#

And I'm able to boot either way. For UEFI boot my ada0p1 is used, for legacy boot ada0p2 is used (which implies that bootloader code during OS update has to be gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 2 ada0 )

I'm mentioning this as you have enough free space to do achieve this on your nvd0.

tingo advice on the boot param checks are worth checking out. I'd add to check any zfs_*, bootenv*.

mefizto · Apr 3, 2021

Hi tingo,

You should try to interrupt the loader, and check out manually if you can see the freebsd-zfs partition from the loader. Also see if the vfs.zfs.boot.primary_vdev and vfs.zfs.boot.primary_pool are set correctly.

Thank you for the suggestion. Could you please advise hot to do this? I am slowly working through understanding how the booting works, but the information is spread in different man-pages, so it is not an easy process to figure out, much less to diagnose.

Hi _martin,

yes, the structure that you offered of course will do boot. My problem is that the motherboard will not boot from the /dev/nvd0 on which the OS is installed, so I am trying to bootstrap it from the /dev/da0.

Kindest regards,

M

_martin · Apr 3, 2021

If your goal is just to boot the machine and you don't care if it's the UEFI or legacy way, with the assumption you put the proper bootcode on the da0 boot partition, you could boot it in legacy this way:
As soon as da0 starts to boot hit <esc>. You'll abort the boot process and will be presented with something like this:

Code:

BIOS drive .. 
-
FreeBSD/x86 boot
Default: zfs:rpool/ROOT/default:/boot/zfsloader
boot:

Here you are able to specify the way it should boot from. Should your ZFS pool on nvd0 be named newpool you could boot with: zfs:newpool/ROOT/default:/boot/zfsloader.
In this example my / is in zfs fileset newpool/ROOT/default.

There's probably some way to do this from UEFI too. But I'm new to this way of booting (ia64/HP-UX boot doesn't count

) so I'm not sure yet.

mefizto · Apr 3, 2021

Hi Martin,

thank you very much for trying to help, but I think that the mystery is solved.

I have done some additional reading and studying the UEFI shell output, and what I discovered is that the UEFI shell does not detect the NVMe drive. I was rather surprised since the bdsinstaller had no problem finding the NVMe drive and install the OS, so I did some further research and found the following:

grawlinson

You might be able to pull off putting /boot on an USB, / on the NVMe once initramfs has loaded. I had to do a similar thing with a M2 SSD.

Thus, as best as I understand, the motherboard does not detect the /dev/nvd0 until later in the boot process. This seems consistent with what Phisfry suggested on another thread.
I do not believe that the bdsinsaller can do this, so I will have to read more and try to put together a script to do so.

Kindest regards,

M