gptzfsboot fails to boot from ZFS pool on secondary disk

A

alex j

Hello, I installed 13.2-RELEASE on virtualbox with the following disk configuration:

ada0: 64M disk, GPT partitioned:
Code: 
# gpart show
=>    40  130992  ada0  GPT  (64M)
      40    1024     1  freebsd-boot  (512K)
    1064  129968        - free -  (63M)

ada1: 8GB disk assigned to ZFS:
Code: 
# zpool status
  pool: zroot
 state: ONLINE
config:

    NAME        STATE     READ WRITE CKSUM
    zroot       ONLINE       0     0     0
      ada1      ONLINE       0     0     0

# zfs list
NAME                 USED  AVAIL     REFER  MOUNTPOINT
zroot               1.51G  5.76G       96K  none
zroot/ROOT          1.51G  5.76G       96K  none
zroot/ROOT/current  1.51G  5.76G     1.51G  /

# zpool get bootfs zroot
NAME   PROPERTY  VALUE               SOURCE
zroot  bootfs    zroot/ROOT/current  local

I have installed the bootcode with gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0.

The system fails to boot automatically, with the following messages:
Code: 
BIOS drive C: is disk0
BIOS drive D: is disk1

Can't find boot/zfsloader

Can't find /boot/loader

Can't find /boot/kernel/kernel

FreeBSD/x86 boot
Default: disk-1:/boot/kernel/kernel
boot:

Can't find /boot/kernel/kernel

FreeBSD/x86 boot
Default: disk-1:/boot/kernel/kernel
boot:

If at the boot prompt I enter zfs:zroot/ROOT/current:/boot/zfsloader, the system boots normally. According to gptzfsboot(8), the bootcode should be able to find the pool in the second disk and boot from it. Am I missing some necessary configuration for this to work?

I have also tried creating a GPT on ada1 and creating the zroot pool on a 'freebsd-zfs' partition, same issue. Another, older system, upgraded to 13.2-RELEASE, where /boot is on a 'freebsd-zfs' partition on the same disk as the bootcode, boots without issue.
 
As it's virtual "hardware", may I suggest setting VM to UEFI boot, and skipping all that BIOS/DOS stuff (by selecting "GPT/UEFI" option in installer)?
 
yuripv79 said:
As it's virtual "hardware", may I suggest setting VM to UEFI boot, and skipping all that BIOS/DOS stuff (by selecting "GPT/UEFI" option in installer)?
Click to expand...
The VM is just a test machine which I created to reproduce this issue. I have the same issue on two production servers, where unfortunately I cannot risk making such changes.
 
Is ada1 also gpt partitioned? I realize you said the vm is trying to reproduce an issue on production hardware and the VM likely is mirroring that.
Sorry, I see that you've tried that already.

I think the directory "/boot" is actually part of the ZFS root dataset, it is not the "freebsd-boot" partition.
 
alex j said:
ada1: 8GB disk assigned to ZFS:
Click to expand...
gptzfsboot(8) won't boot from whole disks:
Rich (BB code): 
BOOTING
       gptzfsboot tries    to find    all ZFS    pools that are composed    of  BIOS-visi-
       ble  hard disks or partitions on    them.  gptzfsboot looks    for ZFS    device
       labels on all visible disks and in discovered supported partitions  for
       all  supported partition    scheme types.  The search starts with the disk
       from which gptzfsboot itself was    loaded.     Other    disks  are  probed  in
       BIOS  defined  order.  After a disk is probed and gptzfsboot determines
       that the    whole disk is not a ZFS    pool member, the individual partitions
       are probed in their partition table order.  Currently GPT and MBR  par-
       tition  schemes are supported.  With the    GPT scheme, only partitions of
       type freebsd-zfs    are probed.  The first pool  seen  during  probing  is
       used as a default boot pool.

Bug 220105 - boot1.efi does not detect ZFS pools on whole disks
Bug 235380 - gptzfsboot does not boot ZFS pool made from whole disks (regression)
Bug 235388 - gptzfsboot does not boot ZFS pool made from whole disks (regression), part two

alex j said:
If at the boot prompt I enter zfs:zroot/ROOT/current:/boot/zfsloader
Click to expand...

Try with GPT freebsd-zfs type and /boot/loader.conf
Code: 
vfs.root.mountfrom="zfs:zroot/ROOT/current"
 
T-Daemon said:
On second thought, if "If at the boot prompt I enter zfs:zroot/ROOT/current:/boot/zfsloader" try whole disk with the /boot/loader.conf setting.

The bug reports might be outdated.
Click to expand...

covacat said:
would a /boot.config work ?
boot.config(5)
Click to expand...

I tried both of these suggestions, nothing changed. To be honest I didn't expect these to have any effect, since it looks like the ZFS pool is not even detected. What I tried next was to get and install 'gptzfsboot' from 11.4-RELEASE and voila, it boots, both when using full disk and GPT for the root pool. So this looks indeed like a regression.

From the bug reports that T-Daemon mentioned it is suggested that some code changes were made that break booting from raw disk ZFS, but in my case ZFS on GPT partition also fails when it is not on the same disk as the 'freebsd-boot' partition. Can anyone else reproduce this?

Also, I wonder, could using the 'gptzfsboot' from an older version have any side-effects given that the system boots?
 
alex j said:
could using the 'gptzfsboot' from an older version have any side-effects
Click to expand...
Older versions of gptzfsboot(8) might not be able to recognize the newer version of ZFS and then fail to boot. The gptzfsboot(8) from 11.x or 12.x won't be able to recognize the OpenZFS that's on 13 and onward. So booting will break if you upgrade the pool itself.
 
alex j said:
But the bootcode from 11.4 does boot the 13.2 pool.
Click to expand...

SirDice said:
It won't anymore if you upgrade the pool.
Click to expand...
zpool upgrade should be used cautiously. If you are not using any special features/characteristics of a ZFS version it should be transparent, but from 12.x to 13.x (13 switched to OpenZFS) one has to be careful.
Aside from the regression in the 13.x bootcode, 13.x should understand older versions of ZFS (as evidenced by your system booting when you specified zroot), but older bootcode likely will have problems with newer versions of ZFS.
 
Indeed. The boot code from 13 for example has no problems with ZFS from previous versions, it is backwards compatible. But the older boot code might not be able to access the newer versions of ZFS, the boot code is not forward-compatible.
 
  • Like
Reactions: mer
You must log in or register to reply here.
Back
Top