Solved zpool upgrade danger, data no longer accessible.

Using openzfs port i did a zpool upgrade which installed two features:
-com.delphix:head_errlog
-org.openzfs:zilsaxattr
My system did no longer boot.
I downloaded FreeBSD-13.1 usb image. But,
I cannot import my zpool because of unsupported version or feature. (two listed above).
This is a rather painfull experience.
So zfs from base & zfs from openzfs port are out of sync.
What do i do now.
As currently i lost all my data until i can import the pool with the newer features....

The same problem exists when i try to import the pool using linux.
Is there an image to download which uses a newer kernel/openzfs ? Because Freebsd-13.1-releng is a no-go.
 

zirias@

Developer
These features are available on -CURRENT, so you *could* try to install bootcode from -CURRENT if you really need to boot from that pool.

Why are you enabling features (and, using openzfs port at all) you most likely won't need?
 
Ouch, that's why one needs to be careful with zpool upgrade and using openzfs port on a system that you can't afford to lose.

I would check if the version in openzfs port is the same as in FreeBSD-CURRENT; if so make a bootable image from that and see if you can import the pool. If so and you have another system you can zfs send/receive the datasets to pull it off.

I see I was too slow and SirDice and zirias@ beat me too it.
 

zirias@

Developer
BTW:
So zfs from base & zfs from openzfs port are out of sync.

That's the whole point of it! The openzfs port enables you to use the most recent openzfs version on any version of FreeBSD. It doesn't make any sense to do so unless you really need a feature only available in this newest version.

IIRC, zpool upgrade will always warn you about bootcode. And of course, bootcode of 13.1-RELEASE only knows the zfs version of 13.1-RELEASE. So never upgrade your boot pool to a newer version...

Yes, I see two possibilities to recover. mer's suggestion is the conservative (arguably better) one, backup everything, recreate the pool in the correct version, then restore. Then stock bootcode will work again. Otherwise, bootcode from 14-CURRENT most likely will work with the newer pool format, but then you can never go back to stock bootcode.
 
  • Thanks
Reactions: jbo
Otherwise, bootcode from 14-CURRENT most likely will work with the newer pool format, but then you can never go back to stock bootcode.
Is there any potential issues when doing freebsd-update? I'm thinking keeping the system on 13.x, doing freebsd-update to the next pN release. I don't think the bootcode is touched, but don't know.
 
Is there a bootable "current" i can download.
Note as long as i cannot import the zpool i also cannot backup anything .
 

zirias@

Developer
Is there any potential issues when doing freebsd-update? I'm thinking keeping the system on 13.x, doing freebsd-update to the next pN release. I don't think the bootcode is touched, but don't know.
As far as I know .... :-/ bootcode is never automatically installed. You need to do it manually with gpart bootcode (for CSM) or by copying /boot/loader.efi to your ESP (for UEFI).
 
  • Like
Reactions: mer
I explain my setup.
I use linux grub to boot:
Code:
menuentry "FREEBSD_DIRECTBOOT" {
insmod part_gpt
insmod part_msdos
insmod ufs2
insmod bsd
set root=(hd2,gpt9)
kfreebsd /boot/loader
}
This towards a ufs partition containing the loader.
In loader.conf I configured to boot the kernel on the ZPOOL which i "upgraded".

My loader.conf is :
Code:
opensolaris_load="YES"
openzfs_load="YES"
vfs.root.mountfrom="zfs:ZT/ROOT/default"
currdev="zfs:ZT/ROOT/default:"
 

zirias@

Developer
This ... makes me wonder ... another "why" :-/

But anyways, if you want to keep your upgraded pool, you could just try to place the /boot/loader from 14-CURRENT there.
 
This ... makes me wonder ... another "why" :-/

But anyways, if you want to keep your upgraded pool, you could just try to place the /boot/loader from 14-CURRENT there.

Actually if Alain De Vos is dual booting linux on this system, I wonder if it would be possible to:
boot into linux
temp mount rw the ufs partition that holds /boot
pull the loader from a CURRENT ISO and copy it over.

Download a CURRENT ISO, mount it on Linux like a FreeBSD md mount, and grab the needed files that way.
 
It's fixed.
Here is what i did.
Boot with the 14-current usb image.
This image imports correctly my newer zpool
I blindly copied the /boot directory from this usb image to my "ufs" boot filesystem. So grub takes this newer loader.
With this newer "loader" the zpool import correctly during the loading process.
Time for a drink.
 
  • Like
Reactions: mer
I learned two lessons.
-1. zpool upgrade can be dangerous
-2. Having a loader on a simple filesystem (ufs) can make life easier/flexible to fix problems.
 

zirias@

Developer
I blindly copied the /boot directory from this usb image to my "ufs" boot filesystem.
That's another dangerous thing to do, /boot contains, among other things, the kernel with all in-tree modules. You could easily end up booting a -CURRENT kernel that way, it just doesn't happen for you because you configured your loader to load the kernel from somwhere else (your ZFS pool). I would have recommended to only copy /boot/loader. Also be aware this is really a "band-aid repair", although I assumed a newer loader will work, it's certainly not a supported scenario.

-1. zpool upgrade can be dangerous
Only when you boot from that pool and fail to update the bootcode accordingly. But yes, it's a permanent operation, you can't "downgrade" a pool.

-2. Having a loader on a simple filesystem (ufs) can make life easier/flexible to fix problems.
I don't see why, as long as you do have some bootable system that can access a ZFS pool, you can easily modify its contents....
 
Using openzfs port i did a zpool upgrade which installed two features:
-com.delphix:head_errlog
-org.openzfs:zilsaxattr
My system did no longer boot.
Happened to me also. Fortunately this was a test machine :)

Forgot that boot loader does not support new OpenZFS features.

Way to downgrade it is to boot from another drive with FreeBSD standard features with OpenZFS (but not upgraded). After that you have a running system with OpenZFS. That allows to import the pool you just "destroyed" and use zfs send and zfs receive to move this pool to a new pool with standard FreeBSD features.
 
Alain De Vos if copied the directory, you may want to doubled check things like /boot/loader.conf to make sure it has what you desire. I'm not sure if the file exists on install media, but if it does you may have overwritten any settings you had.
 
Yes i needed to backup loader.conf, happily i had a backup.
After importing the zpool, freebsd uses openzfs port & then everything works fine.
 
  • Like
Reactions: mer
Top