If you have a bootable FreeBSD USB installer, I would be inclined to take advantage of
zpool-checkpoint(8) to enable some reasonably risk-free experimentation, as you can always boot off the usb stick and import with --rewind-to-checkpoint. (
Be sure to discard the checkpoint once you're happy with everything.)
Under the protection of a zpool checkpoint, I think this will get you back to a ~normal BE configuration:
I believe the sticky wicket you'll encounter is that your
zroot/ROOT/default/usr (and /
var) were set
canmount=on; typically (when placed at the default top of the pool instead of inside zroot/ROOT/*) this is
canmount=off, so they can serve as "containers" to hold child /usr(or var)-mounted zfs filesystems, while leaving any non-explicit-filesystem /usr (var) content on your boot environment. In this (standard) configuration, the explicit filesystems (/usr/ports, ...) under them are
not part of the boot environment.
The main issue is that you have other directories (/usr/bin, for example) that are
actually stored in zroot/ROOT/default/usr, while (again, by default for BEs) those should explicitly be part of (each)
zroot/ROOT/<bename>. To fix this, I would
boot from the USB stick, import zroot with an altroot:
mkdir /tmp/zr; zpool import -NR /tmp/zr zroot; zfs mount zroot/ROOT/13.1-RELEASE; zfs mount -a, and (
after a zpool checkpoint of zroot) :
1) Make sure the expected filesystems/directories are all mounted (like they are when you are running) under /tmp/zr.
2)
Migrate /usr and /var contents (that aren't explicit zfs-fileystems): For usr and var SUBDIRs,
mkdir /tmp/zr/SUBDIR2; rsync -aHvx /tmp/zr/SUBDIR/ /tmp/zr/SUBDIR2/. With -x on the rsync it won't descend into the child
filesystems (/usr/obj, /usr/ports), but will copy everything else (/usr/bin, /usr/local/bin, ...) into a (now zroot/ROOT/13.1-RELEASE - backed) /usr2 (or /var2). At this point, zfs unmount zroot/ROOT/default/usr/ [...]/var/ and their children, make sure /tmp/zr/usr and /tmp/zr/var are gone, (if they are empty directories now, delete them) and
mv /tmp/zr/usr2 /tmp/zr/usr; mv /tmp/zr/var2 /tmp/zr/var
Then:
zfs rename zroot/ROOT/default/usr zroot/usr
zfs rename zroot/ROOT/default/var zroot/var
zfs rename zroot/ROOT/default/tmp zroot/tmp
zfs rename zroot/ROOT/default/root zroot/root
zfs set canmount=no mountpoint=/usr zroot/usr
zfs set canmount=no mountpoint=/var zroot/var
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/root zroot/root
zfs mount -a
And make sure the system under /tmp/zr looks "normal" (check contents of /tmp/zr/usr and /tmp/zr/var in particular.)
At this point try booting into your reworked system. If it didn't work out like hoped, boot into the USB stick again, and use
zpool import -N --rewind-to-checkpoint zroot; zpool export zroot and restart. Don't forget to remove the checkpoint if you don't need it anymore.
And finally, at this point, your boot environments should be "vanilla" -- although on 13.1-RELEASE will be usable (since the others are missing the contents under /usr, /var), but going forward, new ones should be good, and you should be able to get rid of the old ones. Check all that functionality
before removing the checkpoint.
