Solved Recovering from a mistake

"canmount" is off. By having that off, "any dataset under /usr without it's own dataset is part of the root dataset".
OK, that part is a little confusing to me... "any dataset under /usr without it's own dataset is part of the root dataset"... You mean, subdirectory like /usr/local ? So (referencing vermaden 's post), /usr/ports, being a dataset under /usr, does actually have its own dataset with "canmount" being on.
zroot/usr/ports canmount on default
If "canmount" were off there, then the evertyhing under /usr/ports/ would be part of the root dataset, is that correct? Or would it be part of the /usr dataset?
 
Yes.
If /usr/local is not actually a data set, then it winds up as a subdirectory of /usr. Since /usr is a dataset with "canmount off", /usr/local is part of the root dataset (and included in BEs).
/usr/ports is a dataset, with canmount on, mountpoint of /usr/ports so it is not part of the root dataset.

Your second question:
Mostly correct, but since /usr is cannmount off then if /usr/ports was canmount off, it would wind up under root dataset.
 
E
Yes.
If /usr/local is not actually a data set, then it winds up as a subdirectory of /usr. Since /usr is a dataset with "canmount off", /usr/local is part of the root dataset (and included in BEs).
/usr/ports is a dataset, with canmount on, mountpoint of /usr/ports so it is not part of the root dataset.

Your second question:
Mostly correct, but since /usr is cannmount off then if /usr/ports was canmount off, it would wind up under root dataset.
Exactly.
 
I don't think that this is hijacking the thread, since it's essentially the same subject... I am grateful to all contributors, and seek comments for some follow-on questions.

I want to upgrade my FreeBSD 12.2 ZFS server (with ZFS root) to FreeBSD 13.

The Klara System site has some detailed instructions for upgrading from FreeBSD 12 to 13 using ZFS boot environments. This plan is pretty much straight from them:
Code:
bectl list
sudo bectl create FreeBSD13
sudo bectl mount FreeBSD13
mount | grep FreeBSD13 | cut -d' ' -f3  # ==> /tmp/be_mount.nOCj
freebsd-update -b /tmp/be_mount.nOCj -d /tmp/be_mount.nOCj/var/db/freebsd-update
 -r 13.0-RELEASE upgrade
freebsd-update -b /tmp/be_mount.nOCj -d /tmp/be_mount.nOCj/var/db/freebsd-update
 -r 13.0-RELEASE install
# Temporary activation
bectl activate -t FreeBSD13
# Complete the installation
freebsd-update install
# Commit to use it
bectl activate FreeBSD13
Vermaden's plan covers the same process, but is sufficiently different to be worth discussing.

It uses a chroot(8) and therefore needs to mount devfs(8). It does some other things that I need to clarify:
  1. deletes old patches in /var/db/freebsd-update (not sure why)
  2. runs "freebsd-update install" three times (Klara only do it twice)
  3. runs "pkg upgrade" which is fine, but I think could be deferred until FreeBSD13 is running.
The default ZFS installation (Guided Root-on-ZFS) installs on a GPT (BIOS) partition by default, so I don't have a UEFI boot partition. I am not sure if the loader(8) changes mentioned by vermaden are required.

The ZFS production server root is installed as ZFS mirror:
Code:
[sherman.149] $ zfs list zroot
NAME    USED  AVAIL  REFER  MOUNTPOINT
zroot  14.9G   194G    88K  /zroot

[sherman.150] $ zpool status zroot
  pool: zroot
 state: ONLINE
  scan: scrub repaired 0 in 0 days 00:00:54 with 0 errors on Sun Aug  8 03:02:38
 2021
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            ada1p3  ONLINE       0     0     0
            ada0p3  ONLINE       0     0     0

errors: No known data errors

[sherman.151] $ gpart show
=>       40  488397088  ada0  GPT  (233G)
         40       1024     1  freebsd-boot  (512K)
       1064        984        - free -  (492K)
       2048   33554432     2  freebsd-swap  (16G)
   33556480  454840320     3  freebsd-zfs  (217G)
  488396800        328        - free -  (164K)

=>       40  488397088  ada1  GPT  (233G)
         40       1024     1  freebsd-boot  (512K)
       1064        984        - free -  (492K)
       2048   33554432     2  freebsd-swap  (16G)
   33556480  454840320     3  freebsd-zfs  (217G)
  488396800        328        - free -  (164K)

[sherman.154] $ gmirror status
       Name    Status  Components
mirror/swap  COMPLETE  ada0p2 (ACTIVE)
                       ada1p2 (ACTIVE)
So gmirror(8) is mirroring the swap on slice 2, and zfs(8) is mirroring slice 3. Nothing is mirroring slice 1 (freebsd-boot). So if I make changes there, I expect that they will have to be manually duplicated.

Will I need to change anything in /boot?

It seems likely that if I need to update the boot code, then this would be appropriate (taking /boot/gptzfsboot from a FreeBSD 13 system):
Code:
/sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada0
/sbin/gpart bootcode -p /boot/gptzfsboot -i 1 ada1

sum(1) tells me that /boot/gptzfsboot differs from FreeBSD 12 to FreeBSD 13.
 
I've done the vermaden procedure on a system with a mirror pair for zroot. Worked just fine.
pkg upgrade could be left until you've rebooted into the new BE, but if you do it in the chroot, the new BE is completely upgraded so when you reboot it's complete and all you need to do is test.
freebsd-update install: if you don't do it in chroot or specify directory you typically need to run it twice. Once to install new kernel and initial bits, then you'd reboot, then run it again to finish installing the updates.
I'm not 100% sure but perhaps running pkg upgrade could sometimes require the extra one. If it has nothing to do, it tells you.

Bootcode update:
If you do zpool upgrade from the FreeBSD-13 BE then you will need to update the bootcode. You only need to do zpool upgrade if you need new features or you are sure you don't need those pools on FreeBSD-12 (basically won't roll back to a -12 BE).
The commands listed look correct. The general rule is:
Update the bootcode on every drive that is capable of being used to boot. If you had a 5-way mirror, do it on all 5 disks.
 
freebsd-update install: if you don't do it in chroot or specify directory you typically need to run it twice. …

Flip side: whilst chroot(8) approaches (including the one documented by vermaden) do allow some streamlining i.e. fewer restarts of the OS, there's a slight risk.

When I last checked: if the end user follows an on-screen instruction to run freebsd-udpate fetch etc., things can go wrong. I doubt that the official update routine is designed to allow omission of restarts in a chroot environment.

Looking ahead, binary updates should make things simpler for end users.
 
grahamperrin I don't understand what point you are trying to make.
freebsd-update process mirrors the "install from source" process (starting after the make buildworld step)
Install the new kernel
Install the new userland

The reason (only?) for the reboot between the the two steps is the new userland may require/use new interfaces in the new kernel. If you don't reboot you would be installing new userland on top of an old kernel that is running.

When you do the process in a chroot environment you are never in the position of trying to run new userland on an old kernel, so no need to try and do a reboot. In fact until the new BE, updated in the chroot is actually made active and booted into, it has zero impact.

Looking ahead, binary updates should make things simpler for end users.
This is confusing: freebsd-update is doing binary updates. Do you mean "base packages" or "pkg-base" (I'm not sure of the correct current moniker)? I'm not sure how much simpler that will make things. Instead of doing "freebsd-update" you wind up doing "pkg upgrade" which would be a more consistent user experience, but you still have the requirements of keeping kernel and userland in sync, you still have "If I do this in a new BE I have to reboot to use it", if you are doing it on a live system you still have to pkg upgrade kernel, reboot, pkg upgrade userland.
 
To test the process I made a new VM and installed FreeBSD 12.2.

The only significant difference between this test system and my production ZFS server is no root mirror on the VM [so only one (virtual) boot partition to update].

This morning I followed the Vermaden procedure, and with some small changes it worked well. The changes were:
  • I upgraded to 13.0-RELEASE.
  • My korn shell (/bin/ksh for me) complained about loss of access to /dev/tty immediately after the chroot(8), and ultimately became unresponsive. I rebooted, started again, and exec'd a new one immediately after mounting /dev in the chroot'ed environment to fix the problem:
    Code:
    exec /bin/ksh
  • To update the GPT (BIOS) freebsd-boot partition on slice 1 to support any new ZFS features, after booting FreeBSD 13, and before any attempt to run a "zfs upgrade", I re-wrote the boot partition:
    Code:
    /sbin/gpart bootcode -p /boot/gptzfsboot -i 1 vtbd0
    .
The chroot(8) method works well. Thanks to everyone, especially vermaden.
 
Top