multiple boot environments with ZFS

sh4m1l65

sh4m1l65

I've been rooting on ZFS for about a month now, but I'm wondering what is considered best-practice (or, even a workable technique) for rapidly cloning & booting into different (FreeBSD) boot environments?

My (initial) filesystem is on a zraid-1 pool ("zr1") and looks something like this:
Code: 
[user@home ~]$ zfs list -r zr1/Systems/FreeBSD-8.0/amd64
NAME                                                USED  AVAIL  REFER  MOUNTPOINT
zr1/Systems/FreeBSD-8.0/amd64                      1.76G   760G   270M  /
zr1/Systems/FreeBSD-8.0/amd64/usr                  1.38G   760G   243M  /usr
zr1/Systems/FreeBSD-8.0/amd64/usr/local             363M   760G   361M  /usr/local
zr1/Systems/FreeBSD-8.0/amd64/usr/ports             491M   760G   294M  /usr/ports
zr1/Systems/FreeBSD-8.0/amd64/usr/ports/distfiles   162M   760G   162M  /usr/ports/distfiles
zr1/Systems/FreeBSD-8.0/amd64/usr/ports/packages   34.7M   760G  34.7M  /usr/ports/packages
zr1/Systems/FreeBSD-8.0/amd64/usr/src               310M   760G   310M  /usr/src
zr1/Systems/FreeBSD-8.0/amd64/var                   119M   760G   533K  /var
zr1/Systems/FreeBSD-8.0/amd64/var/crash            27.7K   760G  27.7K  /var/crash
zr1/Systems/FreeBSD-8.0/amd64/var/db               94.6M   760G  87.6M  /var/db
zr1/Systems/FreeBSD-8.0/amd64/var/db/pkg           3.57M   760G  3.57M  /var/db/pkg
zr1/Systems/FreeBSD-8.0/amd64/var/empty            26.9K   760G  26.9K  /var/empty
zr1/Systems/FreeBSD-8.0/amd64/var/log               794K   760G   440K  /var/log
zr1/Systems/FreeBSD-8.0/amd64/var/mail             26.9K   760G  26.9K  /var/mail
zr1/Systems/FreeBSD-8.0/amd64/var/run               162K   760G  93.5K  /var/run
zr1/Systems/FreeBSD-8.0/amd64/var/tmp              22.4M   760G  22.3M  /var/tmp

Currently, I've got it booting with
Code: 
[user@home ~]$ zpool get bootfs zr1
NAME  PROPERTY  VALUE                          SOURCE
zr1   bootfs    zr1/Systems/FreeBSD-8.0/amd64  local
[user@home ~]$ zfs get -r mountpoint,canmount zr1/Systems/FreeBSD-8.0/amd64
NAME                                                              PROPERTY    VALUE                                                             SOURCE
zr1/Systems/FreeBSD-8.0/amd64                                     mountpoint  /                                                                 local
zr1/Systems/FreeBSD-8.0/amd64                                     canmount    noauto                                                            local
zr1/Systems/FreeBSD-8.0/amd64/usr                                 mountpoint  /usr                                                              inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/usr                                 canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/usr/local                           mountpoint  /usr/local                                                        inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/usr/local                           canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/usr/ports                           mountpoint  /usr/ports                                                        inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/usr/ports                           canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/usr/ports/distfiles                 mountpoint  /usr/ports/distfiles                                              inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/usr/ports/distfiles                 canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/usr/ports/packages                  mountpoint  /usr/ports/packages                                               inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/usr/ports/packages                  canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/usr/src                             mountpoint  /usr/src                                                          inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/usr/src                             canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var                                 mountpoint  /var                                                              inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var                                 canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/crash                           mountpoint  /var/crash                                                        inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/crash                           canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/db                              mountpoint  /var/db                                                           inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/db                              canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/db/pkg                          mountpoint  /var/db/pkg                                                       inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/db/pkg                          canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/empty                           mountpoint  /var/empty                                                        inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/empty                           canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/log                             mountpoint  /var/log                                                          inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/log                             canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/mail                            mountpoint  /var/mail                                                         inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/mail                            canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/run                             mountpoint  /var/run                                                          inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/run                             canmount    on                                                                default
zr1/Systems/FreeBSD-8.0/amd64/var/tmp                             mountpoint  /var/tmp                                                          inherited from zr1/Systems/FreeBSD-8.0/amd64
zr1/Systems/FreeBSD-8.0/amd64/var/tmp                             canmount    on                                                                default

Now, I'd like to recursively clone the zr1/Systems/FreeBSD-8.0/amd64, and mount everything at /mnt, and do some sysinstall to come up with a zr1/Systems/FreeBSD-8.0/i386 (and eventually a FreeBSD-9-CURRENT/amd64 and so-on). But, of course with mountpoint=/, it's difficult. I can, of course, create the new filesystems by snapshotting and cloning, and it's fine because I set canmount=noauto and mountpoint=/, and I'm fine creating all the "children" filesystems by inheriting mountpoint, because zfs doesn't automatically try to mount those children right away. But, as soon as I try to reboot, I am assuming all bets are off (with a race to see who's last for a given point in the hierarchy).

Another method I've thought of, would be setting mountpoint=legacy at the root of each bootenv, and manually maintain the /etc/fstab in there. This seems to me to be the most likely way to actually achieve multiple boot environments currently. I just really dislike the manual maintenance idea, when zfs generally knows how to "inherit" settings from some parent.

I have some vague idea, from reading a little of what the Internet knows about ZFS, that Solaris does something along the lines of what I want, using something called "zones" to keep dangerous mountpoints from clobbering your working system, but 1) I'm not sure I completely understand this bit and 2) it seems apparent to me that zones are NOT in FreeBSD's current ZFS implementation.

Am I the only one out on this particular limb? If not, what have others done in this regard? Is it reasonable to want to boot multiple different environments from the same pool?

Thanks,
--Tk
 
Search the 'Net for "FreeBSD ZFS Boot Environment" for details on doing just this. There's at least one project with working code for this.
 
phoenix said:
Search the 'Net for "FreeBSD ZFS Boot Environment" for details on doing just this. There's at least one project with working code for this.
Click to expand...

I can see the manageBE project, but from the example.txt (as well as the manageBE script), I gather that it manages "boot environments" on one filesystem each - which loses some of the advantages of zfs. I mean, clearly, /usr (or, more to the point, /usr/bin) compiled for FreeBSD-8.0-RELEASE on amd64 is not going to be the /usr (or /usr/bin) I'll want when I choose to boot FreeBSD-9.0-CURRENT on i386. I'm looking for a way to get all the "children" correctly associated and mounted when I choose to boot a different "parent" (/).
 
yah, that's what opensolaris does. It's really great. I'd LOVE to see this make it's way into FreeBSD. I would imagine it should be possible...there is a special tool for creating new boot environments in OpenSolaris, but i can't remember what it is called...perhaps someone will port it or use some of it's methods.

I would love to see this.
 
If you want to use a single pool for multiple OSes, and select which OS (and subsequent filesystems) to use, then you'll probably have to use the manageBE setup (to select the / filesystem) and a legacy /etc/fstab (to select the filesystems to use).
 
phoenix said:
If you want to use a single pool for multiple OSes, and select which OS (and subsequent filesystems) to use, then you'll probably have to use the manageBE setup (to select the / filesystem) and a legacy /etc/fstab (to select the filesystems to use).
Click to expand...

Yeah, while less flexible than I had in mind (I've been hoping to stumble on some technique that allows the "children" to by dynamically/automatically found/mounted), I was thinking this might be the only way to do it with one pool and fine-grain children with zfs.
 
w7tek said:
I can see the manageBE project, but from the example.txt (as well as the manageBE script), I gather that it manages "boot environments" on one filesystem each - which loses some of the advantages of zfs. I mean, clearly, /usr (or, more to the point, /usr/bin) compiled for FreeBSD-8.0-RELEASE on amd64 is not going to be the /usr (or /usr/bin) I'll want when I choose to boot FreeBSD-9.0-CURRENT on i386. I'm looking for a way to get all the "children" correctly associated and mounted when I choose to boot a different "parent" (/).
Click to expand...

I just commited support for child-filesystems in manageBE, but your setup will not be easily supported out-of-the-box by manageBE. It should be possible to convert it into something compatible though.

http://anonsvn.h3q.com/projects/freebsd-patches/blog/2009/12/24/child_filesystems_in_manageBE
 
You must log in or register to reply here.
Back
Top