Running a FreeBSD system from a whole-disk ZFS pool?

J

jem

I'm pondering the possibility of running a FreeBSD system with the root filesystem on a zpool comprised of whole disks, i.e. not building the zpool vdevs on underlying MBR or GPT partitions.

I know that you can not boot directly from such a pool as ZFS's exclusive use of the disks leaves no space to install a boot loader, but I'm wondering if I can put just the ZFS bootcode on a USB flash stick and run the very first part of the boot process from there.

If I set up a bootable USB stick as follows, would this contain enough of the ZFS boot code to get as far as probing all storage devices, locating and mounting the pool and continuing as normal?

Code: 
gpart create -s GPT da0
gpart add -s 64k -t freebsd-boot da0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

Any input from people with a detailed understanding of the FreeBSD ZFS boot process would be appreciated, along with any potential drawbacks of this idea, even if it is feasible.
 
I experimented with this idea but couldn't get it working.

It seems the gptzfsboot image only looks for zpools that reside within GUID partitions.

I wonder if the zfsboot image might look for whole-disk zpools as well as those residing within MBR partitions? I've inspected the source code of the boot images but my C and assembler skills are not strong.
 
I was able to get a 'whole disk' pool to boot by following the standard GPTZFSBoot wiki article with a few changes based on the ZFS+GELI+USB post in the Howto section.

I'm no expert on the FreeBSD boot process so there may be things wrong here, or things I don't need to do but this does appear to boot successfully for me. I welcome anyone to point out any mistakes.

I'm booting into fixit off the live cd. The live cd is mounted on /dist with the 8.1 install disk (what I had available) mounted on /cdrom. I'm doing all this inside VMware by the way.

Partition the 'USB' stick
Code: 
gpart create -s gpt da0
gpart add -b 34 -s 64k -t freebsd-boot da0
gpart add -t freebsd-ufs da0

Add boot code, format the USB disk and make the all important /boot/zfs directory
Code: 
gpart bootcode -b /dist/boot/pmbr -p /dist/boot/gptzfsboot -i 1 da0
newfs -O2 /dev/da0p2
mkdir /boot/zfs

Mount the USB disk (and install cdrom if needed)
Code: 
mount /dev/da1p2 /mnt
mkdir /cdrom
/dist/sbin/mount_cd9660 /dev/acd1 /cdrom

Load ZFS and create the pool
Code: 
kldload /dist/boot/kernel/opensolaris.ko
kldload /dist/boot/kernel/zfs.ko

zpool create tank da1
zpool set bootfs=tank tank

From here I followed the normal wiki instructions http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/Mirror

I installed base & generic kernel to both the USB stick & the ZFS pool (DESTDIR=/mnt and DESTDIR=/tank)

Then I set up /etc/rc.conf, /boot/loader.conf, root password & timezone on the ZFS pool as per the wiki and created an empty /etc/fstab. I also added /boot/loader.conf to the USB stick.

Once this was done I copied the zpool.cache to /boot/zfs/zpool.cache on both USB & pool and then set mountpoint=legacy on the root zfs file system.

That was about it.
You'll probably want to create more ZFS file systems than just root but you should be able to adjust these instructions to do what you want.

Obviously if you change your kernel, you'll need to install it to the USB stick.

If you want swap you can create a zvol and set the org.freebsd:swap property to on.
 
Thanks for the suggestion usdmatt.

I can understand that it's probably quite straightforward to get a FreeBSD system running from a whole-disk pool if you boot from a USB stick that also contains almost an entire FreeBSD install itself. It's just a matter of booting and loading the kernel and modules from USB stick, then mounting the whole-disk pool as the root filesystem.

However, my objective here is to see if I can get such a setup working with only the ZFS boot code running from the USB stick. The loader, kernel and modules would still be loaded from the pool.

I imagine it to work something like this:

  1. System boots from USB stick containing only ZFS boot code (zfsboot image maybe?)
  2. ZFS boot code probes other storage devices (SATA disks) and finds whole-disk raidz pool
  3. Boot code loads zfsloader from pool and hands control over to it
  4. zfsloader loads kernel and modules and continues booting

The main question is whether the ZFS boot code included with FreeBSD has the capability to search for whole-disk pools, or whether it only looks for pools inside partitions. gptzfsboot doesn't seem to. Maybe zfsboot does?
 
You must log in or register to reply here.
Back
Top