Difficulty getting bootloader working manually (w/ZFS)

K

kniveton

Hi,

I have an HP Proliant Microserver (very nifty, inexpensive little server box BTW), which I want to use as a replacement for my 8 year old Dell PE400SC home gateway/mailserver/webserver FreeBSD box. This small server accepts four 3 1/2" disks, and my intention is to use ZFS in a RAID 10 configuration with the four disks. My OS is FreeBSD 8.1-R, though I am using the PC-BSD DVD I got at MeetBSD as a quick way to boot into live mode (emergency shell actually) since it also loads up the kernel modules I need.

The first install I tried was just a typical 8.1-RELEASE install from an 8.1-R CD. Installed onto the first drive, and that went fine, it boots up and works well.

Next, I tried an install using the RootOnZFS instructions for a GPT-based disk with a ZFS boot partition. I was able to go through the full instructions, create a ZFS pool install FreeBSD onto it, etc. However, when I tried to reboot, the BIOS complained that it could not find boot media. :-(

At this point I figured that the BIOS just doesn't support GPT, so I wiped the GPT info from the beginning and end of the partitions, and went about performing a ZFS-Root on MBR disk build. Again, I got all the way through the wiki instructions, tried to reboot, and nothing.

Finally, I started over once more, with a UFS boot partition (1G) and then ZFS pool after it. Once more, went all the way through the instructions, got the zpool set up and FreeBSD installed on it. In all cases, I could not get it to boot.

I found a smattering of documentation about the boot process and how to install the different boot stages, all of it somewhat differing based on its vintage. I tried to use dd, gpart, and even fdisk (though not as successfully) to install and re-install the MBR and boot / boot0 according to all the documentation I could find. Nothing worked.

At this point I'm about to give up and go back to a standard UFS install, and simply mirror to a second drive. However I did want to play with ZFS and especially for using an SSD drive as a cache which could be very useful for my mailserver. The alternative would be to go with UFS, and then try and use the SSD as a caching mechanism for mysql (though all my mail is cyrus in flat files at the moment).. but I would prefer the ZFS route if I can get it working.

Any thoughts on this at this point? I still have the UFS boot part and then zfs pool and swap parts sitting there on disk. If anyone can help me to get it to boot I would be highly grateful.

Thanks!
 
Use the PC-BSD CD to create a simple 2-disk mirror ZFS pool. Then select "Install FreeBSD". And you're done. Boot into the new OS install. And add the other 2-disk mirror to the pool.

All the boot magic will be done for you. :)
 
Thanks, I will try one more time with dd, disklabel, et al. If that fails, I will use your method.

My original intent was to create a zfs pool including two mirror sets (zpool create zroot mirror disk1 disk2 mirror disk3 disk4). If I use PCBSD install to create the mirror pools for me, can I later put them in the same pool? Are they then concatenated, striped, or some combination? (Need to do more zfs research here perhaps..)

Thanks
 
Keep it simple. :)

Do the install onto a single mirror vdev.

After the install, and after you are booted into the installed OS, then add a second mirror vdev to the pool. After that, you will have a ZFS pool with two mirror vdevs. And ZFS will stripe the data across the two vdevs. Thus giving you the equivalent of a RAID10 (stripe across two mirrors).
 
Now that I'm trying to get the PCBSD install going, it's not at all obvious how to achieve the same goal with PCBSD install, and the documentation basically has this type of config as a "to-do" under documentation. I was hoping it would be more obvious than writing the bootmgr's but it's not :-(

Even after configuring a zfs mirror setup, it just wrote a single disk, and started installation.. after a while, the installer encountered an error and aborted. So unfortunately I'm still in the same spot... wiping the disks and trying once more...
 
It is very, very difficult to tell an installer, "WIPE out the disk, and do THIS". It's back and forth to the shell to use various utilities to continuously try and wipe various parts of the disk to get a clean start.
 
OK, tried a different approach.

1. Booted gparted USB stick, used dd to wipe the entire disk(s).
2. Back to PC-BSD installer. created 1G UFS /boot, large zfs pool with mount points from RootOnZFS wiki (/, /var, /var/tmp, /usr, /usr/home, /usr/src, .....), and then the rest of the disk, approx 9G, for swap. Again, I put the swap at the end of the disk so if I need to replace with a similar disk but different mfgr with slightly smaller size, that's absorbed by the swap.
3. Selected PC-BSD for install (not FreeBSD), and instaall PC-BSD bootloader.

Install almost completed, but failed with an error. Looking at the error log it looks like it was essentially completed. BOOT FINALLY WORKED! Yay! Moral of the story is not to try to install onto disks with existing partition info.

Unfortunately the UFS /boot seemed to really throw things off. The system could not successfully start up.

4. Repeat install from above, but installed FreeBSD, and did not use the /boot UFS. Just used a ZFS partition and a swap. Install fails again at the CD unmount (essentially, success).

Rebooted, and get essentially same set of errors:

Code: 
[..]
Mounting local file systems:.
touch: not found
[..]
ip6addrctl: not found
Starting devd.
logger: not found
Starting ums0 mousedeval: /usr/sbin/moused: not found
[..]


Seems mounting of the zfs pool was unsuccessful. The zfs kernel modules were successfully loaded. Any pointers on how to further dig into this? Can I somehow see the mount sequence by escaping at the bootloader?
 
Sorry for the blog-like postings on this thread.. I'm hoping it will help someone else in debugging their install.

So I booted into fixit mode, imported the tank0 zpool created by the PC-BSD installer, and I note something interesting: half of the created filesystems (tank0, tank0/tmp, tank0/usr,...) are set with a mountpoint prefixed by /mnt (/mnt, /mnt/tmp, /mnt/usr,...), and half of them are set correctly (tank0/var -> /var, tank0/usr/src -> /usr/src,...)

I reset the mountpoints (tank0 to legacy, and all others to the relative /directory), and rebooted. Voila, now fully working.

Next step is to get second drive up as mirror. Plan is to install similarly to the second drive (to get bootloader code), then wipe the zfs partition, reboot from first drive, create zfs vdev and add it to the first disk's pool as a mirror drive.
 
Install on second disk went much smoother when I only added one mountpoint to the ZFS partition: / .. the only issue was some slight difficulty adding a swap partition (perhaps because the one from the first disk was sensed) but I did get it through.
Popped out first disk, rebooted into second disk and everything looks good.
Put first disk back in, booted back into PC-BSD emergency console, and took a look at the pools. I actually re-created the pools as a mirror and re-installed. Everything is working great.

In summary, I wish I could figure out what the PC-BSD installer is doing differently to the FAQ's, in order to get GPT/ZFS smoothly booting. I believe there are some tweaks needed to the manual instructions.
 
You must log in or register to reply here.
Back
Top