HOWTO: Multi-boot FreeBSD + NetBSD + OpenBSD

rbranco

rbranco

I will have the latest version here:
github.com

howtos/BSD/Multiboot.md at master · ricardobranco777/howtos

tips and howtos. Contribute to ricardobranco777/howtos development by creating an account on GitHub.
github.com github.com


Installing NetBSD 10.x & FreeBSD 14.x in multiboot.

## Install NetBSD

Adapted from [NetBSD wiki](https://wiki.netbsd.org/Installation_on_UEFI_systems/):

Bash: 
# Use Korn shell
ksh


dkctl wd0 listwedges
gpt destroy wd0
gpt create wd0
# Use this size to avoid the formatting done as FAT-16
gpt add -l gptboot -t efi -s 512m wd0
gpt add -l NetBSD -t ffs -s 300g wd0
gpt add -l swap -t swap wd0


dkctl wd0 listwedges


newfs_msdos /dev/rdk5
mount -t msdos /dev/dk5 /mnt
mkdir -p /mnt/EFI/boot
cp /usr/mdec/*.efi /mnt/EFI/boot
umount /mnt


newfs -O 2 dk6

## Install FreeBSD

Adapted from [FreeBSD wiki](https://wiki.freebsd.org/RootOnZFS/GPTZFSBoot):

Bash: 
gpart add -a 1m -t freebsd-zfs -l FreeBSD ada0


# Optional: Enable encryption
geli init -g -s 4k gpt/FreeBSD
geli attach gpt/FreeBSD


mount -t tmpfs tmpfs /mnt


# Drop .eli extension if you don't want encrypt
zpool create -o altroot=/mnt zroot gpt/FreeBSD.eli


# Create ZFS pool with transparent compression enabled
zfs set compress=on zroot


zfs create -o mountpoint=none                                  zroot/ROOT
zfs create -o mountpoint=none                                  zroot/ROOT/default
mount -t zfs zroot/ROOT/default /mnt


zfs create -o mountpoint=/tmp -o exec=on -o setuid=off zroot/tmp
zfs create -o canmount=off -o mountpoint=/usr zroot/usr
zfs create zroot/home
zfs create -o setuid=off zroot/usr/src
zfs create zroot/usr/obj
zfs create -o mountpoint=/usr/ports -o setuid=off zroot/usr/ports
zfs create -o exec=off -o setuid=off zroot/usr/ports/distfiles
zfs create -o exec=off -o setuid=off zroot/usr/ports/packages
zfs create -o canmount=off -o mountpoint=/var zroot/var
zfs create -o exec=off -o setuid=off zroot/var/audit
zfs create -o exec=off -o setuid=off zroot/var/crash
zfs create -o exec=off -o setuid=off zroot/var/log
zfs create -o atime=on -o exec=off -o setuid=off zroot/var/mail
zfs create -o exec=on -o setuid=off zroot/var/tmp


chmod 1777 /mnt/var/tmp /mnt/tmp


zpool set bootfs=zroot/ROOT/default zroot


echo /dev/gpt/swap0 none swap sw 0 0 >> /tmp/bsdinstall_etc/fstab


# Backup GELI header
mkdir -m 0750 /mnt/var/backups/
cp -p /var/backups/* /mnt/var/backups/

After installation before reboot:

Bash: 
sysrc zfs_enable="YES"
cat >> /boot/loader.conf << EOF
aesni_load="YES"
geom_eli_load="YES"
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
EOF

Configure Grub on any bootable system. Make sure to populate this partition with the relevant .efi files.

Code: 
cat > /etc/grub.d/40_custom <<EOF
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
menuentry "NetBSD" {
        insmod part_gpt
        insmod chain
        chainloader (hd0,gpt1)/EFI/boot/bootx64.efi
}

menuentry "FreeBSD" {
        insmod part_gpt
        insmod chain
        chainloader (hd0,gpt1)/EFI/freebsd/loader.efi
}
EOF


grub2-mkconfig -o /boot/grub2/grub.cfg
 
I decided to install both on a Dell Precision 5520. I also planned to install OpenBSD but the fact that it doesn't support TRIM on SSD was a deal-breaker to me. Also OpenIndiana but the stupid ISO doesn't seem to work on this laptop.
 
I like to comment some points of your howto:

rbranco said:
geli init -g -s 4k gpt/FreeBSD
Click to expand...
If data key length ( -l ) is not specified, it defaults to 128 ( geli(8) ). Unless 128 is set intentional 256 would be a better choice.

256 is also set by the zfsboot installation script.

/var/log/bsdinstall_log
Rich (BB code): 
geli init -bg -e AES-XTS -J - -l 256 -s 4096 "nvd0p3"
(side note, setting -e AES-XTS explicitly is unnecessary, the default is AES-XTS anyway)


rbranco said:
zpool create -o altroot=/mnt zroot gpt/FreeBSD.eli
...
zfs create -o atime=on -o exec=off -o setuid=off zroot/var/mail
Click to expand...
In your pool creation "atime" is on by default, no need to enable explicitly on child dataset, it would be inherited from parent dataset.

The menu guided installation script disables "atime" in the pool creation.

/var/log/bsdinstall_log
Rich (BB code): 
zpool create -o altroot=/mnt -O compress=lz4 -O atime=off -m none -f "zroot"   nvd0p3.eli


rbranco said:
cat >> /boot/loader.conf << EOF
aesni_load="YES"
Click to expand...
aesni(4) is built in the GENERIC kernel, no need to load explicitly. What can be loaded is cryptodev(4), that's what the installer script sets.



By the way, you could skip the manual FreeBSD installation and perform a guided installation instead, which is much faster and has the advantage all system configuration is done by utilities (hostname, root password network interface configuration, adding users, etc).

Install first encrypted FreeBSD with a large swap size (set in the "ZFS Configuration" menu), then delete swap later to make space for other OS installations.

Choose the swap size as NetBSD installation size (eventually including other OS installations) + FreeBSD swap.

After FreeBSD installation, before reboot, edit FreeBSD /etc/fstab, change swap device name to GPT label. That should be gpt/swap0 (swap0 label set automatic by the installer script). If swap is encrypted gpt/swap0.eli.

Delete swap before leaving the FreeBSD installer (gpart(8) delete). You need swapoff(8) before delete.

Install NetBSD with space left for FreeBSD swap, copy NetBSD boot code into (FreeBSD created) ESP, create FreeBSD swap, encrypted or without encryption.

In the above method, the FreeBSD partition would be placed at the end of the disk. If you want, it can be moved to any position on the disc, provided the position is not to close to the original partition (read the next paragraph why).

Create a partition with the exact size in blocks of the original (not smaller, not larger), use gpart(8) -b option to specify where the partition should start, dd(1) original partition to new partition, delete original partition.
 
T-Daemon said:
By the way, you could skip the manual FreeBSD installation and perform a guided installation instead, which is much faster and has the advantage all system configuration is done by utilities (hostname, network interface configuration, adding users, etc).

Install first encrypted FreeBSD with a large swap size (set in the "ZFS Configuration" menu), then delete swap later to make space for other OS installations.

Choose the swap size as NetBSD installation size (eventually including other OS installations) + FreeBSD swap.

After FreeBSD installation, before reboot, edit FreeBSD /etc/fstab, change swap device name to GPT label. That should be gpt/swap0 (swap0 label set automatic by the installer script). If swap is encrypted gpt/swap0.eli.

Delete swap before leaving the FreeBSD installer (gpart(8) delete). You need swapoff(8) before delete.

Install NetBSD with space left for FreeBSD swap, copy NetBSD boot code into (FreeBSD created) ESP, create FreeBSD swap, encrypted or without encryption.

In the above method, the FreeBSD partition would be placed at the end of the disk. If you want, it can be moved to any position on the disc, provided the position is not to close to the original partition (read the next paragraph why).

Create a partition with the exact size in blocks of the original (not smaller, not larger), use gpart(8) -b option to specify where the partition should start, dd(1) original partition to new partition, delete original partition.
Click to expand...
This is wickedly clever! You already gave me the perfect excuse to install again because of the AES-256 setting but this is a new level. Thanks a lot!
 
rbranco said:
This is wickedly clever! You already gave me the perfect excuse to install again because of the AES-256 setting but this is a new level. Thanks a lot!
Click to expand...
Your are welcome.

I had the swap size idea borrowed from review "bsdinstall: Allow to set ZROOT partition size" to dual boot install FreeBSD and a Linux distribution.

I wanted the zroot partition aft the ESP and FreeBSD swap (positioned at the beginning of the disk), and install a Linux distribution at the end of disk.

I'm sure this can be done with a interactive script, but I'm not aware of such a script publicly available, and to write a script, I have to familiarize myself thoroughly with shell scripting, which I'm not inclined to do that at the moment.

T-Daemon said:
Install NetBSD with space left for FreeBSD swap, copy NetBSD boot code into (FreeBSD created) ESP, create FreeBSD swap, encrypted or without encryption.
Click to expand...
On second thought, the swap creation step can be done at the end of the initial FreeBSD installation.
 
T-Daemon said:
Your are welcome.

I had the swap size idea borrowed from review "bsdinstall: Allow to set ZROOT partition size" to dual boot install FreeBSD and a Linux distribution.

I wanted the zroot partition aft the ESP and FreeBSD swap (positioned at the beginning of the disk), and install a Linux distribution at the end of disk.

I'm sure this can be done with a interactive script, but I'm not aware of such a script publicly available, and to write a script, I have to familiarize myself thoroughly with shell scripting, which I'm not inclined to do that at the moment.


On second thought, the swap creation step can be done at the end of the initial FreeBSD installation.
Click to expand...
I just finished doing it and it worked wonderfully. I just skipped the part of moving the partition since it was taking some time and it's not really necessary.

I think there's a bug in the installer that makes /zroot a mountpoint for zroot. I had to manually fix this later. When I have time I'll open a bug though I didn't see this in -CURRENT.

For OpenIndiana I'm thinking of doing zfs send/receive from/to a VM.

Maybe also OpenBSD and do a weekly TRIM after mounting its partition from another BSD, WDYT?
 
You must log in or register to reply here.
Back
Top