Unattended install on FreeBSD13 with zfs/efi

cluepon

New Member

Reaction score: 1
Messages: 9

Hi,

I'm trying to upgrade my set of FreeBSD12 machines to FreeBSD13. Since I use them for automatically configured jails, the host machines are basically 'reinstall to upgrade' devices on ESX.

My existing installerconfig setup doesn't work: it installs but doesn't boot. I did a manual install to compare, which results in a FAT partition mounted as /boot/efi. But I don't really see how to do that unattended.

Does anyone have experience with this, and a working installerconfig example that I might compare?

Currently the start of my installerconfig file is:

Code:
export DISTRIBUTIONS="kernel.txz base.txz localinstall.txz"
export INTERFACES="vmx0 vmx1"
export ZFSBOOT_VDEV_TYPE=stripe

# for esx vm
export ZFSBOOT_DISKS=da0

# next 2 tests for FBSD13
export ZFSBOOT_BOOT_TYPE=UEFI
export ZFSBOOT_PARTITION_SCHEME=GPT

export ZFSBOOT_SWAP_SIZE=2g
export ZFSBOOT_CONFIRM_LAYOUT=0
export ZFSBOOT_SWAP_ENCRYPTION=YES
export nonInteractive="YES"

Any advice here, perhaps?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,015
Messages: 38,467

I did a manual install to compare, which results in a FAT partition mounted as /boot/efi. But I don't really see how to do that unattended.
The /usr/libexec/zfsboot part only creates the ESP partition and the defined zpool, the required loader.efi file is copied to the ESP in /usr/libexec/bootconfig.
 
OP
C

cluepon

New Member

Reaction score: 1
Messages: 9

But how would I make that happen in my installerconfig file? I can't seem to find a config that will give me a bootable system.
 

Alain De Vos

Daemon

Reaction score: 613
Messages: 2,071

I think the easiest is to go to the commandline and write all your instructions line by line, or to write a short shell script to do it.
And the bootloader is also important. I use grub to chainload the freeebsd /boot/loader.
Otherwise look at "man gpart" for freebsd bootloaders.
 

T-Daemon

Daemon

Reaction score: 829
Messages: 1,691

cluepon, if you are still interested in a unattended installation on a virtual UEFI system I might have a solution.

I've done some testing on VirtualBox. There also a unattended installation root on ZFS fails on a UEFI system, but succeeds on a non-UEFI system. The ZFS environment variables in /etc/installerconf won't create a bootable system. The following workaround could resolve the problem on ESXi as well.

The issue here is, the FreeBSD loader (loader.efi) isn't copied into the ESP and no UEFI boot variable is created. This seems to be a bug in the 'bootconfig' installation script. There is the code to populate the ESP, at the end of the script the code to create a UEFI boot entry. However I couldn't put my finger on what might be the cause to fail on virtual UEFI systems. On the other hand the cause could be somewhere else, 'zfsboot' maybe?

The workaround is to let the 'installerconfig' script copy the loader into the ESP.

The VirtualBox system setup:
  • EFI system setting
  • New systems virtual image attached on SATA port 0
  • FreeBSD installer on USB stick, configured as a rawdisk, attached on SATA port 1
The /etc/installerconfig of the test setup:
Code:
DISTRIBUTIONS="kernel.txz base.txz"

export ZFSBOOT_DISKS=ada0
export ZFSBOOT_VEDV_TYPE=stripe

export ZFSBOOT_SWAP_SIZE=2g
export ZFSBOOT_SWAP_ENCRYPTION="YES"

export nonInteractive="YES"

#!/bin/sh

# mount ESP, copy loader

mount_msdosfs /dev/ada0p1 /mnt
mkdir -p /mnt/efi/boot
cp /boot/loader.efi /mnt/efi/boot
umount /mnt/mnt

# /mnt/mnt is not a typo. ESP is mounted in chroot on
# /mnt of the new system, which makes it /mnt/mnt outside
# of chroot.

# Create UEFI boot variable

efibootmgr -a -c -l ada0p1:/efi/boot/loader.efi -L FreeBSD

This script will install on a virtual UEFI system, in this case VirtualBox VM, root on ZFS, GPT partition scheme (no need to set explicitly a UEFI, GPT partition scheme or layout confirmation variables as you did) mount the ESP, copy the FreeBSD loader, create a UEFI boot variable named 'FreeBSD' as first boot device.

After the installation process ends it will reboot automatically into the new installed system.
 
OP
C

cluepon

New Member

Reaction score: 1
Messages: 9

I'll check when I upgrade the next box. Instructions sound easy. I've looked a little bit at diff'ing 12-stable and 13-stable for /usr.sbin/bsdinstall/scripts. It seems to be producing the label (if I dd the bootpart of the device, and do the install, the bios shows me a FreeBSD label to boot. So bootconfig seems to be called. Next stop: see how I could possible debug that.

Do you have enough info and the interest to create a bug report? Or should I look when I find the time?
 

T-Daemon

Daemon

Reaction score: 829
Messages: 1,691

It seems to be producing the label (if I dd the bootpart of the device, and do the install, the bios shows me a FreeBSD label to boot. So bootconfig seems to be called.
According to bsdinstall_log bootconfig is called on my test system as well but the ESP isn't populated.

Have you checked if loader.efi (or bootx64.efi) is present. A interactive installation creates in the ESP these directories and files:
Code:
/efi/boot/bootx64.efi
/efi/freebsd/loader.efi

Do you have enough info and the interest to create a bug report? Or should I look when I find the time?
The question isn't if I have interest, I could file a PR but your case has a real use case impact (happening on ESXi), mine is a test environment in VirtualBox. I doubt there is much demand in unattended installations on VirtualBox. You should open a PR, your case probably attracts more attention then mine and the developers are more interested to resolve the issue.

But I'm willing to contribute to the PR with what info I have. Drop a PR number here in this thread when there is a report. Or, I also look at bugzilla regularly and may find it there if one is opened.
 
OP
C

cluepon

New Member

Reaction score: 1
Messages: 9

I checked (install iso and single usermode), looked at gpart list, mounted da0p1, nothing there. I'll see if I can free a machine from jails, so I can test it on a reinstall.
 
OP
C

cluepon

New Member

Reaction score: 1
Messages: 9

I thought of the fact that I can simply edit /usr/libexec/bsdinstall/* files for debugging. Started with a f_dprintf "`mount`", and it seems /mnt/boot/efi is not even mounted. That looks like something to look for.
 
OP
C

cluepon

New Member

Reaction score: 1
Messages: 9

I got it to work. In /usr/libexec/bsdinstall/script, near line 113 I added a bsdinstall mount after bsdinstall zfsboot, and near line 167 I added a bsdinstall umount before zpool export.

I need to add a if (zfsboottype="UEFI" or "BIOS+UEFI") around those commands, but that should be the fix
 

Alain De Vos

Daemon

Reaction score: 613
Messages: 2,071

The installer does very little.
An alternative what I did was create a script that given a partition create a zpool, create a zfs filesystem fetch the txz's from internet and extracts them, copies an fstab rc.conf and loader.conf.
 
OP
C

cluepon

New Member

Reaction score: 1
Messages: 9

The installer does very little.
An alternative what I did was create a script that given a partition create a zpool, create a zfs filesystem fetch the txz's from internet and extracts them, copies an fstab rc.conf and loader.conf.
It used to do exactly what I wanted it to do :) for now it seems I can restore it to that state.
There are a gazillion ways to do unattended installs I guess (I've read aboud mfsbsd, bsdinstall+mfsbsd, there are probably more)
 
Top