UFS Boot Environments for ARM

vermaden

Son of Beastie

Reaction score: 1,536
Messages: 2,979

Several days ago I introduced UFS Boot Environments that work great on AMD64 (or 64-bit PC if you prefer). I was interested it it will also work on less powerful devices that ZFS is not always the best idea – ARM based devices. After some testing I found out that after one simple modification the UFS Boot Environments work like a charm on ARM devices.

The Table of Contents is as follows.

  • ARM Testing
  • Setup UFS Boot Environments
  • Needed Fix to Make FreeBSD bootme Flags Work
  • Reboot into Other Boot Environment Test

There is not suitable TL;DR here – you will have to read it all or not at all this time
🙂


ARM Testing​


I currently do not own 64-bit ARM device … so I thought I will try the qemu(1) emulator and ready to download and use ARM images provided by the FreeBSD project.

First we will install needed packages and fetch the ARM64 (also known as aarch64) images.

host # pkg install -y qemu u-boot-qemu-arm64

host % fetch https://download.freebsd.org/ftp/re.../Latest/FreeBSD-13.0-RC4-arm64-aarch64.raw.xz

host % xz -d FreeBSD-13.0-RC4-arm64-aarch64.raw.xz



We will now increase the image size to add additional boot environment partition.

host % ls -lh FreeBSD-13.0-RC4-arm64-aarch64.raw
-rw-r--r-- 1 vermaden vermaden 5.1G 2021-04-04 12:37 FreeBSD-13.0-RC4-arm64-aarch64.raw

host % truncate -s +9G FreeBSD-13.0-RC4-arm64-aarch64.raw

host % ls -lh FreeBSD-13.0-RC4-arm64-aarch64.raw
-rw-r--r-- 1 vermaden vermaden 15G 2021-04-04 12:38 FreeBSD-13.0-RC4-arm64-aarch64.raw



Using qemu(1) emulator we can boot using either UEFI or U-BOOT option. We will test both as some ARM devices use UEFI and some (like Raspberry Pi devices) use U-BOOT mode.

host % export VMDISK=FreeBSD-13.0-RC4-arm64-aarch64.raw

// UEFI
host % qemu-system-aarch64 \
-m 4096M \
-cpu cortex-a57 \
-M virt \
-bios edk2-aarch64-code.fd \
-serial telnet::4444,server \
-nographic \
-drive if=none,file=${VMDISK},format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-device virtio-net-device,netdev=net0 \
-netdev user,id=net0


// U-BOOT
host % qemu-system-aarch64 \
-m 4096M \
-cpu cortex-a57 \
-M virt \
-bios /usr/local/share/u-boot/u-boot-qemu-arm64/u-boot.bin \
-serial telnet::4444,server \
-nographic \
-drive if=none,file=${VMDISK},format=raw,id=hd0 \
-device virtio-blk-device,drive=hd0 \
-device virtio-net-device,netdev=net0 \
-netdev user,id=net0


After starting the qemu(1) process it will display the following information.

(...)
QEMU 5.0.1 monitor - type 'help' for more information
(qemu) qemu-system-aarch64: -serial telnet::4444,server: info: QEMU waiting for connection on: disconnected:telnet::::4444,server


We can now use telnet(1) to connect to our serial console on emulated ARM64 system. We will add additional freebsd-ufs partition for our second boot environment.

host % telnet localhost 4444
(...)

login: root

ARM # pkg install -y lsblk

ARM # lsblk
DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT
vtbd0 0:62 14G GPT - -
vtbd0p1 0:63 33M efi gpt/efiesp /boot/efi
vtbd0p2 0:64 1.0G freebsd-swap gpt/swapfs -
vtbd0p3 0:65 4.0G freebsd-ufs ufs/rootfs /

ARM # gpart show
=> 3 10552344 vtbd0 GPT (14G) [CORRUPT]
3 66584 1 efi (33M)
66587 2097152 2 freebsd-swap (1.0G)
2163739 8388608 3 freebsd-ufs (4.0G)

ARM # gpart recover vtbd0
vtbd0 recovered

ARM # gpart add -s 4G -t freebsd-ufs vtbd0
vtbd0p4 added

ARM # gpart show
=> 3 29426709 vtbd0 GPT (14G)
3 66584 1 efi (33M)
66587 2097152 2 freebsd-swap (1.0G)
2163739 8388608 3 freebsd-ufs (4.0G)
10552347 8388608 4 freebsd-ufs (4.0G)
18940955 10485757 - free - (5.0G)



We will now make some manual preparations for ufsbe.sh to work.

For example the FreeBSD images come with GPT labels used in /etc/fstab file which are currently not supported by UFS Boot Environments so we will modify the /etc/fstab file to mount root filesystem from raw devices and partitions.

ARM # mkdir -p /ufsbe/3 /ufsbe/4

ARM # cat /etc/fstab
# Custom /etc/fstab for FreeBSD VM images
/dev/gpt/rootfs / ufs rw 1 1
/dev/gpt/efiesp /boot/efi msdosfs rw 2 2
/dev/gpt/swapfs none swap sw 0 0

ARM # vi /etc/fstab

ARM # cat /etc/fstab
# Custom /etc/fstab for FreeBSD VM images
/dev/vtbd0p3 / ufs rw 1 1
/dev/vtbd0p4 /ufsbe/4 ufs rw 1 1
/dev/gpt/efiesp /boot/efi msdosfs rw 2 2
/dev/gpt/swapfs none swap sw 0 0

ARM # newfs /dev/vtbd0p4
/dev/vtbd0p4: 4096.0MB (8388608 sectors) block size 32768, fragment size 4096
using 7 cylinder groups of 625.22MB, 20007 blks, 80128 inodes.
super-block backups (for fsck_ffs -b #) at:
192, 1280640, 2561088, 3841536, 5121984, 6402432, 7682880


We now have second boot environment ready and /etc/fstab file modified to boot from raw devices instead of GPT labels. We will now reboot(8) to make these changes apply.

ARM # reboot

Setup UFS Boot Environments​


We will not fetch the ufsbe.sh command and finish the setup process.

ARM # fetch https://raw.githubusercontent.com/vermaden/ufsbe/main/ufsbe.sh

ARM # chmod +x ./ufsbe.sh

ARM # ./ufsbe.sh

NOPE: did not found boot environment setup with 'ufsbe' label

INFO: setup each boot environment partition with appropriate label

HELP: list all 'freebsd-ufs' partitions type:

# gpart show -p | grep freebsd-ufs
2098216 33554432 ada0p3 freebsd-ufs [bootme] (16G)
35652648 33554432 ada0p4 freebsd-ufs (16G)
69207080 33554432 ada0p5 freebsd-ufs (16G)

HELP: to setup partitions 3/4/5 as boot environments type:

# gpart modify -i 3 -l ufsbe/3 ada0
# gpart modify -i 4 -l ufsbe/4 ada0
# gpart modify -i 5 -l ufsbe/5 ada0

ARM # gpart show
=> 3 29426709 vtbd0 GPT (14G)
3 66584 1 efi (33M)
66587 2097152 2 freebsd-swap (1.0G)
2163739 8388608 3 freebsd-ufs (4.0G)
10552347 8388608 4 freebsd-ufs (4.0G)
18940955 10485757 - free - (5.0G)

ARM # gpart modify -i 3 -l ufsbe/3 vtbd0
vtbd0p3 modified

ARM # gpart modify -i 4 -l ufsbe/4 vtbd0
vtbd0p4 modified

ARM # ./ufsbe.sh
INFO: flag 'bootme' successfully set on / filesystem
usage:
ufsbe.sh (l)ist
ufsbe.sh (a)ctivate
ufsbe.sh (s)ync


The UFS Boot Environments are now properly deployed on this ARM64 test system.

Needed Fix to Make FreeBSD bootme Flags Work​


At the first try I was not able to use UFS Boot Environments as the bootme flag was ignored.

I then submitted a FreeBSD bug – 254764 – GPT ‘bootme’ flag is not respected on AARCH64 – to make sure I am doing everything well on my side. As it turns out the bootme flag is a FreeBSD specific extension and nobody else uses it. The needed fix is to copy /boot/gptboot.efi in place of bootaa64.efi file.

Lets now make that fix.

ARM # cp /boot/gptboot.efi /boot/efi/EFI/BOOT/bootaa64.efi

Reboot into Other Boot Environment Test​


We will now synchronize boot environments 3 and 4 and then reboot into the 4 boot environments.

ARM # ./ufsbe.sh list
PROVIDER LABEL ACTIVE
vtbd0p3 ufsbe/3 NR
vtbd0p4 ufsbe/4 -

ARM # ./ufsbe.sh sync 3 4
INFO: syncing '3' (source) => '4' (target) boot environments ...
INFO: boot environments '3' (source) => '4' (target) synced

ARM # ./ufsbe.sh activate 4
INFO: boot environment '4' now activated

ARM # reboot


After the reboot the currently active boot environment is 4. It means that UFS Boot Environments work properly on ARM devices.

ARM # ./ufsbe.sh list
PROVIDER LABEL ACTIVE
vtbd0p3 ufsbe/3 -
vtbd0p4 ufsbe/4 NR

ARM # df -h
Filesystem Size Used Avail Capacity Mounted on
/dev/vtbd0p4 3.9G 2.6G 935M 74% /
devfs 1.0K 1.0K 0B 100% /dev
/dev/vtbd0p3 3.9G 2.6G 932M 74% /ufsbe/3
/dev/gpt/efiesp 32M 1.3M 31M 4% /boot/efi



I have tested both U-BOOT and UEFI boot modes and they both allow to use UFS Boot Environments.

EOF

Continue reading...
 
Top