Solved Pandaboard ES Rev B3

[ComradeSlice]

I have a Pandaboard ES Rev B3, using sysutils/u-boot-pandaboard. This is how I use u-boot on my board:

dd if=MLO of=/dev/da3 count=1 seek=1 bs=128k conv=sync
dd if=u-boot.img of=/dev/da3 count=2 seek=1 bs=384k conv=sync

I know the standard way is to make a Fat32 partition but I have had the best luck with this. I created a zpool on part 1 with version 28 (because u-boot doesn't support SPA 5000, meaning a pool with feature flags enabled). This is the output I get from u-boot:

=> zfsls mmc 0:1 /zarm/@/
ZFS Pool GUID: 15026369511490358064 (d088637524248b30) Label: GUID: 911316744154507322 (0ca5a5c01d31e83a), txg: 97, SPA v28, ashift: 9
ZFS Pool GUID: 15026369511490358064 (d088637524248b30) Label: GUID: 911316744154507322 (0ca5a5c01d31e83a), txg: 97, SPA v28, ashift: 9
ZFS Pool GUID: 15026369511490358064 (d088637524248b30) Label: GUID: 911316744154507322 (0ca5a5c01d31e83a), txg: 97, SPA v28, ashift: 9
zfs No good uberblocks found in label 2
ZFS Pool GUID: 15026369511490358064 (d088637524248b30) Label: GUID: 911316744154507322 (0ca5a5c01d31e83a), txg: 97, SPA v28, ashift: 9
zfs No good uberblocks found in label 3
zfs fsname = '/zarm/' snapname='<NULL>' filename = '/'
couldn't find 'zarm'

Before I try re-inventing the wheel (i.e. patching u-boot's zfs) I was hoping someone here had gone though all this before me. Does anyone here have any experience with Pandaboard Rev B3? Did you use the stock kernel config PANDABOARD? Did you have to modify the device trees? Did you use the sysutils/u-boot-pandaboard port? What filesystem did you have the best results with? Does it annoy you that sysutils/u-boot-master depends on gcc and its draconian license? Any issues with SDRAM timings?

 

[SirDice]

There's no ZFS support on FreeBSD-ARM. Not yet at least.

 

[ComradeSlice]

Well I guess it's a good thing I didn't try patching u-boot with GRUB's zfs implementation. That was my next idea. u-boot's ZFS is a modified version of a very old GRUB implementation (scandalous!). I found a u-boot patch to add UFS support because the stock version does not support it. Does FreeBSD on ARM support booting from UFS? I really don't want to have to use TFTP.

Thanks for your help!

 

[ComradeSlice]

There's no ZFS support on FreeBSD-ARM. Not yet at least.

You saved me a lot of time! It did not even occur to me that ZFS would be missing from a platform, considering how much work has gone into ZFS on FreeBSD over the years.

I did a lot of digging and now have a boot process that functions, though I am not happy with it. I have a FAT32 partition with kernel.bin and a UFS partition for root. MBR scheme. U-boot is dd'd directly onto the SD card in a 1 MB gap I left before the first partition. If anybody else needs to get Pandaboard ES to work I'll post my scripts and configs. It takes a good amount of tinkering and patching.

Even the latest version of Das u-boot is incapable of booting ZFS with feature flags (SPA 5000 in most implementations outside of Oracle). It's hard coded for version 28 and doesn't support feature flags at all. u-boot's implementation is based on GRUB but now that GRUB's licensed under GPL 3 I don't know what they're going to do about it. No point implementing RootOnZFS if the bootloader won't function. I'm going to look into the feasibility of patching u-boot with a GPL-free implementation of ZFS.

 

[SirDice]

You saved me a lot of time! It did not even occur to me that ZFS would be missing from a platform, considering how much work has gone into ZFS on FreeBSD over the years.

Yeah, it was a bit of surprise for me too. I really wanted to build a cheap, low-power, ARM based machine to use as a fileserver. But alas, not happening.

 

[ComradeSlice]

Here are my scripts if anyone digs this up in the future. I hope this can be of use.

1. Copy /usr/src/sys/arm/conf/PANDABOARD to PANDABOARD-ES
2. Change the FDT_DTS_FILE makeoption to "pandaboard-es.dts". This is necessary because the PANDABOARD-ES config uses a static device tree.
3. Add this line: options ROOTDEVNAME=\"ufs:mmcsd0s2\" this tells the kernel loader to use the second slice of your SD card for the root device.
4. cd to /usr/ports/sysutils/u-boot-pandaboard and modify the Makefile's BOARD_CONFIG variable to omap4_panda_defconfig (I don't think this is completely necessary but this is how u-boot is supposed to be fed the board definitions)
5. Apply the patch for omap4_panda.h (below).
6. make and install the port: make install; make clean;
7. Run the scripts to compile the kernel and world (below).
8. Run the script to format the SD card and inject u-boot (below).
9. Mount the FAT slice to /mnt/boot and the UFS slice to /mnt/rootfs.
10. Run the script to install the distribution (below).
11. Copy kernel.bin from ${MAKEOBJDIRPREFIX}/arm.armv6/usr/src/sys/PANDABOARD-ES/kernel.bin to /mnt/boot. (see reply for more info on this)
12. Create /mnt/rootfs/etc/fstab and put this in it: /dev/mmcsd0s2 / ufs rw 1 1
13. Create /mnt/rootfs/etc/rc.conf and put this in it:

    hostname="arm"
    ifconfig_ue0="DHCP link XX:XX:XX:XX:XX:XX"
    sshd_enable="YES"

    Note the "link" argument to ifconfig. The NIC does not have a burned-in MAC address so you have to spoof one to be compliant with the Ethernet protocol. I forgot where I got mine from but I'm sure there's websites that can help you generate a unique MAC.
14. Done!

Here are the scripts I made. You'll have to modify some variables. For all the build commands I use hw.ncpu * 1.5. If you have super fast SSD's use hw.ncpu * 1 (take out * 3 / 2)

Kernel build. Modify KERNLOG and MAKEOBJDIRPREFIX.

~yourusername/panda/kernel:

#!/usr/bin/env bash

THREADS=$((`sysctl -n hw.ncpu` * 3 / 2))
KERNLOG="$(realpath ~yourusername)/panda/kernel.log"

pushd /usr/src

export MAKEOBJDIRPREFIX=$(realpath ~yourusername)/obj
make -j${THREADS} -DNO_CLEAN __MAKE_CONF=/dev/null SRCCONF=/dev/null TARGET_ARCH=armv6 KERNCONF=PANDABOARD-ES buildkernel >> ${KERNLOG}

popd

World build. Modify WORLDLOG and MAKEOBJDIRPREFIX. Note: when you build world for ARM a UBLDR_LOADADDR must be specified. I use 0x88000000.

~yourusername/panda/world:

#!/usr/bin/env bash

THREADS=$((`sysctl -n hw.ncpu` * 3 / 2))
UBOOTADDR=0x88000000
WORLDLOG="$(realpath ~yourusername)/panda/world.log"

pushd /usr/src

export MAKEOBJDIRPREFIX=$(realpath ~yourusername)/obj
make -j${THREADS} -DNO_CLEAN __MAKE_CONF=/dev/null SRCCONF=/dev/null TARGET_ARCH=armv6 UBLDR_LOADADDR=${UBOOTADDR} buildworld >> ${WORLDLOG}

popd

Distribution installation. Modify DISTLOG and MAKEOBJDIRPREFIX.

~yourusername/panda/dist:

#!/usr/bin/env bash

THREADS=$((`sysctl -n hw.ncpu` * 3 / 2))
DESTDIR=/mnt/rootfs
DISTLOG="$(realpath ~yourusername)/panda/dist.log"

pushd /usr/src

export MAKEOBJDIRPREFIX=$(realpath ~yourusername)/obj
make -j${THREADS} -DNO_CLEAN __MAKE_CONF=/dev/null SRCCONF=/dev/null TARGET_ARCH=armv6 DESTDIR=${DESTDIR} installworld >> ${DISTLOG}
make -j${THREADS} -DNO_CLEAN __MAKE_CONF=/dev/null SRCCONF=/dev/null TARGET_ARCH=armv6 DESTDIR=${DESTDIR} distribution >> ${DISTLOG}
make -j${THREADS} -DNO_CLEAN __MAKE_CONF=/dev/null SRCCONF=/dev/null TARGET_ARCH=armv6 KERNCONF=PANDABOARD-ES DESTDIR=${DESTDIR} installkernel >> ${DISTLOG}

popd

sd formatter - modify DISK to match your sd card device. You shouldn't have to modify BOOTDEV or ROOTDEV. Because I'm lazy I used manual block sizes for the "gpart add" commands for my specific SD card. Leave the FAT command the same and modify the FreeBSD command for your own SD card.

~yourusername/panda/sd:

#!/usr/bin/env bash

DISK=/dev/da3
BOOTDEV=${DISK}s1
ROOTDEV=${DISK}s2
UBOOTROOT=/usr/local/share/u-boot/u-boot-pandaboard

gpart destroy -F ${DISK} 2>/dev/null

dd if=/dev/zero of=${DISK} count=10 bs=1M conv=sync

gpart create -s MBR ${DISK}
gpart add -t \!14    -b 2048 -s 131072     ${DISK}
gpart add -t freebsd -b 133120 -s 62388224 ${DISK}
# Total: 62,521,344

dd if=${UBOOTROOT}/MLO of=${DISK} count=1 seek=1 bs=128k conv=sync
dd if=${UBOOTROOT}/u-boot.img of=${DISK} count=2 seek=1 bs=384k conv=sync

# JAS: Would have been nice...
#zpool create -f -o version=28 zarm ${ROOTDEV}
newfs_msdos -F 16 ${BOOTDEV}
newfs ${ROOTDEV}

u-boot-pandaboard port patch (see reply for more info)

/usr/ports/sysutils/u-boot-master/files/patch-include-configs-omap4_panda.h:

--- include/configs/omap4_panda.h.orig  2017-07-27 01:40:37.000000000 -0400
+++ include/configs/omap4_panda.h   2017-08-30 07:32:26.176722000 -0400
@@ -16,6 +16,14 @@
  * High Level Configuration Options
  */

+/*#define DEBUG*/
+
+/* UFS */
+/*#define CONFIG_CMD_FFS*/
+
+/* ZFS */
+/*#define CONFIG_CMD_ZFS*/
+
 /* USB UHH support options */
 #define CONFIG_SYS_USB_EHCI_MAX_ROOT_PORTS 3

@@ -41,4 +49,15 @@
 #define FAT_ENV_FILE                    "uboot.env"
 #define CONFIG_ENV_OVERWRITE

+#ifdef CONFIG_BOOTCOMMAND
+#undef CONFIG_BOOTCOMMAND
+#endif
+
+#define CONFIG_BOOTCOMMAND \
+   "if mmc rescan; then " \
+       "echo SD/MMC found on device ${mmc_dev};" \
+       "fatload mmc 0 0x80200000 kernel.bin;" \
+       "go 0x80200000;" \
+   "fi;" \
+
 #endif /* __CONFIG_PANDA_H */

 

[ComradeSlice]

I've since revisited the process I used to create this image and have improved it. Booting to kernel.bin is unnecessary because 1) FreeBSD has a u-boot bootloader and 2) u-boot can read elf headers. I needed to boot into single user mode to debug some issues and found nextboot was not working because loading kernel.bin into memory and jumping to it bypasses the loader. This boot command modification to patch-include-configs-omap4_panda.h will allow you to use ubldr, which gives you the standard FreeBSD loader prompt from which you can interrupt the boot process and load the kernel with whatever arguments you want.

+#define CONFIG_BOOTCOMMAND \
+   "if mmc rescan; then " \
+       "echo SD/MMC found on device ${mmc_dev};" \
+       "fatload mmc 0 0x88000000 ubldr;" \
+       "bootelf 0x88000000;" \
+   "fi;" \

Using this method the only file that needs to go onto the FAT partition is ubldr, which is the FreeBSD u-boot loader with elf headers.

Note that I used 0x88000000 in the "world" script as the load address for ubldr. When building FreeBSD on ARM (AArch32) you must specify the load address for u-boot or it will default to 0x1000000.