FreeBSD for ESPRESSOBin Marvell Armada 3700?

KDragon75

KDragon75

Hello everyone, I'm quite green with the ARM and SBC scene and need some help getting started. I recently got an ESPRESSObin as the title may suggest. In retrospect, this is not the best supported board out there. I do know it will happily run FreeBSD as the pfSene project/Netgate is using in in there production SG-1100.

I need to know more about building the images and if the FreeBSD is statically linked to the load address (Is that the correct terminology?). I would love to find a guide that not only walks me through the process of building a full system image but also explains the entire process and why each step is what it is.

I know this is a bit open ended and I may be missing a lot of basics here but I appreciate any and all help and links.
 
I'm not looking to make this into a firewall using prebuilt systems. My goal is simply to boot to FreeBSD in any capacity. I have access to the pfSense images as we have one of the SG-1100 units at work that I get to tinker with. However, I am unable to get even the recovery image to boot. Presently I get stuck with the below messages. I also get the same when using the aarch64 installer image.
Code: 
>> FreeBSD EFI boot block
   Loader path: /boot/loader.efi

   Initializing modules: UFS
   Probing 1 block devices.... done
    UFS found no partitions
Failed to load '/boot/loader.efi'
panic: No bootable partitions found!
## Application terminated, r = -22

So It looks like bootaa64.efi is running but for whatever reason, but for whatever reason it will not find /boot/loader.efi. Does bootaa64.efi contain the UFS module or is this still depending on the uboot environment? Everything in uboot is set to load form FAT partitions.

slice 1 (FAT)
/efi/bootaa64.efi
/armada-3700.dtb
slice 2 (UFS)
/boot/loader.efi
/the reset of the system

Again, I am new to SBCs and UEFI for that mattor. This is intended as a learning exercise more than anything.
 
Now that I have had some time to look into this, I see /efi/bootaa64.efi is what's running. I have this all on the sdcard at this point with the same layout as before but it still does not see the UFS partition. Does bootaa64.efi have it's own UFS driver or does this still depend on das uboot for file system access? If the latter is the case, I need to re build uboot with ufs or preferably ZFS.
 
Everything I'm reading would suggest this is setup correctly and should at least get to the loader.efi but it's not. Does anyone have any ideas?
 
Hi KDragon! Have you made any progress on this? I'm trying to get there too, with a goal of eventually getting OPNSense running on top of it, but I'm new to the Espressobin.
 
KDragon75 said:
Code: 
>>....
Failed to load '/boot/loader.efi'
..
Click to expand...

This message is familiar to me from Rock64 ....
don`t know the details of the EspressoBin but the Rock64 then only worked on Netboot / no SD card support :
forums.freebsd.org

Rock64

Just came across the Rock64 Anyone tried installing FreeBSD on one of these? Seems to be a lot better than an RPi3 for about the same money...
forums.freebsd.org forums.freebsd.org
 
Hi there. I just ordered one of this boards. Do you have any update yet? As soon mine arrived I’ll try to install FreeBSD and post my result
 
KDragon75 said:
Now that I have had some time to look into this, I see /efi/bootaa64.efi is what's running. I have this all on the sdcard at this point with the same layout as before but it still does not see the UFS partition. Does bootaa64.efi have it's own UFS driver or does this still depend on das uboot for file system access? If the latter is the case, I need to re build uboot with ufs or preferably ZFS.
Click to expand...

Minimal implementations of the UFS and ZFS drivers for the EFI bootloader are in /usr/src/stand/efi/boot1/, with some pieces in /usr/src/stand/libsa/. These are part of the ingredients to make bootaa64.efi.
How are you building u-boot? Did you build it in the way that EspressoBin recommends?

Can you enumerate the partitions on the disk yourself at the U-Boot prompt, before you attempt to boot FreeBSD?
 
RobCrowston said:
Minimal implementations of the UFS and ZFS drivers for the EFI bootloader are in /usr/src/stand/efi/boot1/, with some pieces in /usr/src/stand/libsa/. These are part of the ingredients to make bootaa64.efi.
How are you building u-boot? Did you build it in the way that EspressoBin recommends?

Can you enumerate the partitions on the disk yourself at the U-Boot prompt, before you attempt to boot FreeBSD?
Click to expand...
I have not built uboot from scratch. This getting a bit out of my depth.
 
Some experiences with the EspressoBIN and FreeBSD. First of all the uBoot needs to be updated. The Armbian project has some preconfigured available.

In v5 the Topaz switch is configurable from uBoot. It works as a switch. First port index '0' is connected to the CPU, indexes 1 to 3 are the visible RJ45 ports and the last ones are management ports to be configured as a serial management port if read from the available document. Is there a serial management tool to get and set register addresses of the switch chip of the board from the FreeBSD? It looks like it is a MII interface ('SNI' mentioned).

There is no documentation available from Marvell to use the network chip unless you are a corporate user. The only document found was "Marvell Link Street 88E6060". It is older and similar. In uBoot and Linux source code the same code is used for both of the models. The document describes memory addresses and bits in them to configure the switch. There is a remark of restricted material to be accessed with NDA and a placeholder of information not shown.

There is a 'switch' command in uboot. For example switch write 0 0x04 0x7E disables the forwarding of the packets from the CPU interface, the mvneta0. There is no setting available to set the port to be used separately. It could be the isolation setting not available in 6060. This v5 is not a community router board. And not to mention, the board is first a switch before it is configured. It looks like the v5 is just a switch.

It looks like the instructions to use the board are secret and closed from the community users. EspressoBIN v5 is a good NAS or similar device. It should not be considered as a community card based firewall or a router.

The release of the community card should help in developing the commercial products. Maby some kind of a community support rating should be given by the communities with explanations to recognize these issues before anyone uses time buying the card and using the time to configure it. For example the Netgate pfSense (as a commercial product) looks like a good product to buy to be used as a firewall and a router. It uses the same board. How is it different? There is no sense in the community card v5.

Ok. And as a remark, the SCSI SATA/PATA -disk is available only from the uBoot in FreeBSD version 12.1. The FreeBSD does not give access with the current device tree file (if it has anything to do with it) or with the added 'ada' support compiled in the kernel.
 
I own a V7 and if I got it right the things which differentiates from v5 to v7 are small things like RAM size, CPU speed and some details like JTAG etc.

Why did you have to update the uBoot? My v7 booted slightly adjusted 13 HEAD with the shipped uBoot. What I didn't figure out is, how to boot the kernel automatically, I always need to issue the following. Does your ESPRESSObin auto boots after you updated uBoot?
Code: 
set currdev=disk0p2
boot

I think the switch is not different from v5 to v7. I can change the ports configuration of the switch with ETHERSWITCHCFG(8). What I did is, configure VLANs sub-interfaces on the mvneta0 and than assign the ports to the VLANS. I'm planing to switch my router with the ESPRESSObin in the upcoming days
 
The installer made the correct settings. It boots directly to login -prompt. It created /dev/da0p1 and /dev/da0s2 partitions. Note the 'p' and 's' the installer gave. Additionally I had to copyboot1.efi as a first file in the FAT partition as boot/efi/BOOTaa64.efi. As a second file the startup.nsh. In UEFI the order is first with a correct filename, if not found, it boots from the first file in the disk. Instructions in URL uses loader.efi instead. The used efi -file here may be from the installer.

It may be possible to apply /boot.config:
Code: 
0:da(0,2,)/boot/loader
in it. The man page man -S8 boot and man -S8 boot.config.

Finally in partitioning only GPT partition table works as mentioned in the installer. MSB won't work. There are only slice-numbers. The 'a' -partition is missing from the installer installed system. If read from the man -page, the boot.config is read from the 'a' -partition. It could be it is not found. I have not tried adding a bsdlabel in /dev/da0s2, bsdlabel -w /dev/da0s2.

I will compile the etherswitch -tool and try it :).
 
Did I mention the device tree? It had to be copied from the obj -directory. I'm using
Code: 
setenv fdt_name /dtb/marvell/armada-3720-espressobin.dtb
setenv bootcmd 'mmc rescan; usb start; scsi reset; fatload mmc 0:1 $kernel_addr $image_name; fatload mmc 0:1 $fdt_addr $fdt_name; bootefi $kernel_addr $fdt_addr'
saveenv
from the uboot. What everything is found using it would be good to know. What are the correct memory ranges would be good to know.
 
I still can't get my had around with autoboot. What I figured out so far is, that I actually created the the partition as da0s2. I can also start manually with the following command set bootargs "rootdev=disk0s2";mmc dev 0; fatload mmc 0:1 $kernel_addr $image_name;fatload mmc 0:1 $fdt_addr $fdt_name; bootefi $kernel_addr $fdt_addr

It doesn't make a difference if I write disk0s2 or disk0p2, it boots. Wit the command you posted, which is similar to the one I had before I get the following:


## Starting EFI application at 07000000 ...
WARNING: using memory device/image path, this may confuse some payloads!
Scanning disk sdhci@d0000.blk...
Card did not respond to voltage select!
Scanning disk sdhci@d8000.blk...
Disk sdhci@d8000.blk not ready
Found 3 disks
Consoles: EFI console
Reading loader env vars from /efi/freebsd/loader.env
Setting currdev to disk0:


You mentioned the installer. How did you set up your ESPRESSObin? Do you also use FreeBSD HEAD?
 
tscho said:
can't get my had around with autoboot.
Click to expand...
I've checked this. The BOOTaa64.efi was from the 12.1-RELEASE installation media. It had the same partitions. The currdev or rootfs is usually changed from the loader.conf. I had tried the same with the same results.
 
tscho said:
I still can't get my had around with autoboot. What I figured out so far is, that I actually created the the partition as da0s2. I can also start manually with the following command set bootargs "rootdev=disk0s2";mmc dev 0; fatload mmc 0:1 $kernel_addr $image_name;fatload mmc 0:1 $fdt_addr $fdt_name; bootefi $kernel_addr $fdt_addr

It doesn't make a difference if I write disk0s2 or disk0p2, it boots. Wit the command you posted, which is similar to the one I had before I get the following:


## Starting EFI application at 07000000 ...
WARNING: using memory device/image path, this may confuse some payloads!
Scanning disk sdhci@d0000.blk...
Card did not respond to voltage select!
Scanning disk sdhci@d8000.blk...
Disk sdhci@d8000.blk not ready
Found 3 disks
Consoles: EFI console
Reading loader env vars from /efi/freebsd/loader.env
Setting currdev to disk0:


You mentioned the installer. How did you set up your ESPRESSObin? Do you also use FreeBSD HEAD?
Click to expand...

Did you manage to fix your autoboot issue? I have the same problem. I also tried to change currdev/rootdev in loader.conf but it seems that it is ignored.
 
I hacked on this for about a week, and I finally got FreeBSD 12.1-stable working. Here are some notes:

- You need to use EFI. EFI is the only way to boot BSD on arm64.
- The u-boot that's shipped (u-boot-2017.03-armada-17.10) does not have full EFI support. The earliest version that has EFI support is u-boot-2018.03. But when I tried to compile that it wouldn't boot. I ended up manually backporting the efiloader changes from 2018.03 to the u-boot-2017.03-armada-17.10 branch and got something that works. And you do need a Linux machine to build this.
- You really need to run stable/12, which has the dts files for this board, and the kernel support to parse the dtb. If you care about controlling the on-board switch chip, then you need a couple of more changes from master.
- Even on CURRENT, the mvneta (nic) driver has a bug which can corrupt packets. There is a small patch to fix this.

I have git trees for all of the above. I'll throw them up on github and post links here.
 
As promised, my github trees:

github.com

GitHub - iucoen/freebsd-espressobin: FreeBSD src tree (read-only mirror)

FreeBSD src tree (read-only mirror). Contribute to iucoen/freebsd-espressobin development by creating an account on GitHub.
github.com github.com
github.com

GitHub - iucoen/u-boot-marvell: Marvell Armada U-Boot

Marvell Armada U-Boot. Contribute to iucoen/u-boot-marvell development by creating an account on GitHub.
github.com github.com

For building firmware image, follow the directions here exactly:

ESPRESSObin Wiki | Build From Source - Bootloader

ESPRESSObin Wiki
wiki.espressobin.net wiki.espressobin.net

My u-boot-tree already has the v7 patches applied. For the 2 other firmware trees, download the patches and apply them yourself. Also, make sure you use the exact version of Linaro toolchain as prescribed. I first tried to build with the default arm cross compiler toolchain installed on Ubuntu, and the resulting binary would not boot.

To create a boot image, follow the general EFI boot scheme. Partition the SD card with GPT, create a small FAT partition for EFI, and copy over /boot/boot1.efi from rootfs to /efi/boot/bootaa64.efi on the EFI partition. Also place the the dtb file in the same partition, and tell u-boot to load both, and run bootefi. The DTB file will then be passed from EFI to the kernel by loader. Make sure you use the dtb that's built from the FreeBSD source tree if you want the networking and switch to work.
 
Last edited:
iucoen said:
As promised, my github trees:

github.com

GitHub - iucoen/freebsd-espressobin: FreeBSD src tree (read-only mirror)

FreeBSD src tree (read-only mirror). Contribute to iucoen/freebsd-espressobin development by creating an account on GitHub.
github.com github.com
github.com

GitHub - iucoen/u-boot-marvell: Marvell Armada U-Boot

Marvell Armada U-Boot. Contribute to iucoen/u-boot-marvell development by creating an account on GitHub.
github.com github.com

For building firmware image, follow the directions here exactly:

ESPRESSObin Wiki | Build From Source - Bootloader

ESPRESSObin Wiki
wiki.espressobin.net wiki.espressobin.net

My u-boot-tree already has the v7 patches applied. For the 2 other firmware trees, download the patches and apply them yourself. Also, make sure you use the exact version of Linaro toolchain as prescribed. I first tried to build with the default arm cross compiler toolchain installed on Ubuntu, and the resulting binary would not boot.

To create a boot image, follow the general EFI boot scheme. Partition the SD card with GPT, create a small FAT partition for EFI, and copy over /efi/boot/bootaa64.efi. Also place the the dtb file in the same partition, and tell u-boot to load both, and run bootefi. The DTB file will then be passed from EFI to the kernel by loader. Make sure you use the dtb that's built from the FreeBSD source tree if you want the networking and switch to work.
Click to expand...

Sadly I am unable to get your u-boot to work. My espressobin starts to boot and then freezes at the DLL TUNING part. Has anyone had a similar issue?

Code: 
TIM-1.0
WTMI-armada-17.10.5-32a03bb
WTMI: system early-init

DDR topology parameters:
========================
ddr type               DDR4
ddr speedbin           10
bus width              16-bits
cs num                 1
  cs[0] - group num    0
  cs[0] - bank num     8
  cs[0] - capacity     1024MiB
SVC REV: 5, CPU VDD voltage: 1.108V

DRAM windows:
=============
WIN[0] - base addr     0x60000000
WIN[0] - size          0x40000000

memory test region:
===================
CS[0]                  0x60000000 - 0x9fffffff

Fill memory before self refresh...done
Now in Self-refresh Mode
Exited self-refresh ...
Self refresh Pass.
DDR self test mode test done!!
Vref read training
===================
Final vdac_value 0x00000022
Vref write training
===================
Final vref_value 0x0000001F

DLL TUNING
==============
   DLL 0xc0001050[21:16]: [7,30,1b]
   DLL 0xc0001050[29:24]: [9,30,1c]
   DLL 0xc0001054[21:16]: [b,32,1e]
   DLL 0xc0001054[29:24]: [d,30,1e]
   DLL 0xc0001074[21:16]: [0,3f,1f]
   DLL 0xc0001074
 
I am quite new to compiling freebsd from source and found it extremely difficult to get the right information, so I want to share how it worked for me.

I compiled inside a virtual machine based on FreeBSD 12.1 stable and logged in as root user. Then i installed bash and exported the build environment variable

Code: 
pkg install bash
bash
mkdir /root/obj
export MAKEOBJDIRPREFIX=/root/obj

Then i cloned iucoen freebsd version and compiled it (u can use "make -jX" where X is the amount of compiling processes to speed things up). I had to specify the target architecture because my vm runs on amd64.
Code: 
git clone https://github.com/iucoen/freebsd-espressobin
cd freebsd-espressobin
make TARGET=arm64 TARGET_ARCH=aarch64 buildworld
make TARGET=arm64 TARGET_ARCH=aarch64 buildkernel

After compiling has finished I had to create the dtb file for the espressobin which will be located in
"/root/obj/root/freebsd-espressobin/arm64.aarch64/sys/GENERIC/armada-3720-espressobin.dtb"
Code: 
make TARGET=arm64 TARGET_ARCH=aarch64 builddtb FDT_DTS_FILE=/root/freebsd-espressobin/sys/gnu/dts/arm64/marvell/armada-3720-espressobin.dts

Next up I prepared the SD card (make sure you pick the right path, for me it was "/dev/da1") and installed the source.
Code: 
gpart destroy -F /dev/da1
gpart create -s gpt /dev/da1
gpart add -t efi -s 40m /dev/da1
gpart add -t freebsd /dev/da1
newfs_msdos -F 32 -c 1 /dev/da1p1
newfs -U -L FreeBSD /dev/da1s2
mount /dev/da1s2 /mnt
make TARGET=arm64 TARGET_ARCH=aarch64 DESTDIR=/mnt installkernel installworld distribution
cp /mnt/boot/boot1.efi /root/bootaa64.efi
umount /mnt

Now I copied the dtb file and efi over to the fat partition.

Code: 
mount -t msdosfs /dev/da1p1 /mnt
mkdir -p /mnt/EFI/BOOT
cp /root/bootaa64.efi /mnt/EFI/BOOT/
cp /root/obj/root/freebsd-espressobin/arm64.aarch64/sys/GENERIC/armada-3720-espressobin.dtb /mnt/EFI/BOOT/
umount /mnt

Now my only problem left is u-boot. I am able to boot the image with the original u-boot (u-boot-2017.03-armada-17.10) but there is the autoboot problem. I also tried a precompiled u-boot image from armbian which seems to be
u-boot-2018.03-devel-18.12.3 but with this u-boot version I am not able at all to boot into the os. I am getting the erros below:
Code: 
FreeBSD/arm64 EFI loader, Revision 1.1
(Sat Dec 14 19:58:45 UTC 2019 tsgan@nanopc-t4)

   Command line arguments: loader.efi console=ttyMV0,115200 earlycon=ar3700_uart,0xd0012000 root=/dev/nfs rw ip=0.0.0.0:0.0.0.0:10.4.50.254:255.255.255.0:marvell:eth0:none nfsroot=0.0.0.0:/srv/nfs/,tcp,v3 pci=pcie_bus_safe
   Image base: 0x7000000
   EFI version: 2.70
   EFI Firmware: Das U-Boot (rev 0.00)
   Console: efi (0x1000)
   Load Path: /MemoryMapped(0x1,0x7000000,0x70a41a0)
Trying ESP: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(0)/SD(0)
Setting currdev to disk0:
Setting currdev to net0:
net0: cannot set rx. filters (status=3)
bootp: no reply
No response for RARP request
net_open: RARP failed
net0: cannot set rx. filters (status=3)
bootp: no reply
No response for RARP request
net_open: RARP failed
net0: cannot set rx. filters (status=3)
bootp: no reply
No response for RARP request
net_open: RARP failed
Startup error in /boot/lua/loader.lua:
LUA ERROR: cannot open /boot/lua/loader.lua: input/output error.
 
Last edited:
napyk said:
Sadly I am unable to get your u-boot to work. My espressobin starts to boot and then freezes at the DLL TUNING part. Has anyone had a similar issue?
Click to expand...

Yes. That's because you are not using the exact version of Linaro toolchain. Use 5.2-2015.11-2. Download both the 32bit and 64bit versions.
 
napyk said:
Code: 
cp /mnt/boot/loader.efi /root/bootaa64.efi
Click to expand...

You copied the wrong efi file. You should copy /boot/boot1.efi to bootaa64.efi. boot1.efi then loads loader.efi, and it tells loader.efi which partition it's loaded from, so loader.efi can then load the kernel.
 
You must log in or register to reply here.
Back
Top