Running FreeBSD on Radxa Rock 3C RK3566 board

Hi.
I've been playing around for the last few days to boot FreeBSD on this SBC. In the end, the process was quite easy, but getting to that point required some learning :)
Maybe this will help someone

1. Get FreeBSD base image, I've used
Code:
FreeBSD-14.0-CURRENT-arm64-aarch64-PINE64-20230608-653738e895ba-263444.img
and dd it into SD

2. Get latest u-boot for this board (use Linux box or WSL)
Code:
git clone https://github.com/radxa-repo/bsp
cd bsp
./bsp u-boot latest
...
Two files you want from ".root/usr/lib/u-boot/rock-3c" are "idbloader.img" and "u-boot.itb"

3. Overwrite u-boot, either on image or SD card, then write image to SD card (adjust freebsd.img to image of your choice or SD directly)
Code:
dd if=idbloader.img of=freebsd.img seek=64 conv=notrunc
dd if=u-boot.itb of=freebsd.img seek=16384 conv=notrunc

4. Now the fun part, DTS, I've used DTS from https://github.com/radxa/kernel/pull/133/files, clone the repo with those changes, checkout the current branch and you should be able to do something like
Code:
ARCH=arm64 make rockchip_linux_defconfig
ARCH=arm64 make dtbs
find . -name rk3566-rock-3c.dtb
For this step use Linux box or WSL.

5. Now mount the SD card and copy "rk3566-rock-3c.dtb" to /boot/dtb/rockchip/

6. Add those lines to /boot/loader.conf also
Code:
fdt_overlays="rk3566-rock-3c-tsadc"
hw.fdt.console="/serial@fe660000"
first line is adding FDT to disable bogus device
second one is adding serial output while booting kernel

7. Create file /boot/dtb/overlays/rk3566-rock-3c-tsadc.dtso with content:
Code:
/dts-v1/;
/plugin/;

/ {
        compatible = "radxa,rock-3c", "rockchip,rk3566";
};

&{/tsadc@fe710000} {
        compatible = "rockchip,rk3568-tsadc";
        status = "disabled";
};
Compile it with command "dtc -I dts -O dtb -o rk3566-rock-3c-tsadc.dtbo rk3566-rock-3c-tsadc.dtso"
alternative way is to run command "fdt rm /tsadc@fe710000" at loader, before booting kernel - this device is causing board to reset while booting kernel
same goes with "set hw.fdt.console=/serial@fe660000"

8. Insert SD into board, connect serial (speed 1500000) and boot

9. It should boot, blue LED should also light up, there will be no "login console" on serial - haven't looked up for that yet

10. Connect ethernet cable, it should get IP from DHCP server in your network - check your router for IP and ssh into the box (freebsd / freebsd)

11. There are some errors with USB, but one port (the bottom one on the right side) should be working.

Code:
root@generic:~ # uname -a
FreeBSD generic 14.0-CURRENT FreeBSD 14.0-CURRENT #0 main-n263444-653738e895ba: Thu Jun  8 11:08:03 UTC 2023     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64

Hope this helps someone :)
Attached binary files if you just want to boot it up.
 

Attachments

  • rock3c.zip
    563.1 KB · Views: 125
Last edited:
Thanks for the detailed write-up. I had no idea that Rock3C used a different CPU than Rock3A.
Utterly ridiculous. Terrible for porting efforts.
I was wondering why Rock3C with Wifi costs less than Rock3A without Wifi. Maybe RK3568 cost more than RK3566..

For this step use Linux box or WSL.
Can I ask why you used Linux for the Raxda U-boot? Did you try to compile it on FreeBSD?

How about MiniPCIe M.2 slot? Does it work? I see Rock3C has no NVMe on bottom and M.2 on top is for NVMe.
 
As for building u-boot - it's using docker/podman, that's why I've used Linux directly - haven't tried under FreeBSD, but for 99,9% it won't work :)

3C does have M.2 - does it work? Hard to tell as I don't have anything to install there :) Maybe I will try to find and order something and then I can let you know.
 
I have used your u-boot bsp files flashed to microSD card. It booted up on Rock3A but had MMC0 errors.
I was surprised it worked at all. There must be a good deal of similarities between rk3566 and rk3568.
NVMe uboot-module is built but drive not detected. Same with PCI uboot module. No Rockchip PCI bridge found.
 
I would really enjoy diving into EDK2 to add a GPIO module like Beaglebone supplies.
I guess it is a Dxe directory build? I poked at the workings.

Tried out the RPI3 EDK2 port and I like it. Where is the GPIO stuff? Seems a barebones build.

To me Minnowboard EDK2 and the GPIO settings are a deluxe feature.
 
I should have NVMe drive today to test it. In case this would help - there's some probe about the board https://bsd-hardware.info/?probe=8357f0f72e.

edit: got NVMe drive installed, checked it's working under Linux, not visible under FreeBSD - my guesss is that it's the device "tsadc@fe710000" that I have disabled, as it's causing restart during boot, so my guess it's either something wrong with DTS or it's not yet supported in -CURRENT

edit 2: tried DTS from 4.19 tree, same reset issue, need to "fdt rm /tsadc@fe710000" - did hwprobe also here https://bsd-hardware.info/?probe=615e7cbf52
:(
 
Using those DTS files from Linux tree makes the board heat up quickly (even ethernet and USB ports are getting hot) - something must be off :D
Did try to boot with rk3566-quartz64-a.dts from -CURRENT - PCI bus was found, but failed to attach I think
Code:
pcib0: <RockChip RK3568 PCI-express controller> mem 0x3c0000000-0x3c03fffff,0xfe260000-0xfe26ffff,0x33f000000-0x33fffffff irq 28,29,30,31,32 on ofwbus0
rk3568_combphy1: configuring for PCIe ref_clk=25000000
pci0: <OFW PCI bus> on pcib0
pcib1: <PCI-PCI bridge> at device 0.0 on pci0
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial I/O port window: 0-0xfff
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial prefetch window: 0-0xfffff
pci1: <PCI bus> on pcib1

Also ethernet using this DTS is not working (but DTS from B have ethernet working) :)
Also tried all DTS from -CURRENT for RK3566, but none of them had PCIe enabled in DTS - only quartz64-a :(
 
Also tried all DTS from -CURRENT for RK3566, but none of them had PCIe enabled in DTS - only quartz64-a
I see that in alot of boards. The maunfaucurer builds minimal required u-boot for boards to boot.

User is left to compile u-boot and modifying defconfig to add stuff like SATA and PCI
Even Armbian u-boot is missing PCI hence no NVMe on Rock3A images.

the board heat up quickly (even ethernet and USB ports are getting hot)
I found that quickly too.
MMC chip was so hot I couldn't touch for 2 seconds and I am a machinist with very calloused hands.
Hence I got a 102 piece heatsink kit. Outfitted both topside chips. M.2 PCIe slot gets in way of heatsink so I used copper slug and ofsett the heatsink to clear slot.

I didn't notice USB/Ethernet port being above temps.
 
Hi Szczepan

I looked at dmesg and the rest.

[...] rk3568_combphy0: <RockChip combo PHY> mem 0xfe830000-0xfe8300ff on ofwbus0
[...] rk3568_combphy0: Cannot allocate memory resources
[...] device_attach: rk3568_combphy0 attach returned 6


A lot of problems.

The main thing that does not work rk3568_combphy!!!
Without this, USB and PCIe will not work.
The easiest way to start FreeBSD is to create an EDK2 UEFI, even easier than creating a proper DTS and then DTB,
You can move through trial and error for a very long time.
You can write to me in the telegram group (available on my website),

I will try to help.
 
@Phishfry

My Rock-3A has been running for three days without shutting down,
the last reboot was over 20 hours ago. That's what I see.


root@Rock-3A:~ # uptime
5:25PM up 22:34, 1 user, load averages: 0.21, 0.20, 0.17

# sysctl hw.model
hw.model: ARM Cortex-A55 r2p0

# sysctl hw.ncpu
hw.ncpu: 4

# sysctl hw.acpi.thermal.tz0.temperature
hw.acpi.thermal.tz0.temperature: 59.4C

# sysctl hw.acpi.thermal.tz1.temperature
hw.acpi.thermal.tz1.temperature: 58.2C

root@Rock-3A:~ #


You can update UEFI to the latest version, I recommend.
 
SleepWalker
So I did play around with DTS. Got rk3568_combphy0 and 1 to attach. Newest probe - https://bsd-hardware.info/?probe=6b4f214b72 - I know that USB are disabled in DTS. But still no PCIe/NVMe visible, as
Code:
pcib0: <RockChip RK3568 PCI-express controller> mem 0x3c0000000-0x3c03fffff,0xfe260000-0xfe26ffff,0x33f000000-0x33fffffff irq 28,29,30,31,32 on ofwbus0
rk3568_combphy1: configuring for PCIe ref_clk=25000000
pcib0: Bus is not cache-coherent
pcib0: Using iATU unroll mode
pcib0: Detected outbound iATU regions: 8
pci0: <OFW PCI bus> on pcib0
pci0: domain=0, physical bus=0
found-> vendor=0x1d87, dev=0x3566, revid=0x01
        domain=0, bus=0, slot=0, func=0
        class=06-04-00, hdrtype=0x01, mfdev=0
        cmdreg=0x0107, statreg=0x0010, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=a, irq=255
        powerspec 3  supports D0 D1 D2 D3  current D0
        MSI supports 32 messages, 64 bit
        MSI-X supports 1 message in map 0x10
        secbus=1, subbus=255
pcib1: <PCI-PCI bridge> at device 0.0 on pci0
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial I/O port window: 0-0xfff
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial prefetch window: 0-0xfffff
pcib1:   domain            0
pcib1:   secondary bus     1
pcib1:   subordinate bus   255
pcib1:   memory decode     0x0-0xfffff
pci1: <PCI bus> on pcib1
pcib1: allocated bus range (1-1) for rid 0 of pci1
pci1: domain=0, physical bus=1

This looks like my NVMe drive connected in there, I think so :)
I don't think I will go further with DTS, as this is getting more and more magic :)

What's the steps needed to boot in ACPI mode?
What needs to be done to add support for this board?
 
I had to get my hands dirty and bring in rktools to build ROCKCHIP_TPL for ROCK3A.
While doing so I notice this additional USB file that needs bringing in to the TPL

Path1=bin/rk35/rk356x_usbplug_v1.17.bin

I wonder if it fixes the USB port problem...
 
Using newest DTB's from CURRENT, I took rk3566-quartz64-b.dtb and booted with that on mine 3C (also I did compare this DTS with latest available DTS for 3C from Radxa kernel line to check if pcie matches etc.). It boots, even NVMe is detected, but it's not working in the end :(

pcib0: <RockChip RK3568 PCI-express controller> mem 0x3c0000000-0x3c03fffff,0xfe260000-0xfe26ffff,0xf4000000-0xf40fffff irq 34,35,36,37,38 on ofwbus0
pci0: <OFW PCI bus> on pcib0
pcib1: <PCI-PCI bridge> at device 0.0 on pci0
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial I/O port window: 0-0xfff
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial memory window: 0-0xfffff
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial prefetch window: 0-0xfffff
pci1: <PCI bus> on pcib1
nvme0: <Generic NVMe Device> at device 0.0 on pci1

root@generic:~ # nvmecontrol devlist
nvme0: pt->len (4096) exceeds max_xfer_size (0)

Kind of sad, but does this look like it's not supported (yet) by the kernel.
Newest u-boot have boot support from NVME, which works and both SD card and NVMe drive are visible inside FreeBSD loader.efi.

Some more debug info about pci/nvme, maybe someone will be able to help
pcib0: <RockChip RK3568 PCI-express controller> mem 0x3c0000000-0x3c03fffff,0xfe260000-0xfe26ffff,0xf4000000-0xf40fffff irq 34,35,36,37,38 on ofwbus0
rk3568_combphy1: configuring for PCIe ref_clk=25000000
pcib0: Bus is not cache-coherent
pcib0: Using iATU unroll mode
pcib0: Detected outbound iATU regions: 8
pci0: <OFW PCI bus> on pcib0
pci0: domain=0, physical bus=0
found-> vendor=0x1d87, dev=0x3566, revid=0x01
domain=0, bus=0, slot=0, func=0
class=06-04-00, hdrtype=0x01, mfdev=0
cmdreg=0x0107, statreg=0x0010, cachelnsz=0 (dwords)
lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
intpin=a, irq=255
powerspec 3 supports D0 D1 D2 D3 current D0
MSI supports 32 messages, 64 bit
MSI-X supports 1 message in map 0x10
secbus=1, subbus=255
pcib1: <PCI-PCI bridge> at device 0.0 on pci0
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial I/O port window: 0-0xfff
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial memory window: 0-0xfffff
pcib0: failed to reserve resource for pcib1
pcib1: failed to allocate initial prefetch window: 0-0xfffff
pcib1: domain 0
pcib1: secondary bus 1
pcib1: subordinate bus 255
pci1: <PCI bus> on pcib1
pcib1: allocated bus range (1-1) for rid 0 of pci1
pci1: domain=0, physical bus=1
found-> vendor=0x1217, dev=0x8760, revid=0x01
domain=0, bus=1, slot=0, func=0
class=01-08-02, hdrtype=0x00, mfdev=0
cmdreg=0x0000, statreg=0x0010, cachelnsz=0 (dwords)
lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
intpin=a, irq=255
powerspec 3 supports D0 D3 current D0
MSI supports 16 messages, 64 bit, vector masks
MSI-X supports 16 messages in map 0x10
map[10]: type Memory, range 64, base 0, size 14, memory disabled
nvme0: <Generic NVMe Device> at device 0.0 on pci1
pcib1: allocated initial memory window of 0x40000000-0x400fffff
pcib1: allocated memory range (0x40000000-0x40003fff) for rid 10 of nvme0
nvme0: Lazy allocation of 0x4000 bytes rid 0x10 type 3 at 0x40000000
ofw_pci mapdev: start 300000000, len 16384
nvme0: attempting to allocate 5 MSI-X vectors (16 supported)
nvme0: using IRQs 82-86 for MSI-X
nvme0: CapLo: 0xffffffff: MQES 65535, CQR, AMS WRRwUPC VS, TO 255
nvme0: CapHi: 0xffffffff: DSTRD 15, NSSRS, CSS ff, BPS, MPSMIN 15, MPSMAX 15, PMRS, CMBS
nvme0: Version: 0xffffffff: 65535.255
nvme0: PMRCap: 0xffffffff: BIR 7, RDS, WDS, PMRTU 3, PMRWBM f, PMRTO 255, CMSS
 
got an orange pi 3b (rk3566)
trying 14-BETA5 / not too cool as of now
uart, sdcard work
usb does not (driver loads but no device detected, probably something wrong with the phy)
thermal adc/controller causes a reboot if some options are in dtb but can be worked around
pmic causes a panic but it's fixed in 15 so you can replace the file
ethernet driver is half assed so if the ethernet is not init by u-boot it wont work (i probably can fix this)
when it works it's slow

pci seems to attach (i have no device connected)
i have 2 dtbs one from (https://github.com/Kwiboo/u-boot-rockchip.git)
and one from orange-pi official images
pci won't work with the orange-pi dtb but works with the other
dmesg @ https://termbin.com/6v3h
 
Create file /boot/dtb/overlays/rk3566-rock-3c-tsadc.dtso with content:
there is a bug is rk_tsadc.c which i fixed (thermal sensor controller)
C:
/sys/arm64/rockchip/rk_tsadc.c#L486
in the first line should be tsadc_temp_to_raw instead of tsadc_raw_to_temp
the way its set for the alarm value

writing a bogus value causes a reboot when shutdown_mode is 0 (cru)
on rk3566. could cause the same effect on other boards too


    val =  tsadc_raw_to_temp(sc, sc->shutdown_temp);
    WR4(sc, TSADC_COMP_SHUT(sensor->channel), val);
    val = RD4(sc, TSADC_AUTO_CON);
    val |= TSADC_AUTO_SRC_EN(sensor->channel);
    WR4(sc, TSADC_AUTO_CON, val);

    /* Alarm temperature */
    val =  tsadc_temp_to_raw(sc, sc->alarm_temp);
    WR4(sc, TSADC_COMP_INT(sensor->channel), val);
    val = RD4(sc, TSADC_INT_EN);
    val |= TSADC_COMP_INT_SRC_EN(sensor->channel);
    WR4(sc, TSADC_INT_EN, val);
hopefully it will be commited soon
for me it worked if the shut_mode was set to gpio and only rebooted via cru
so you can probably change shutmode to 1 and still have temperature sensors running until the code is fixed
 
Looking at your dmesg, it doesn't look like PCIe is working - there are errors with APB etc., that's because Linux DTS are not 100% usuable on FreeBSD :)
But I've tried teh DTS from Kwiboo (it's 99% similar do Quartz64-B one) - the effects is the same, nvm is detected, but drive is not working - I may be wrong but this looks like something support in kernel is missing. I've submitted a bug to track it https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=274484

Also temp thing looks like it's fixed already in -CURRENT (haven't tried 14-RC yet).
 
i did not attach anything to the pcie as of now, i still have to fix usb, ethernet
i tested an official opi image just to make sure usb works becuase they don't in the official spi-rom u-boot and i taught the board might have a problem
usb works in debian so i have to dig more. i suspect a regulator might have to be gpio enabled but not sure right now
after i fix ethernet and usb ill look at pcie
 
nvme works on the orange pi 3b albeit extremly slow (with the quartz64 dtb fails as yours)
i added support for scmi-smc (smc transport) so now i have cpu clock and cpufreq_dt
when run at 1.8ghz performance seems the same with raspberry pi 4
Code:
scmi0: <ARM SCMI interface driver> on ofw_firmware0
scmi_clk0: <SCMI Clock Management Unit> on scmi0
scmi_clk0: Found 4 clocks.
scmi_clk0: Clock 'clk_scmi_cpu' registered.
scmi_clk0: Clock 'clk_scmi_gpu' registered.
scmi_clk0: Clock 'clk_scmi_npu' registered.
scmi_clk0: Clock 'clk_scmi_ddr' registered.
scmi_clknode_set_freq: 1800000000
 
I haven't tried the DTB from OrangePi - which one did you used that seems to have working nvme with it?
For ethernet and USB - quartz-b seems to have ethernet and 2 or 1 usb ports working for sure last time I checked :)
Do you have your 'working' dts put somewhere
 
here it is
build dtb with
dtc -I dts thisfile.dts.txt >out.dtb

keep in mind that it may cause problems with usb because of the resets for combphy (the usb3 port)
the code does not use reset-names and only uses the first which has to be "combphy" (i fixed combphy driver here)
i suspect the ranges in the pcie node causes the difference but did not test
 

Attachments

  • opi3b.dts.txt
    187.5 KB · Views: 245
Yes the ranges for pcie are invalid- using those from orangepi (or any other rock-3c specific dts) as overlay for quartz64-b dtb - makes the drive attach, speeds looks OK, although controller sometimes restart, so it's not 100% stable I think.
 
Code:
root@generic:~ # dmesg|grep spi0
spi0: <Rockchip Serial Flash Controller> mem 0xfe300000-0xfe303fff irq 65 on ofwbus0
spi0: Rockchip SFC version 5
spibus0: <OFW SPI bus> on spi0
GEOM: flash/spi0: the secondary GPT header is not in the last LBA.
root@generic:~ # gpart show flash/spi0
=>  34  8125  flash/spi0  GPT  (16M) [CORRUPT]
    34    30              - free -  (15K)
    64   960           1  linux-data  (480K)
  1024  6144           2  linux-data  (3.0M)
  7168   991              - free -  (496K)

i'll post the patch later
works read/write
 
Back
Top