Running FreeBSD on Radxa Rock 3C RK3566 board

covacat · Nov 2, 2023

i usually decompile the the provided dtb and edit it directly
not very clean but don't have to deal with includes and shit

SleepWalker · Nov 2, 2023

covacat said:
you need disable-wp under the sdhci node

Thank you, I definitely put it in the wrong place.

SleepWalker · Nov 2, 2023

covacat and Phishfry

Thanks, all worked well !!!!

 

root@Radax-E25:/ # dmesg | grep mmc

rockchip_dwmmc0: <Synopsys DesignWare Mobile Storage Host Controller (RockChip)> mem 0xfe2b0000-0xfe2b3fff irq 33 on ofwbus0

rockchip_dwmmc0: Hardware version ID is 270a

mmc0: <MMC/SD bus> on rockchip_dwmmc0

mmc1: <MMC/SD bus> on sdhci_fdt0

mmcsd0: 31GB <SDHC SD16G 2.0 SN 00000D2B MFG 10/2022 by 254 42> at mmc0 50.0MHz/4bit/1016-block

mmcsd1: 16GB <MMCHC D9D16G 8.7 SN B1166BA0 MFG 02/2021 by 136 0x0003> at mmc1 200.0MHz/8bit/8192-block

mmcsd1boot0: 4MB partition 1 at mmcsd1

mmcsd1boot1: 4MB partition 2 at mmcsd1

mmcsd1rpmb: 4MB partition 3 at mmcsd1

covacat · Nov 2, 2023

clock-output-names = "rk808-clkout1", "rk808-clkout2"

the bug that caused a panic without this line in the dtb is fixed now in 14-STABLE

SleepWalker · Nov 6, 2023

Hi covacat (Titus).

Thanks for your help.

I'm trying the suggested patches for eqos.
Your patches are missing changes to struct eqos_softc, so compilation fails. Log attached

covacat · Nov 6, 2023

Code:

--- /home/titus/builds/freebsd-src/sys/dev/eqos/if_eqos_var.h    2023-10-19 14:25:53.844535000 +0300
+++ /sys/dev/eqos/if_eqos_var.h    2023-10-18 08:47:19.235640000 +0300
@@ -76,8 +76,14 @@
     struct resource     *res[EQOS_RES_COUNT];
     void            *irq_handle;
 #ifdef FDT
-    struct syscon        *grf;
+    struct syscon        *grf, *php_grf;
     int            grf_offset;
+    int            phy_mode;
+    struct eqos_ops        *ops;
+    clk_t            *clks;
+    clk_t            clk_stmmaceth;
+    bool            clock_in;
+    uint8_t            idx;
 #endif
     uint32_t        csr_clock;
     uint32_t        csr_clock_range;

try this patch
i forgot to add the header in the previous diff

Phishfry · Nov 6, 2023

I would like to follow along now. What source tree are you using? FreeBSD 14 or head ?
I don't see /dev/eqos in 13.2 src.

covacat · Nov 6, 2023

this is in 14
13.x has no support for rk35xx

SleepWalker · Nov 6, 2023

I'm testing on 14.0-RC4 on a FriendlyELEC R5S (rk3568)
We need an extra clock

eqos0: <DesignWare EQOS Gigabit ethernet> mem 0xfe2a0000-0xfe2affff irq 82,83 on ofwbus0
clknode_link_set_gate: Attempt to use unresolved linked clock: gmac0_clkin
eqos0: could not enable clock stmmaceth
device_attach: eqos0 attach returned 9

Code:

root@NanoPi-R5S:~ # grep Unres dmesg-r5s.txt
Unresolved linked clock found: clk_32k_pvtm
Unresolved linked clock found: usb480m_phy
Unresolved linked clock found: mpll
Unresolved linked clock found: i2s0_mclkin
Unresolved linked clock found: i2s1_mclkin
Unresolved linked clock found: i2s2_mclkin
Unresolved linked clock found: i2s3_mclkin
Unresolved linked clock found: gpu_pvtpll_out
Unresolved linked clock found: npu_pvtpll_out
Unresolved linked clock found: gmac0_clkin
Unresolved linked clock found: gmac1_clkin
Unresolved linked clock found: clk_gmac0_xpcs_mii
Unresolved linked clock found: clk_gmac1_xpcs_mii
Unresolved linked clock found: dummy
root@NanoPi-R5S:~ #

covacat · Nov 6, 2023

what your clocks and clock-names dtb line (under ethernet)

covacat · Nov 6, 2023

Code:

                clocks = <0x20 0x186 0x20 0x189 0x20 0x189 0x20 0xc7 0x20 0xc3 0x20 0xc4 0x20 0x189 0x20 0xc8 0x20 0xac 0x20 0xab>;
                clock-names = "stmmaceth", "mac_clk_rx", "mac_clk_tx", "clk_mac_refout", "aclk_mac", "pclk_mac", "clk_mac_speed", "ptp_ref", "pclk_xpcs", "clk_xpcs_eee";

i also have this

Code:

        external-gmac1-clock {

                compatible = "fixed-clock";
                clock-frequency = <0x7735940>;
                clock-output-names = "gmac1_clkin";
                #clock-cells = <0x0>;
                phandle = <0x13e>;
        };
        xpcs-gmac1-clock {

                compatible = "fixed-clock";
                clock-frequency = <0x7735940>;
                clock-output-names = "clk_gmac1_xpcs_mii";
                #clock-cells = <0x0>;
                phandle = <0x13f>;
        };

SleepWalker · Nov 6, 2023

Thanks to the code from Soren,, I can use eqos in 13.2 in acpi mode.
I don't understand why this is missing in 14.0

SleepWalker · Nov 6, 2023

covacat said:

Code:

                clocks = <0x20 0x186 0x20 0x189 0x20 0x189 0x20 0xc7 0x20 0xc3 0x20 0xc4 0x20 0x189 0x20 0xc8 0x20 0xac 0x20 0xab>;
                clock-names = "stmmaceth", "mac_clk_rx", "mac_clk_tx", "clk_mac_refout", "aclk_mac", "pclk_mac", "clk_mac_speed", "ptp_ref", "pclk_xpcs", "clk_xpcs_eee";

i also have this

Code:

        external-gmac1-clock {

                compatible = "fixed-clock";
                clock-frequency = <0x7735940>;
                clock-output-names = "gmac1_clkin";
                #clock-cells = <0x0>;
                phandle = <0x13e>;
        };
        xpcs-gmac1-clock {

                compatible = "fixed-clock";
                clock-frequency = <0x7735940>;
                clock-output-names = "clk_gmac1_xpcs_mii";
                #clock-cells = <0x0>;
                phandle = <0x13f>;
        };

I need time to carefully check my DTS.
What kind of speed tests do you get on OrangePi-3B?
I test using iperf3 and on some boards the speeds do not exceed 300Mbit/sec.
This is very sad.

SleepWalker · Nov 6, 2023

Code:

ethernet@fe2a0000 {
                        
                compatible = "rockchip,rk3568-gmac", "snps,dwmac-4.20a";
                reg = <0x0 0xfe2a0000 0x0 0x10000>;
                interrupts = <0x0 0x1b 0x4 0x0 0x18 0x4>;
                interrupt-names = "macirq", "eth_wake_irq";
                clocks = <0x21 0x182 0x21 0x185 0x21 0x185 0x21 0xb8 0x21 0xb4 0x21 0xb5 0x21 0x185 0x21 0xb9>;
                clock-names = "stmmaceth", "mac_clk_rx", "mac_clk_tx", "clk_mac_refout", "aclk_mac", "pclk_mac", "clk_mac_speed", "ptp_ref";
                resets = <0x21 0xd7>;
                reset-names = "stmmaceth";
                rockchip,grf = <0x1b>;
                snps,axi-config = <0x22c>;
                snps,mixed-burst;
                snps,mtl-rx-config = <0x22d>;
                snps,mtl-tx-config = <0x22e>;
                snps,tso;
                status = "okay";
                assigned-clocks = <0x21 0x185 0x21 0x182>;
                assigned-clock-parents = <0x21 0x183 0x21 0xb6>;
                assigned-clock-rates = <0x0 0x7735940>;
                clock_in_out = "output";
                pinctrl-0 = <0xef 0xf3 0xf2 0xf4 0xf5 0xf0>;
                pinctrl-names = "default";
                phy-mode = "rgmii";
                phy-handle = <0x22b>;
                phy-supply = <0x34>;
                snps,reset-gpio = <0xb9 0x15 0x1>;
                snps,reset-active-low;
 snps,reset-delays-us = <0x0 0x3a98 0xc350>;
                tx_delay = <0x3c>;
                rx_delay = <0x2f>;
                phandle = <0x229>;

Code:

&gmac0 {
        assigned-clocks = <&cru SCLK_GMAC0_RX_TX>, <&cru SCLK_GMAC0>;
        assigned-clock-parents = <&cru SCLK_GMAC0_RGMII_SPEED>, <&cru CLK_MAC0_2TOP>;
        assigned-clock-rates = <0>, <125000000>;
        clock_in_out = "output";
        phy-handle = <&rgmii_phy0>;
        phy-mode = "rgmii";
        pinctrl-names = "default";
        pinctrl-0 = <&gmac0_miim
                     &gmac0_tx_bus2
                     &gmac0_rx_bus2
                     &gmac0_rgmii_clk
                     &gmac0_rgmii_bus>;
        snps,reset-gpio = <&gpio0 RK_PC5 GPIO_ACTIVE_LOW>;
        snps,reset-active-low;
        /* Reset time is 15ms, 50ms for rtl8211f */
        snps,reset-delays-us = <0 15000 50000>;
        tx_delay = <0x3c>;
        rx_delay = <0x2f>;
        status = "okay";
};

covacat · Nov 6, 2023

this socks have 2 ethernet gmacs
yours uses gmac0 and mine gmac1 it seems

looks like the parent might be wrong by default and set to the external clock
try sysctl hw.clock|grep gmac0.*parent

covacat · Nov 6, 2023

also this may fix the parent if is bad

C:

        /* Set the assigned clocks parent and freq */
        if (clk_set_assigned(dev, node) != 0) {
                device_printf(dev, "clk_set_assigned failed\n");
                goto fail;
        }

insert it before
error = clk_enable(sc->clk_stmmaceth);
in /sys/dev/eqos/if_eqos_fdt.c

covacat · Nov 6, 2023

rx seems wire speed and tx seems to suck masively like 60mb/s

SleepWalker · Nov 6, 2023

I think I solved the problem by changing the DTS, I described fixed-clock,
and changed the description of gmac0

Code:

gmac0_clkin: external-gmac0-clock {
                compatible = "fixed-clock";
                clock-frequency = <125000000>;
                clock-output-names = "gmac0_clkin";
                #clock-cells = <0>;
 };

Code:

&gmac0 {
        phy-mode = "rgmii";
        phy-handle = <&rgmii_phy0>;
        clock_in_out = "input";

        snps,reset-gpio = <&gpio0 RK_PC5 GPIO_ACTIVE_LOW>;
        snps,reset-active-low;
        snps,reset-delays-us = <0 15000 50000>;

        assigned-clocks = <&cru SCLK_GMAC0_RX_TX>, <&cru SCLK_GMAC0>;
        assigned-clock-parents = <&cru SCLK_GMAC0_RGMII_SPEED>, <&gmac0_clkin>;

        pinctrl-names = "default";
        pinctrl-0 = <&gmac0_miim
                     &gmac0_tx_bus2
                     &gmac0_rx_bus2
                     &gmac0_rgmii_clk
                     &gmac0_rgmii_bus
                     &gmac0_clkinout>;

        tx_delay = <0x3c>;
        rx_delay = <0x2f>;

        status = "okay";

C:

eq0: <DesignWare EQOS Gigabit ethernet> mem 0xfe2a0000-0xfe2affff irq 87,88 on ofwbus0
eq0: Clock input from the PHY
eq0: no phy-supply property
eq0: DesignWare EQOS ver 0x51 (0x30)
eq0: hw features 160171e7 010c01c8 10000000 00000000
eq0: Ethernet address f2:00:91:0b:20:61
eq0: TX ring @ 0xb1a000, RX ring @ 0x2adb000
miibus0: <MII bus> on eq0
rgephy0: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 0 on miibus0
rgephy0: OUI 0x00e04c, model 0x0011, rev. 6
rgephy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto
rgephy1: <RTL8169S/8110S/8211 1000BASE-T media interface> PHY 1 on miibus0
rgephy1: OUI 0x00e04c, model 0x0011, rev. 6
rgephy1:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT-FDX, 1000baseT-FDX-master, auto
eq0: bpf attached
eq0: Ethernet address: f2:00:91:0b:20:61

it's better than it was before

covacat · Nov 7, 2023

a crappy test shows 20x more interrupts on tx than rx
that is probably not good
rx is 96MB/s and TX is 16MB/s
(just piping dd of /dev/zero thru nc then to /dev/null on the remote host) (lazy man's iperf)

Code:

TX
==============
gic0,s32: eqos0                                       24308      22846
gic0,s32: eqos0                                       15840      15086
gic0,s32: eqos0                                       21298      21234
gic0,s32: eqos0                                        6607       6210
gic0,s32: eqos0                                       13457      13027
gic0,s32: eqos0                                       13922      13072
gic0,s32: eqos0                                       20601      19490
gic0,s32: eqos0                                       15779      14914
gic0,s32: eqos0                                       14499      13614
gic0,s32: eqos0                                       12951      12912
gic0,s32: eqos0                                       14373      14330
=======================
RX
gic0,s32: eqos0                                         729        726
gic0,s32: eqos0                                         733        731
gic0,s32: eqos0                                         756        736
gic0,s32: eqos0                                         760        757
gic0,s32: eqos0                                         736        734
gic0,s32: eqos0                                         779        731
gic0,s32: eqos0                                         755        753
gic0,s32: eqos0                                         739        738
gic0,s32: eqos0                                         733        731
gic0,s32: eqos0                                         766        720
gic0,s32: eqos0                                         743        741

SleepWalker · Nov 7, 2023

Hi covacat !

Great, your patches on eqos work. Now I have to get this to work and acpi mode,
then they can be used on the RK3588, while we do not have clock drivers for the RK3588.

A small note, the switch is set to 100Mbit Full-duplex, but the device shows Half-duplex.

Code:

root@NanoPi-R5S:~ # ifconfig
re0: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether 58:9c:fc:10:24:6e
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
re1: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=201b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,WOL_MAGIC>
        ether 58:9c:fc:10:e7:10
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
eq0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=80008<VLAN_MTU,LINKSTATE>
        ether 22:30:f4:8e:eb:d2
        inet 192.168.1.208 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (100baseTX <half-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
root@NanoPi-R5S:~ # ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=107 time=51.961 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=107 time=51.899 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=107 time=51.783 ms
^C
--- 8.8.8.8 ping statistics ---
4 packets transmitted, 3 packets received, 25.0% packet loss
round-trip min/avg/max/stddev = 51.783/51.881/51.961/0.074 ms
root@NanoPi-R5S:~ #

covacat · Nov 7, 2023

the half duplex stuff happens here too, i have to dig some more
also need to investigate the slow tx problem
i don't have easy access to a 100tx switch an the 1000t i have is unmanaged so i tested with an old usb to ethernet dongle
but the problem was the same.the dongle said fdx, eqos said hdx

covacat · Nov 11, 2023

i enabled support for rx/tx hw cksum offload but its not going any faster
its just better at lower cpu speeds but still max tx i can get is about 50MB/s
rx works at 110MB/s
surprisingly it wont work any faster in the official opi linux distro

covacat · Nov 14, 2023

fixed a (self induced) bug in sdhci_fdt.c (patch is on reviews.freebsd.org)
fiddled with eqos and broke something in the phy (hardware) and now it works full gigabit
also added tx and rx hw checksums
seems like somehow the external gmac clock which passes thru the phy caused problems

Phishfry · Nov 14, 2023

What about WOL? Have you seen anything regarding?

RK3399 doesn't support it.

Code:

dwc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
    options=8000b<RXCSUM,TXCSUM,VLAN_MTU,LINKSTATE>
    ether ca:84:69:7b:12:5c

wake ca:84:69:7b:12:5c

SleepWalker · Nov 15, 2023

covacat said:
fixed a (self induced) bug in sdhci_fdt.c (patch is on reviews.freebsd.org)
fiddled with eqos and broke something in the phy (hardware) and now it works full gigabit
also added tx and rx hw checksums
seems like somehow the external gmac clock which passes thru the phy caused problems

Tell me how to do this, we need corrections in the driver source code, or corrections in DTS?

Running FreeBSD on Radxa Rock 3C RK3566 board

covacat

SleepWalker

SleepWalker

covacat

SleepWalker

Attachments

covacat

Phishfry

covacat

SleepWalker

Attachments

covacat

covacat

SleepWalker

Attachments

SleepWalker

SleepWalker

covacat

covacat

covacat

SleepWalker

covacat

SleepWalker

covacat

covacat

covacat

Phishfry

SleepWalker