can't get sfp+ nic to work amd driver issue?

hi all,
I have a baremetal pc on which i'm trying to get the onboard 2 sfp+ ports to work. I tried with a freebsd 13.2 install and freebsd 14 livecd. Also tested with ubuntu, where everything works fine with the same setup of cables etc.
With freebsd the sfp+ ports are recognized, but not sure what is going on, although cables are connected, no matter what i try, i get: status: no carrier.
I had some help & suggestions on IRC, I was told it could be a driver bug. I'm familiar with Linux but new to *bsd, so tried to collect relevant information in the hope to get it working or correct my mistakes.

full dmesg output from freebsd 14 live usb medium boot
partial dmesg output on freebsd 14 for sfp+ nics:
Code:
ax0: xgbe_phy_sfp_signals: port_sfp_inputs: 0x0
ax0: SFP detected:
ax0:   vendor:    OEM
ax0:   part number:    SFP-H10GB-CU5M
ax0:   revision level: 09
ax0:   serial number:  CSC231202820016
ax1: xgbe_phy_sfp_signals: port_sfp_inputs: 0x7
ax1: xgbe_phy_sfp_detect: mod absent
ax1: xgbe_phy_sfp_signals: port_sfp_inputs: 0x2
ax1: SFP detected:
ax1:   vendor:    Walsun
ax1:   part number:    HXSX-ATRI-1
ax1:   revision level: 1.0
ax1:   serial number:  H230S014913
ax1: xgbe_phy_sfp_signals: port_sfp_inputs: 0x0
https://pastebin.com/vVagXkaS
Also added upon recommendation output of pciconf -lv

I can configure ax0 or ax1 fine and bring them also up but it will show 'no carrier' in ifconfig
Code:
ax0: flags=8867<UP,BROADCAST,DEBUG,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   description: OPT1 (opt1)
   options=4e0032b<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
   ether d0:63:b4:05:47:43
   inet 10.133.120.125 netmask 0xffffff00 broadcast 10.133.120.255
   media: Ethernet autoselect
   status: no carrier
   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ax1: flags=8867<UP,BROADCAST,DEBUG,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
   description: LANSFP (opt1)
   options=4e0032b<RXCSUM,TXCSUM,VLAN_MTU,JUMBO_MTU,TSO4,TSO6,RXCSUM_IPV6,TXCSUM_IPV6,NOMAP>
   ether d0:63:b4:05:47:44
   media: Ethernet autoselect
   status: no carrier
   nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>

when I check with tcpdump on the baremetal pc and ping it from my other arch linux on the local lan i see this:
Code:
tcpdump --interface ax0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ax0, link-type EN10MB (Ethernet), capture size 262144 bytes
21:28:10.551283 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:11.287919 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:11.821977 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
21:28:12.287909 LLDP, length 69
21:28:13.284656 LLDP, length 69
21:28:14.281404 LLDP, length 69
21:28:15.278147 LLDP, length 69
21:28:15.586787 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
21:28:16.053341 IP 10.133.120.5.38313 > 239.255.255.250.ssdp: UDP, length 101
21:28:16.053354 IP 10.133.120.5.33435 > 239.255.255.250.ssdp: UDP, length 101
21:28:16.053363 IP 10.133.120.3.59149 > 239.255.255.250.ssdp: UDP, length 101
21:28:16.053382 IP 10.133.120.4.33015 > 239.255.255.250.ssdp: UDP, length 101
21:28:16.053388 IP 10.133.120.3.51253 > 239.255.255.250.ssdp: UDP, length 101
21:28:16.053391 IP 10.133.120.4.44274 > 239.255.255.250.ssdp: UDP, length 101
21:28:20.901812 IP 10.133.120.3.59695 > 239.255.255.250.ssdp: UDP, length 129
21:28:20.901941 IP 10.133.120.4.48273 > 239.255.255.250.ssdp: UDP, length 129
21:28:20.902080 IP 10.133.120.5.39369 > 239.255.255.250.ssdp: UDP, length 129
21:28:21.102922 IP 10.133.120.3.59695 > 239.255.255.250.ssdp: UDP, length 129
21:28:21.103053 IP 10.133.120.4.48273 > 239.255.255.250.ssdp: UDP, length 129
21:28:21.103188 IP 10.133.120.5.39369 > 239.255.255.250.ssdp: UDP, length 129
21:28:21.302014 IP 10.133.120.3.59695 > 239.255.255.250.ssdp: UDP, length 129
21:28:21.302027 IP 10.133.120.4.48273 > 239.255.255.250.ssdp: UDP, length 129
21:28:21.302152 IP 10.133.120.5.39369 > 239.255.255.250.ssdp: UDP, length 129
21:28:23.113808 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:23.821144 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
21:28:24.037080 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:26.053695 IP 10.133.120.3.34175 > 239.255.255.250.ssdp: UDP, length 101
21:28:26.053703 IP 10.133.120.5.40299 > 239.255.255.250.ssdp: UDP, length 101
21:28:26.053713 IP 10.133.120.4.42403 > 239.255.255.250.ssdp: UDP, length 101
21:28:27.473666 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
21:28:34.769972 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
21:28:35.573080 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:36.053367 IP 10.133.120.3.48149 > 239.255.255.250.ssdp: UDP, length 101
21:28:36.053383 IP 10.133.120.4.59157 > 239.255.255.250.ssdp: UDP, length 101
21:28:36.053391 IP 10.133.120.5.52090 > 239.255.255.250.ssdp: UDP, length 101
21:28:36.166390 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:36.820373 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
21:28:37.206600 LLDP, length 69
21:28:38.203331 LLDP, length 69
21:28:39.200100 LLDP, length 69
21:28:40.196843 LLDP, length 69
21:28:40.943626 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
21:28:46.053949 IP 10.133.120.3.60867 > 239.255.255.250.ssdp: UDP, length 101
21:28:46.053966 IP 10.133.120.4.38949 > 239.255.255.250.ssdp: UDP, length 101
21:28:46.053970 IP 10.133.120.5.47332 > 239.255.255.250.ssdp: UDP, length 101
21:28:46.053973 IP 10.133.120.3.51494 > 239.255.255.250.ssdp: UDP, length 101
21:28:46.053976 IP 10.133.120.4.55277 > 239.255.255.250.ssdp: UDP, length 101
21:28:46.053984 IP 10.133.120.5.38689 > 239.255.255.250.ssdp: UDP, length 101
21:28:48.142246 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:48.592245 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::16: HBH ICMP6, multicast listener report v2, 4 group record(s), length 88
21:28:48.819592 IP6 fe80::3efd:feff:fe9e:cb7c > ff02::2: ICMP6, router solicitation, length 16
^C
51 packets captured
51 packets received by filter
0 packets dropped by kernel


tested a few thing, below snippet includes ifconfig ouput, then I bring the sfp+ devices up, assign static IPs and try to ping the other machine to which it is connected and in between ifconfig outputs

Here some output from working ubuntu running on the baremetal in case it helps in some way

lspci output from working ubuntu
Code:
0a:00.2 Ethernet controller: Advanced Micro Devices, Inc. [AMD] Device 1458
    Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1458
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin B routed to IRQ 42
    IOMMU group: 32
    Region 0: Memory at fcb60000 (32-bit, non-prefetchable) [size=128K]
    Region 1: Memory at fcb40000 (32-bit, non-prefetchable) [size=128K]
    Region 2: Memory at fcb82000 (64-bit, non-prefetchable) [size=8K]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [64] Express (v2) Endpoint, MSI 00
        DevCap:    MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0W
        DevCtl:    CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 256 bytes, MaxReadReq 512 bytes
        DevSta:    CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap:    Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 16GT/s, Width x16
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
             10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [c0] MSI-X: Enable+ Count=7 Masked-
        Vector table: BAR=2 offset=00000000
        PBA: BAR=2 offset=00001000
    Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [2a0 v1] Access Control Services
        ACSCap:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
        ACSCtl:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
    Kernel driver in use: amd-xgbe
    Kernel modules: amd_xgbe

0a:00.3 Ethernet controller: Advanced Micro Devices, Inc. [AMD] Device 1458
    Subsystem: Advanced Micro Devices, Inc. [AMD] Device 1458
    Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0, Cache Line Size: 64 bytes
    Interrupt: pin C routed to IRQ 50
    IOMMU group: 33
    Region 0: Memory at fcb20000 (32-bit, non-prefetchable) [size=128K]
    Region 1: Memory at fcb00000 (32-bit, non-prefetchable) [size=128K]
    Region 2: Memory at fcb80000 (64-bit, non-prefetchable) [size=8K]
    Capabilities: [48] Vendor Specific Information: Len=08 <?>
    Capabilities: [50] Power Management version 3
        Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [64] Express (v2) Endpoint, MSI 00
        DevCap:    MaxPayload 256 bytes, PhantFunc 0, Latency L0s <4us, L1 unlimited
            ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0W
        DevCtl:    CorrErr- NonFatalErr- FatalErr- UnsupReq-
            RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
            MaxPayload 256 bytes, MaxReadReq 512 bytes
        DevSta:    CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
        LnkCap:    Port #0, Speed 16GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
            ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
        LnkCtl:    ASPM Disabled; RCB 64 bytes, Disabled- CommClk+
            ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
        LnkSta:    Speed 16GT/s, Width x16
            TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
        DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR-
             10BitTagComp+ 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
             EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
             FRS- TPHComp- ExtTPHComp-
             AtomicOpsCap: 32bit- 64bit- 128bitCAS-
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled,
             AtomicOpsCtl: ReqEn-
        LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1-
             EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
             Retimer- 2Retimers- CrosslinkRes: unsupported
    Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
        Address: 0000000000000000  Data: 0000
    Capabilities: [c0] MSI-X: Enable+ Count=7 Masked-
        Vector table: BAR=2 offset=00000000
        PBA: BAR=2 offset=00001000
    Capabilities: [100 v1] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
    Capabilities: [2a0 v1] Access Control Services
        ACSCap:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
        ACSCtl:    SrcValid- TransBlk- ReqRedir- CmpltRedir- UpstreamFwd- EgressCtrl- DirectTrans-
    Kernel driver in use: amd-xgbe
    Kernel modules: amd_xgbe
ethtool output on ubuntu
Code:
Settings for enp10s0f2:
    Supported ports: [ TP ]
    Supported link modes:   10000baseCR/Full
    Supported pause frame use: No
    Supports auto-negotiation: No
    Supported FEC modes: Not reported
    Advertised link modes:  10000baseCR/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: No
    Advertised FEC modes: Not reported
    Speed: 10000Mb/s
    Duplex: Full
    Auto-negotiation: off
    Port: None
    PHYAD: 0
    Transceiver: internal
        Current message level: 0x00000034 (52)
                               link ifdown ifup
    Link detected: yes
   
    Settings for enp10s0f3:
    Supported ports: [ FIBRE ]
    Supported link modes:   10000baseSR/Full
    Supported pause frame use: No
    Supports auto-negotiation: No
    Supported FEC modes: Not reported
    Advertised link modes:  10000baseSR/Full
    Advertised pause frame use: No
    Advertised auto-negotiation: No
    Advertised FEC modes: Not reported
    Speed: 10000Mb/s
    Duplex: Full
    Auto-negotiation: off
    Port: None
    PHYAD: 0
    Transceiver: internal
        Current message level: 0x00000034 (52)
                               link ifdown ifup
    Link detected: yes

ethtool -i enp10s0f2
driver: amd-xgbe
version: 6.2.0-39-generic
firmware-version: 19.118.48
expansion-rom-version:
bus-info: 0000:0a:00.2
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

ethtool -i enp10s0f3
driver: amd-xgbe
version: 6.2.0-39-generic
firmware-version: 19.118.48
expansion-rom-version:
bus-info: 0000:0a:00.3
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

ifconfig entries for sfp+ nic's on ubuntu, I setup a static network to test connectivity with another local arch linux server on local lan:
Code:
enp10s0f2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.133.120.11  netmask 255.255.255.0  broadcast 10.133.120.255
        inet6 fe80::d263:b4ff:fe05:4743  prefixlen 64  scopeid 0x20<link>
        ether d0:63:b4:05:47:43  txqueuelen 1000  (Ethernet)
        RX packets 2  bytes 128 (128.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 59  bytes 4448 (4.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 43

enp10s0f3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.133.120.12  netmask 255.255.255.0  broadcast 10.133.120.255
        inet6 fe80::d263:b4ff:fe05:4744  prefixlen 64  scopeid 0x20<link>
        ether d0:63:b4:05:47:44  txqueuelen 1000  (Ethernet)
        RX packets 1352  bytes 92127 (92.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 47  bytes 3975 (3.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 58

ubuntu lshw network devices info:
Code:
lshw -class network
  *-network                
       description: Wireless interface
       product: Wi-Fi 6 AX210/AX211/AX411 160MHz
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:03:00.0
       logical name: wlp3s0
       version: 1a
       serial: 2c:33:58:f3:cc:4e
       width: 64 bits
       clock: 33MHz
       capabilities: pm msi pciexpress msix bus_master cap_list ethernet physical wireless
       configuration: broadcast=yes driver=iwlwifi driverversion=6.2.0-39-generic firmware=72.a764baac.0 ty-a0-gf-a0-72.uc latency=0 link=no multicast=yes wireless=IEEE 802.11
       resources: irq:113 memory:fcd00000-fcd03fff
  *-network
       description: Ethernet interface
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:04:00.0
       logical name: enp4s0
       version: 04
       serial: d0:63:b4:05:47:3f
       capacity: 1Gbit/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=igc driverversion=6.2.0-39-generic firmware=2022:889d latency=0 link=no multicast=yes port=twisted pair
       resources: irq:37 memory:fc900000-fc9fffff memory:fca00000-fca03fff memory:fc800000-fc8fffff
  *-network
       description: Ethernet interface
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:05:00.0
       logical name: enp5s0
       version: 04
       serial: d0:63:b4:05:47:40
       capacity: 1Gbit/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=igc driverversion=6.2.0-39-generic firmware=2022:889d latency=0 link=no multicast=yes port=twisted pair
       resources: irq:37 memory:fc600000-fc6fffff memory:fc700000-fc703fff memory:fc500000-fc5fffff
  *-network
       description: Ethernet interface
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:06:00.0
       logical name: enp6s0
       version: 04
       serial: d0:63:b4:05:47:41
       capacity: 1Gbit/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=igc driverversion=6.2.0-39-generic firmware=2022:889d latency=0 link=no multicast=yes port=twisted pair
       resources: irq:78 memory:fc300000-fc3fffff memory:fc400000-fc403fff memory:fc200000-fc2fffff
  *-network
       description: Ethernet interface
       product: Intel Corporation
       vendor: Intel Corporation
       physical id: 0
       bus info: pci@0000:07:00.0
       logical name: enp7s0
       version: 04
       serial: d0:63:b4:05:47:42
       capacity: 1Gbit/s
       width: 32 bits
       clock: 33MHz
       capabilities: pm msi msix pciexpress bus_master cap_list rom ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation
       configuration: autonegotiation=on broadcast=yes driver=igc driverversion=6.2.0-39-generic firmware=2022:889d latency=0 link=no multicast=yes port=twisted pair
       resources: irq:78 memory:fc000000-fc0fffff memory:fc100000-fc103fff memory:fbf00000-fbffffff
  *-network:0
       description: Ethernet interface
       product: Advanced Micro Devices, Inc. [AMD]
       vendor: Advanced Micro Devices, Inc. [AMD]
       physical id: 0.2
       bus info: pci@0000:0a:00.2
       logical name: enp10s0f2
       version: 00
       serial: d0:63:b4:05:47:43
       size: 10Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pm pciexpress msi msix bus_master cap_list ethernet physical tp
       configuration: autonegotiation=off broadcast=yes driver=amd-xgbe driverversion=6.2.0-39-generic duplex=full firmware=19.118.48 ip=10.133.120.11 latency=0 link=yes multicast=yes speed=10Gbit/s
       resources: irq:42 memory:fcb60000-fcb7ffff memory:fcb40000-fcb5ffff memory:fcb82000-fcb83fff
  *-network:1
       description: Ethernet interface
       product: Advanced Micro Devices, Inc. [AMD]
       vendor: Advanced Micro Devices, Inc. [AMD]
       physical id: 0.3
       bus info: pci@0000:0a:00.3
       logical name: enp10s0f3
       version: 00
       serial: d0:63:b4:05:47:44
       size: 10Gbit/s
       width: 64 bits
       clock: 33MHz
       capabilities: pm pciexpress msi msix bus_master cap_list ethernet physical fibre
       configuration: autonegotiation=off broadcast=yes driver=amd-xgbe driverversion=6.2.0-39-generic duplex=full firmware=19.118.48 ip=10.133.120.12 latency=0 link=yes multicast=yes speed=10Gbit/s
       resources: irq:50 memory:fcb20000-fcb3ffff memory:fcb00000-fcb1ffff memory:fcb80000-fcb81fff

I also had found this https://reviews.freebsd.org/D25793 maybe it is a driver issue?
 
I think this must be a bug. Tested other cables, same results, freebsd dmesg shows nothing particular besides what I posted earlier, in linux side I get every 12 seconds or so this:

Mon Apr 22 19:26:58 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:26:58 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:27:10 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:27:11 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:27:23 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:27:24 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:27:36 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:27:36 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:27:49 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:27:49 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:28:01 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:28:02 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:28:14 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:28:14 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:28:27 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:28:27 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:28:39 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:28:40 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:28:52 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:28:53 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:29:05 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:29:05 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None
[Mon Apr 22 19:29:18 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Down
[Mon Apr 22 19:29:18 2024] i40e 0000:07:00.0 enp7s0f0: NIC Link is Up, 10 Gbps Full Duplex, Flow Control: None

i also tried opnsense, pfsense with same results/behaviour, what else could I try to get this fixed?
 
what else could I try to get this fixed?
I would contact the author of the driver.

axp(4)
Code:
AUTHORS
     The axp device driver was written by Advanced Micro Devices Inc.

     For any issues and support requirements, email the details to
     <rajesh1.kumar@amd.com>.

. https://reviews.freebsd.org/D25793
10Gigabit Ethernet driver for AMD SoC

Authored by rajesh1.kumar_amd.com on Jul 24 2020, 6:09 AM.

Attach dmesg(8) and pciconf(8) to the mail.

Eventually open a PR.
 
I contacted Rajesh, so indeed the linux drivers will be ported at some point to freebsd but that will probably towards the end of the year 24, until then no luck.
I have done some further testing, interestingly Red-Hat linux is also not working but any other linux distro (well I only tested Debian and derivates like ubuntu, VyOS, also Arch, all working).
I also did look into submitting a bug/PR as it was suggested but that seems a bit complicated, might still do it at some point...
 
Just ran into this issue with a new appliance from solid-run bedrock v3000. I have yet to find an SFP that will work on the AMD ax interfaces .. all show no carrier. tcpdump does show traffic.

Not sure why I dont even get SFP detected? I have tried multiple SFP, including ones tested by vendor. The below ones are from fs.com .. one 10G RJ45 SFP+ and the other a 10G DAC SFP+

[2.7.2-RELEASE][root@pfSense.home.arpa]/root: ifconfig ax0
ax0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e503bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether d0:63:b4:05:09:8b
inet6 fe80::d263:b4ff:fe05:98b%ax0 prefixlen 64 scopeid 0x5
media: Ethernet autoselect
status: no carrier
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
[2.7.2-RELEASE][root@pfSense.home.arpa]/root: ifconfig ax1
ax1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=4e503bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
ether d0:63:b4:05:09:8c
inet 10.10.42.4 netmask 0xfffffe00 broadcast 10.10.43.255
inet6 fe80::d263:b4ff:fe05:98c%ax1 prefixlen 64 scopeid 0x6
media: Ethernet autoselect
status: no carrier
nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

from /var/log/system.log

Nov 1 00:19:22 pfSense kernel: ax0: <AMD 10 Gigabit Ethernet Driver> mem 0xfcd60000-0xfcd7ffff,0xfcd40000-0xfcd5ffff,0xfcd82000-0xfcd83fff at device 0.2 on pci8
Nov 1 00:19:22 pfSense kernel: ax0: Using 512 TX descriptors and 512 RX descriptors
Nov 1 00:19:22 pfSense kernel: ax0: Using 3 RX queues 3 TX queues
Nov 1 00:19:22 pfSense kernel: ax0: Using MSI-X interrupts with 7 vectors
Nov 1 00:19:22 pfSense kernel: ax0: Ethernet address: d0:63:b4:05:09:8b
Nov 1 00:19:22 pfSense kernel: ax0: xgbe_config_sph_mode: SPH disabled in channel 0
Nov 1 00:19:22 pfSense kernel: ax0: xgbe_config_sph_mode: SPH disabled in channel 1
Nov 1 00:19:22 pfSense kernel: ax0: xgbe_config_sph_mode: SPH disabled in channel 2
Nov 1 00:19:22 pfSense kernel: ax0: RSS Enabled
Nov 1 00:19:22 pfSense kernel: ax0: Receive checksum offload Enabled
Nov 1 00:19:22 pfSense kernel: ax0: VLAN filtering Enabled
Nov 1 00:19:22 pfSense kernel: ax0: VLAN Stripping Enabled
Nov 1 00:19:22 pfSense kernel: ax0: netmap queues/slots: TX 3/512, RX 3/512
Nov 1 00:19:22 pfSense kernel: ax1: <AMD 10 Gigabit Ethernet Driver> mem 0xfcd20000-0xfcd3ffff,0xfcd00000-0xfcd1ffff,0xfcd80000-0xfcd81fff at device 0.3 on pci8
Nov 1 00:19:22 pfSense kernel: ax1: Using 512 TX descriptors and 512 RX descriptors
Nov 1 00:19:22 pfSense kernel: ax1: Using 3 RX queues 3 TX queues
Nov 1 00:19:22 pfSense kernel: ax1: Using MSI-X interrupts with 7 vectors
Nov 1 00:19:22 pfSense kernel: ax1: Ethernet address: d0:63:b4:05:09:8c
Nov 1 00:19:22 pfSense kernel: ax1: xgbe_config_sph_mode: SPH disabled in channel 0
Nov 1 00:19:22 pfSense kernel: ax1: xgbe_config_sph_mode: SPH disabled in channel 1
Nov 1 00:19:22 pfSense kernel: ax1: xgbe_config_sph_mode: SPH disabled in channel 2
Nov 1 00:19:22 pfSense kernel: ax1: RSS Enabled
Nov 1 00:19:22 pfSense kernel: ax1: Receive checksum offload Enabled
Nov 1 00:19:22 pfSense kernel: ax1: VLAN filtering Enabled
Nov 1 00:19:22 pfSense kernel: ax1: VLAN Stripping Enabled
Nov 1 00:19:22 pfSense kernel: ax1: netmap queues/slots: TX 3/512, RX 3/512
Nov 1 00:19:22 pfSense kernel: ax0: Link is DOWN
Nov 1 00:19:22 pfSense kernel: ax1: Link is DOWN
 
Has someone submitted a bug report with FreeBSD for this driver issue? Do you have any updated information? I was actually looking at picking up Solid Run HoneyComb Ryzen V3000 to use as a FreeBSD bhyve host for network function virtualization, but at this point it looks like there may still be an issue with the SFP driver.
 
Has someone submitted a bug report with FreeBSD for this driver issue? Do you have any updated information? I was actually looking at picking up Solid Run HoneyComb Ryzen V3000 to use as a FreeBSD bhyve host for network function virtualization, but at this point it looks like there may still be an issue with the SFP driver.
Just updated to pfsense 2.8.0 using FreeBSD 15.0 .. still no luck. I have tried multiple vendor SFP+ modules
SFP+ is fully identified, I can see traffic on the interface, but interface status is still "no carrier".
Code:
ax1: SFP detected:                                                                                                                         
ax1:   vendor:   CISCO-AMPHENOL                                                                                                       
ax1:   part number:    NDCCGF-C401                                                                                                     
ax1:   revision level: A                                                                                                                   
ax1:   serial number:  APF21292849
ax1: xgbe_config_sph_mode: SPH disabled in channel 0
ax1: xgbe_config_sph_mode: SPH disabled in channel 1 
ax1: xgbe_config_sph_mode: SPH disabled in channel 2 
ax1: RSS Enabled                                                                                                                           
ax1: Receive checksum offload Enabled
ax1: VLAN filtering Enabled                                                                                                                
ax1: VLAN Stripping Enabled                                                                                                                
[2.8.0-RELEASE][[email]root@pfSense.home.arpa[/email]]/root: ifconfig -vvv ax1                                                                            
ax1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500     options=4e503bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_I
PV6,HWSTATS,MEXTPG>                                                                                                                        
        ether d0:63:b4:05:09:8c                                                                                                            
        inet6 fe80::d263:b4ff:fe05:98c%ax1 prefixlen 64 scopeid 0x6 
        media: Ethernet autoselect                                                                                                         
        [B][COLOR=rgb(209, 72, 65)]status: no carrier      [/COLOR][/B]                                                                                                           
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> 
        drivername: ax1 
                                                                                                                   
[2.8.0-RELEASE][[email]root@pfSense.home.arpa[/email]]/root: uname -a                                                                                     
FreeBSD pfSense.home.arpa 15.0-CURRENT FreeBSD 15.0-CURRENT #1 RELENG_2_8_0-n256081-401ec5f685b9: Wed May 21 23:53:51 UTC 2025     root@fre
ebsd:/var/jenkins/workspace/pfSense-CE-snapshots-2_8_0-main/obj/amd64/0q9vjGjc/var/jenkins/workspace/pfSense-CE-snapshots-2_8_0-main/source
s/FreeBSD-src-RELENG_2_8_0/amd64.amd64/sys/pfSense amd64
 
pfSense is not supported here.

 
pfSense is not supported here.

This is aFReeBSD base OS issue. I have installed
FreeBSD 14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC amd64

And have the same issue with the BaseOS network drivers. Is there a way around?
Code:
# ifconfig -v ax0
ax0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500                                                   
        options=4e503bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWFILTER,VLAN_HWTSO,RXCSU
M_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>                                                                                          
        ether d0:63:b4:05:09:8b                                                                                             
        inet 10.10.42.6 netmask 0xfffffe00 broadcast 10.10.43.255                                                           
        media: Ethernet autoselect                                                                                          
        [B]status: no carrier [/B]                                                                                                 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>                                                                
        drivername: ax0                                                                                                     
                                                                                
# uname -a                                                                                                                  
FreeBSD  14.3-RELEASE FreeBSD 14.3-RELEASE releng/14.3-n271432-8c9ce319fef7 GENERIC amd64

# tcpdump -vv -i ax0                                                                                                        
tcpdump: listening on ax0, link-type EN10MB (Ethernet), snapshot length 262144 bytes                                        
17:20:11.411527 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.10.42.6 tell 10.10.43.142, length 46                
17:20:11.644634 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.10.42.1 tell 10.10.43.171, length 46                
17:20:12.197326 IP (tos 0x0, ttl 128, id 52711, offset 0, flags [none], proto UDP (17), length 291)                         
    10.10.43.227.54915 > 10.10.43.255.54915: [udp sum ok] UDP, length 263                                                   
17:20:12.435433 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.10.42.6 tell 10.10.43.142, length 46                
17:20:12.637100 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.10.42.1 tell 10.10.43.171, length 46                
17:20:12.695741 IP (tos 0x0, ttl 64, id 9796, offset 0, flags [DF], proto UDP (17), length 49)                              
    10.10.42.24.57429 > 10.10.43.255.32414: [udp sum ok] UDP, length 21                                                     
17:20:12.695748 IP (tos 0x0, ttl 64, id 46632, offset 0, flags [DF], proto UDP (17), length 49)                             
    10.10.42.23.46915 > 10.10.43.255.32414: [udp sum ok] UDP, length 21                                                     
17:20:12.695774 IP (tos 0x0, ttl 64, id 9795, offset 0, flags [DF], proto UDP (17), length 49)                              
    10.10.42.24.49476 > 10.10.43.255.32414: [udp sum ok] UDP, length 21                                                     
17:20:13.191816 IP (tos 0x0, ttl 128, id 52713, offset 0, flags [none], proto UDP (17), length 291)                         
    10.10.43.227.54915 > 10.10.43.255.54915: [udp sum ok] UDP, length 263                                                   
17:20:13.251256 24:5e:be:63:cf:c7 (oui Unknown) > Broadcast, ethertype Unknown (0x8874), length 60:                         
        0x0000:  0000 0100 0005 0000 c255 ac59 0000 0000  .........U.Y....                                                  
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................                                                  
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............                                                    
17:20:13.459394 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.10.42.6 tell 10.10.43.142, length 46                
17:20:13.637071 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 10.10.42.1 tell 10.10.43.171, length 46                

# dmesg | grep ax0                                                                                                          
ax0: <AMD 10 Gigabit Ethernet Driver> mem 0xfcd60000-0xfcd7ffff,0xfcd40000-0xfcd5ffff,0xfcd82000-0xfcd83fff at device 0.2 on
 pci8                                                                                                                       
ax0: Using 512 TX descriptors and 512 RX descriptors                                                                        
ax0: Using 3 RX queues 3 TX queues                                                                                          
ax0: Using MSI-X interrupts with 7 vectors                                                                                  
ax0: Ethernet address: d0:63:b4:05:09:8b                                                                                    
ax0: SPH Enabled                                                                                                            
ax0: xgbe_config_sph_mode: SPH enabled in channel 0                                                                         
ax0: xgbe_config_sph_mode: SPH enabled in channel 1                                                                         
ax0: xgbe_config_sph_mode: SPH enabled in channel 2                                                                         
ax0: RSS Enabled                                                                                                            
ax0: Receive checksum offload Enabled                                                                                       
ax0: VLAN filtering Enabled                                                                                                 
ax0: VLAN Stripping Enabled                                                                                                 
ax0: xgbe_phy_sfp_signals: port_sfp_inputs: 0x2                                                                             
ax0: netmap queues/slots: TX 3/512, RX 3/512                                                                                
ax0: Link is DOWN                                                                                                           
ax0: SPH Enabled                                                                                                            
ax0: xgbe_config_sph_mode: SPH enabled in channel 0                                                                         
ax0: xgbe_config_sph_mode: SPH enabled in channel 1                                                                         
ax0: xgbe_config_sph_mode: SPH enabled in channel 2                                                                         
ax0: RSS Enabled                                                                                                            
ax0: Receive checksum offload Enabled                                                                                       
ax0: VLAN filtering Enabled                                                                                                 
ax0: VLAN Stripping Enabled                                                                                                 
ax0: promiscuous mode enabled                                                                                               
ax0: promiscuous mode disabled                                                                                              
ax0: xgbe_phy_sfp_signals: port_sfp_inputs: 0x0                                                                             
ax0: promiscuous mode enabled                                                                                               
ax0: promiscuous mode disabled                                                                                              
ax0: promiscuous mode enabled                                                                                               
ax0: promiscuous mode disabled                                                                                              
ax0: promiscuous mode enabled                                                                                               
ax0: promiscuous mode disabled                                                                                              
# ping 10.10.42.1                                                                                                           
PING 10.10.42.1 (10.10.42.1): 56 data bytes                                                                                 
ping: sendto: Network is down                                                                                               
ping: sendto: Network is down                                                                                               
ping: sendto: Network is down
 
Back
Top