Hello Guys,
We have been pleasantly using Truenas 12.4/FreeBSD for over a year now.
We recently upgraded to a set of Supermicro 5049P-E1CTR36L 4U and upgraded them with a HPE Broadcom BCM57414 25G network card.
We thought everything was working fine untill some of them randomly gave "TX completion errors" and are unable to forward any traffic.
Right now we are running the onboard 10Gbit without any issues, but we need the 25Gbit working as soon as possible. Does anyone has a clue what to try?
Some logs below:
We have been pleasantly using Truenas 12.4/FreeBSD for over a year now.
We recently upgraded to a set of Supermicro 5049P-E1CTR36L 4U and upgraded them with a HPE Broadcom BCM57414 25G network card.
We thought everything was working fine untill some of them randomly gave "TX completion errors" and are unable to forward any traffic.
Right now we are running the onboard 10Gbit without any issues, but we need the 25Gbit working as soon as possible. Does anyone has a clue what to try?
Some logs below:
Code:
bnxt0@pci0:179:0:0: class=0x020000 card=0x159714e4 chip=0x16d714e4 rev=0x01 hdr=0x00
vendor = 'Broadcom Inc. and subsidiaries'
device = 'BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller'
class = network
subclass = ethernet
bar [10] = type Prefetchable Memory, range 64, base 0x38fffff10000, size 65536, enabled
bar [18] = type Prefetchable Memory, range 64, base 0x38ffffe00000, size 1048576, enabled
bar [20] = type Prefetchable Memory, range 64, base 0x38fffff22000, size 8192, enabled
cap 01[48] = powerspec 3 supports D0 D3 current D0
cap 03[50] = VPD
cap 05[58] = MSI supports 8 messages, 64 bit
cap 11[a0] = MSI-X supports 74 messages, enabled
Table in map 0x20[0x0], PBA in map 0x20[0x4a0]
cap 10[ac] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
link x8(x8) speed 8.0(8.0) ClockPM disabled
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
ecap 0003[13c] = Serial 1 e43d1afffe0deca0
ecap 0004[150] = Power Budgeting 1
ecap 0002[160] = VC 1 max VC0
ecap 000b[180] = Vendor 1 ID 0
ecap 0018[1b0] = LTR 1
ecap 000e[1b8] = ARI 1
ecap 0017[230] = TPH Requester 1
ecap 0019[300] = PCIe Sec 1 lane errors 0
ecap 001f[200] = Precision Time Measurement 1
PCI-e errors = Correctable Error Detected
Unsupported Request Detected
Corrected = Advisory Non-Fatal Error
VPD ident = '10/25Gb 2-port SFP28 BCM57414 Adapter'
VPD ro PN = 'P26264-001'
VPD ro MN = '14E4'
VPD ro V0 = '214.0.194.0'
VPD ro V1 = '214.0.305.0'
VPD ro V3 = '214.4.91.1'
VPD ro V6 = '214.0.241.0'
VPD ro V7 = '216.0.52.1'
VPD ro V8 = '214.4.42.1'
VPD ro VA = '216.0.333.11'
VPD ro SN = 'VNE111024Y'
VPD ro VB = 'REV005DEV000'
VPD ro V2 = '5111'
VPD ro V4 = 'E43D1A0DECA0'
bnxt1@pci0:179:0:1: class=0x020000 card=0x159714e4 chip=0x16d714e4 rev=0x01 hdr=0x00
vendor = 'Broadcom Inc. and subsidiaries'
device = 'BCM57414 NetXtreme-E 10Gb/25Gb RDMA Ethernet Controller'
class = network
subclass = ethernet
bar [10] = type Prefetchable Memory, range 64, base 0x38fffff00000, size 65536, enabled
bar [18] = type Prefetchable Memory, range 64, base 0x38ffffd00000, size 1048576, enabled
bar [20] = type Prefetchable Memory, range 64, base 0x38fffff20000, size 8192, enabled
cap 01[48] = powerspec 3 supports D0 D3 current D0
cap 03[50] = VPD
cap 05[58] = MSI supports 8 messages, 64 bit
cap 11[a0] = MSI-X supports 74 messages, enabled
Table in map 0x20[0x0], PBA in map 0x20[0x4a0]
cap 10[ac] = PCI-Express 2 endpoint max data 256(512) FLR RO NS
link x8(x8) speed 8.0(8.0) ClockPM disabled
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
ecap 0003[13c] = Serial 1 e43d1afffe0deca0
ecap 0004[150] = Power Budgeting 1
ecap 000b[180] = Vendor 1 ID 0
ecap 000e[1b8] = ARI 1
ecap 0017[230] = TPH Requester 1
PCI-e errors = Correctable Error Detected
Unsupported Request Detected
Corrected = Advisory Non-Fatal Error
VPD ident = '10/25Gb 2-port SFP28 BCM57414 Adapter'
VPD ro PN = 'P26264-001'
VPD ro MN = '14E4'
VPD ro V0 = '214.0.194.0'
VPD ro V1 = '214.0.305.0'
VPD ro V3 = '214.4.91.1'
VPD ro V6 = '214.0.241.0'
VPD ro V7 = '216.0.52.1'
VPD ro V8 = '214.4.42.1'
VPD ro VA = '216.0.333.11'
VPD ro SN = 'VNE111024Y'
VPD ro VB = 'REV005DEV000'
VPD ro V2 = '5111'
VPD ro V4 = 'E43D1A0DECA0'
bnxt0: <Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Ethernet> mem 0x38fffff10000-0x38fffff1ffff,0x38ffffe00000-0x38ffffefffff,0x38fffff22000-0x38fffff23fff irq 50 at device 0.0 numa-domain 0 on pci11
bnxt0: Using 256 TX descriptors and 256 RX descriptors
bnxt0: Using 8 RX queues 8 TX queues
bnxt0: Using MSI-X interrupts with 9 vectors
bnxt0: Ethernet address: e4:3d:1a:0d:ec:a0
bnxt1: <Broadcom BCM57414 NetXtreme-E 10Gb/25Gb Ethernet> mem 0x38fffff00000-0x38fffff0ffff,0x38ffffd00000-0x38ffffdfffff,0x38fffff20000-0x38fffff21fff irq 53 at device 0.1 numa-domain 0 on pci11
bnxt1: Using 256 TX descriptors and 256 RX descriptors
bnxt1: Using 8 RX queues 8 TX queues
bnxt1: Using MSI-X interrupts with 9 vectors
bnxt1: Ethernet address: e4:3d:1a:0d:ec:a1
bnxt1: Unknown phy type
