Hi all,
I’m finally upgrading to 10Gbe with a pair of Chelsio T420-CR cards and one of my goals is to get off of `bridge` and `epair` devices for my jails and instead use VF devices from SR-IOV.
However, when trying to enable these virtual devices the drivers don’t attach. What I have so far:
12.0-RELEASE p3
Intel Xeon 1245v3
Then I have an
I can confirm that this enables the VF portions of the card:
However, when I look at
So it looks like the drivers can’t get any interrupts for the interfaces. Looking at other entries in
So, anyone have any thoughts? MSI-X appears to be an automatic thing on FreeBSD, should I be tweaking the BIOS or something?
thanks!
I’m finally upgrading to 10Gbe with a pair of Chelsio T420-CR cards and one of my goals is to get off of `bridge` and `epair` devices for my jails and instead use VF devices from SR-IOV.
However, when trying to enable these virtual devices the drivers don’t attach. What I have so far:
12.0-RELEASE p3
Intel Xeon 1245v3
cat /boot/loader.conf
Code:
zfs_load="YES"
t4fw_cfg_load="YES"
t5fw_cfg_load="YES"
if_cxgbe_load="YES"
if_cxgbev_load="YES"
t4_tom_load="YES"
iovctl
config file for my device: cat /etc/iovctl.cxgbe0.conf
Code:
PF {
device : "cxgbe0";
num_vfs : 4;
}
doas pciconf -lvc
Code:
t4iov0@pci0:1:0:0: class=0x020000 card=0x00001425 chip=0x40011425 rev=0x00 hdr=0x00
vendor = 'Chelsio Communications Inc'
device = 'T420-CR Unified Wire Ethernet Controller'
class = network
subclass = ethernet
cap 01[40] = powerspec 3 supports D0 D3 current D0
cap 05[48] = MSI supports 8 messages, 64 bit, vector masks
cap 11[60] = MSI-X supports 8 messages
Table in map 0x20[0x0], PBA in map 0x20[0x1000]
cap 10[6c] = PCI-Express 2 endpoint max data 256(2048) FLR NS
link x8(x8) speed 5.0(5.0) ASPM disabled(L0s/L1)
cap 03[a8] = VPD
ecap 0001[100] = AER 1 0 fatal 0 non-fatal 1 corrected
ecap 0008[138] = MFVC 1
ecap 0003[1b8] = Serial 1 0000000000000000
ecap 0010[1c4] = SR-IOV 1 IOV enabled, Memory Space enabled, ARI disabled
4 VFs configured out of 16 supported
First VF RID Offset 0x0008, VF RID Stride 0x0004
VF Device ID 0x4801
Page Sizes: 4096 (enabled)
iov bar [1e8] = type Memory, range 64, base 0xef000000, size 4096, enabled
iov bar [1f8] = type Memory, range 64, base 0xef004000, size 8192, enabled
ecap 000e[204] = ARI 1
ifconfig
I don’t see any new interfaces. Sure enough, if I look at dmesg
I see 4 entries of the following:
Code:
t4vf0: <Chelsio T420-CR VF> at device 1.0 on pci1
t4vf0: failed to find a usable interrupt type. allowed=7, msi-x=0, msi=0, intx=1device_attach: t4vf0 attach returned 6
pciconf
I do see several using MSI and MSI-X, so it’s not totally broken, but it might be interesting that the Chelsio devices don’t have MSI/MSI-X enabled...So, anyone have any thoughts? MSI-X appears to be an automatic thing on FreeBSD, should I be tweaking the BIOS or something?
thanks!