I've been dealing with an issue for about 6 months now. Curious if anyone has ideas how to expand troubleshooting.
Summary of issue:
References:
1: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273372
2: https://markmcb.com/freebsd/vs_linux/sriov_is_first_class/
Summary of issue:
- Use any SR-IOV capable network card on a Supermicro motherboard
- Enable SR-IOV
- Create only vnet jails using SR-IOV VFs: works great
- Create only bhyve VMs using SR-IOV VFs: works great
- Create BOTH vnet jails AND bhyve VMs using SR-IOV VFs: works for about 3 minutes after starting the 2nd thing, then the host card network functionality stops. (By "2nd thing" I mean if you start the jails and also have a VF passed through to bhyve that's not started, it will work; but once you start the bhyve VM and the VF gets the driver assigned, the clock is ticking until failure.)
- Has existed in 13.1-RELEASE through 14-STABLE (possibly earlier, that's just when I noticed it).
- I have submitted a bug (273372) [1]
- I have written up exactly what I do to trigger the bug [2]
- Different motherboards and processors (all Intel + Supermicro)
- Different NICs (Intel, Chelsio, and Mellanox)
- Different PCIe slots
- Various combinations of motherboard settings that have anything to do with IOMMU
References:
1: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273372
2: https://markmcb.com/freebsd/vs_linux/sriov_is_first_class/