bhyve PCI device is not using the ppt driver when it ought to

[zeekno]

I added a GPU according to https://wiki.freebsd.org/bhyve/pci_passthru. This shows up as expected, but bhyve complains that "PCI device at 5/0/0 is not using the ppt(4) driver".

/boot/loader.conf:

kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
cryptodev_load="YES"
zfs_load="YES"
vmm_load="YES"
pptdevs="5/0/0 5/0/1"
amdtemp_load="YES"

pciconf -lv | grep -A4 ppt:

ppt0@pci0:5:0:0:    class=0x030000 rev=0xc1 hdr=0x00 vendor=0x1002 device=0x743f subvendor=0x1849 subdevice=0x5226
    vendor     = 'Advanced Micro Devices, Inc. [AMD/ATI]'
    device     = 'Navi 24 [Radeon RX 6400/6500 XT/6500M]'
    class      = display
    subclass   = VGA
ppt1@pci0:5:0:1:    class=0x040300 rev=0x00 hdr=0x00 vendor=0x1002 device=0xab28 subvendor=0x1002 subdevice=0xab28
    vendor     = 'Advanced Micro Devices, Inc. [AMD/ATI]'
    device     = 'Navi 21/23 HDMI/DP Audio Controller'
    class      = multimedia
    subclass   = HDA

bhyve -AHP -w -c 2 -S -m 1G \
    -s 0:0,hostbridge \
    -s 1:0,lpc \
    -s 2:0,virtio-net,tap0 \
    -s 3:0,virtio-blk,/dev/zvol/zroot/prison/bhyve/ubuntu/disk \
    -s 4:0,ahci-cd,./install.iso \
    -s 5:0,passthru,5/0/0 \
    -s 5:1,passthru,5/0/1 \
    -s 29,fbuf,tcp=0.0.0.0:5900,w=940,h=720,wait \
    -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd ubuntu

I am at a loss.

 

[Ole]

You need to set `hw.vmm.amdvi.enable=1` in loader.conf

 

[zeekno]

Well I put that in the wrong place. Fixed, rebooted and confirmed with sysctl hw.vmm.amdvi, but the error persists.

 

[Emrion]

Do your machine and card meet the requirements specified in the wiki page?
I mean: acpidump -t | grep DMAR and pciconf -lc | grep MSI

Edit: acpidump -t | grep DMAR gives nothing with a i5-3570 and a MSI B75MA-P45 mainboard. Yet, this processor has VT-d capability and passthru in bhyve works.

 

[zeekno]

Ah, no DMAR. Also, bad case of RTFM, sorry for the noise.

 

[Emrion]

No problem. Find and activate VT-d in your BIOS, provided your CPU supports it.

 

[zeekno]

I was just about to add, this is on a ryzen 5600x, and SVM was always enabled. So something's still off.

 

[zeekno]

Enabled IOMMU, I've got a similar mb: https://forums.freebsd.org/threads/amd-bhyve-iommu-pci-passthru.78843/

Now the bios hangs.

So I've got IOMMU and MSI, still no DMAR.

 

[zeekno]

Looks like I'm out of luck with an ASRock x570 Steel Legend. It has the option to enable DMAR, but upon reboot the BIOS is unresponsive for a while, then has no USB/SATA/peripherals of any kind. Updating didn't help. I've contacted support, let's see.