bhyve PCI device not masked in AMD host

L

lib13

For my system,

Code: 
# sysctl -a | egrep -i 'hw.machine|hw.model|hw.ncpu'
hw.machine: amd64
hw.model: AMD Ryzen 5 1600 Six-Core Processor           
hw.ncpu: 12
hw.machine_arch: amd64

as said in the bhyve/pci_passthru wiki page, I've included the line
Code: 
hw.vmm.amdvi.enable=1
in /boot/loader.conf.

After a reboot, that instruction is effective as shown by:
Code: 
# sysctl hw.vmm.amdvi.enable
hw.vmm.amdvi.enable: 1

I want to passthrough the 3.1 USB controller to a bhyve Linux vm.

Code: 
pciconf -lv
shows the controller:
Code: 
xhci0@pci0:1:0:0:       class=0x0c0330 rev=0x01 hdr=0x00 vendor=0x1022 device=0x43d5 subvendor=0x1b21 subdevice=0x1142
    vendor     = 'Advanced Micro Devices, Inc. [AMD]'
    device     = '400 Series Chipset USB 3.1 XHCI Controller'
    class      = serial bus
    subclass   = USB

This is meant for an external USB disk to be passed through to the Linux virtual machine.

In /boot/loader.conf I've included the line
Code: 
pptdevs="1/0/0"
as per the instructions of the wiki page.

The problem is that the USB 3.1 controller is never masked on the host and, thus, cannot be passed to the Linux virtual machine.

What can I do to solve this issue?
 
Motherboard USB can be problematic to pass thru on any arch.
My solution was to add NEC based PCIe USB3 cards for each VM.
That worked perfectly and can detect hotplug Mass Storage.

I think in your case XHCI controls all the USB3 ports. You can't split them up.
If you have any USB3 PCIe cards give that a try.

The weird thing is on Intel E5 Xeon the USB ports show up as lower PCI addresses.
Along with the other motherboard perephrials.
For example:
pci0:0:29:0
pci0:0:30:0

Yours seems to be showing a top level PCI address 1:0:0 for USB.
Those on my boards are the PCIe slots. 1:0:0, 2:0:0 ect.. ect..
 
This is the thing that is preventing me from having FreeBSD as my desktop system. There are some services and applications that need Linux, as the Brother printer and scanner.

I think I've got a spare USB card (maybe only 2.0), I'll give it a try.
Thank you for your suggestion @Phishfry

Here's the complete
Code: 
pciconf -vl

hostb0@pci0:0:0:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1450 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Root Complex' class = bridge subclass = HOST-PCI none0@pci0:0:0:2: class=0x080600 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1451 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) I/O Memory Management Unit' class = base peripheral subclass = IOMMU hostb1@pci0:0:1:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1452 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge' class = bridge subclass = HOST-PCI pcib1@pci0:0:1:3: class=0x060400 rev=0x00 hdr=0x01 vendor=0x1022 device=0x1453 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) PCIe GPP Bridge' class = bridge subclass = PCI-PCI hostb2@pci0:0:2:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1452 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge' class = bridge subclass = HOST-PCI hostb3@pci0:0:3:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1452 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge' class = bridge subclass = HOST-PCI pcib8@pci0:0:3:1: class=0x060400 rev=0x00 hdr=0x01 vendor=0x1022 device=0x1453 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) PCIe GPP Bridge' class = bridge subclass = PCI-PCI hostb4@pci0:0:4:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1452 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge' class = bridge subclass = HOST-PCI hostb5@pci0:0:7:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1452 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge' class = bridge subclass = HOST-PCI pcib9@pci0:0:7:1: class=0x060400 rev=0x00 hdr=0x01 vendor=0x1022 device=0x1454 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B' class = bridge subclass = PCI-PCI hostb6@pci0:0:8:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1452 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-1fh) PCIe Dummy Host Bridge' class = bridge subclass = HOST-PCI pcib10@pci0:0:8:1: class=0x060400 rev=0x00 hdr=0x01 vendor=0x1022 device=0x1454 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B' class = bridge subclass = PCI-PCI intsmb0@pci0:0:20:0: class=0x0c0500 rev=0x59 hdr=0x00 vendor=0x1022 device=0x790b subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'FCH SMBus Controller' class = serial bus subclass = SMBus isab0@pci0:0:20:3: class=0x060100 rev=0x51 hdr=0x00 vendor=0x1022 device=0x790e subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'FCH LPC Bridge' class = bridge subclass = PCI-ISA hostb7@pci0:0:24:0: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1460 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0' class = bridge subclass = HOST-PCI hostb8@pci0:0:24:1: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1461 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1' class = bridge subclass = HOST-PCI hostb9@pci0:0:24:2: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1462 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2' class = bridge subclass = HOST-PCI hostb10@pci0:0:24:3: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1463 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3' class = bridge subclass = HOST-PCI hostb11@pci0:0:24:4: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1464 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4' class = bridge subclass = HOST-PCI hostb12@pci0:0:24:5: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1465 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5' class = bridge subclass = HOST-PCI hostb13@pci0:0:24:6: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1466 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 6' class = bridge subclass = HOST-PCI hostb14@pci0:0:24:7: class=0x060000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1467 subvendor=0x0000 subdevice=0x0000 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7' class = bridge subclass = HOST-PCI xhci0@pci0:1:0:0: class=0x0c0330 rev=0x01 hdr=0x00 vendor=0x1022 device=0x43d5 subvendor=0x1b21 subdevice=0x1142 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset USB 3.1 XHCI Controller' class = serial bus subclass = USB ahci0@pci0:1:0:1: class=0x010601 rev=0x01 hdr=0x00 vendor=0x1022 device=0x43c8 subvendor=0x1b21 subdevice=0x1062 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset SATA Controller' class = mass storage subclass = SATA pcib2@pci0:1:0:2: class=0x060400 rev=0x01 hdr=0x01 vendor=0x1022 device=0x43c6 subvendor=0x1b21 subdevice=0x0201 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset PCIe Bridge' class = bridge subclass = PCI-PCI pcib3@pci0:2:0:0: class=0x060400 rev=0x01 hdr=0x01 vendor=0x1022 device=0x43c7 subvendor=0x1b21 subdevice=0x3306 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset PCIe Port' class = bridge subclass = PCI-PCI pcib4@pci0:2:4:0: class=0x060400 rev=0x01 hdr=0x01 vendor=0x1022 device=0x43c7 subvendor=0x1b21 subdevice=0x3306 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset PCIe Port' class = bridge subclass = PCI-PCI pcib5@pci0:2:5:0: class=0x060400 rev=0x01 hdr=0x01 vendor=0x1022 device=0x43c7 subvendor=0x1b21 subdevice=0x3306 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset PCIe Port' class = bridge subclass = PCI-PCI pcib6@pci0:2:6:0: class=0x060400 rev=0x01 hdr=0x01 vendor=0x1022 device=0x43c7 subvendor=0x1b21 subdevice=0x3306 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset PCIe Port' class = bridge subclass = PCI-PCI pcib7@pci0:2:7:0: class=0x060400 rev=0x01 hdr=0x01 vendor=0x1022 device=0x43c7 subvendor=0x1b21 subdevice=0x3306 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = '400 Series Chipset PCIe Port' class = bridge subclass = PCI-PCI re0@pci0:7:0:0: class=0x020000 rev=0x15 hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x1043 subdevice=0x8677 vendor = 'Realtek Semiconductor Co., Ltd.' device = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller' class = network subclass = ethernet vgapci0@pci0:8:0:0: class=0x030000 rev=0x00 hdr=0x00 vendor=0x1002 device=0x6779 subvendor=0x1043 subdevice=0x3015 vendor = 'Advanced Micro Devices, Inc. [AMD/ATI]' device = 'Caicos [Radeon HD 6450/7450/8450 / R5 230 OEM]' class = display subclass = VGA hdac0@pci0:8:0:1: class=0x040300 rev=0x00 hdr=0x00 vendor=0x1002 device=0xaa98 subvendor=0x1043 subdevice=0xaa98 vendor = 'Advanced Micro Devices, Inc. [AMD/ATI]' device = 'Caicos HDMI Audio [Radeon HD 6450 / 7450/8450/8490 OEM / R5 230/235/235X OEM]' class = multimedia subclass = HDA none1@pci0:9:0:0: class=0x130000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x145a subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Zeppelin/Raven/Raven2 PCIe Dummy Function' class = non-essential instrumentation none2@pci0:9:0:2: class=0x108000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1456 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) Platform Security Processor' class = encrypt/decrypt xhci1@pci0:9:0:3: class=0x0c0330 rev=0x00 hdr=0x00 vendor=0x1022 device=0x145f subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Zeppelin USB 3.0 Host controller' class = serial bus subclass = USB none3@pci0:10:0:0: class=0x130000 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1455 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Zeppelin/Renoir PCIe Dummy Function' class = non-essential instrumentation ahci1@pci0:10:0:2: class=0x010601 rev=0x51 hdr=0x00 vendor=0x1022 device=0x7901 subvendor=0x1043 subdevice=0x8747 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'FCH SATA Controller [AHCI mode]' class = mass storage subclass = SATA hdac1@pci0:10:0:3: class=0x040300 rev=0x00 hdr=0x00 vendor=0x1022 device=0x1457 subvendor=0x1043 subdevice=0x86c7 vendor = 'Advanced Micro Devices, Inc. [AMD]' device = 'Family 17h (Models 00h-0fh) HD Audio Controller' class = multimedia subclass = HDA
 
Much different output there than Intel buses.
Any USB card will work fine for printers and scanners.
I had four GT710/GT730 video cards and four USB3 cards installed running four virtual FreeBSD desktops..
That left very few slots for 10G cards and NVMe even on the 11 slot Supermicro X10DRX.
Might be nice for a 4 man office with one server. Not my deal.
 
You must log in or register to reply here.
Back
Top