bhyve: pci_passthru not working

R

richardbader

Hello,

i'm trying to use pci_passthru to handover a USB interface to windows7 running under bhyve.

window7 is running perfect under bhyve.


As this does not work i tried to pass a simple network adapter, but the error message is the same when starting bhyve:

bhyve: PCI device at 5/1/0 is not using the ppt(4) driver


here the details:
Code: 
uname -a:

FreeBSD main2rest2.bader.loc 11.2-RELEASE-p10 FreeBSD 11.2-RELEASE-p10 #1 r349319: Mon Jun 24 06:27:24 CEST 2019 root@main2rest2.bader.loc:/usr/obj/usr/src/sys/MYKERNEL  amd64

cat /boot/loader.conf:
Code: 
vmm_load="YES"
pptdevs="5/1/0"
hw.vmm.topology.threads_per_core=2
hw.vmm.topology.cores_per_package=4
Code: 
pciconf -v -l:

ppt0@pci0:5:1:0:        class=0x020000 card=0x813910ec chip=0x813910ec rev=0x10 hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL-8100/8101L/8139 PCI Fast Ethernet Adapter'
    class      = network
    subclass   = ethernet

the commandline for starting bhyve:
Code: 
bhyve -S -c 7 -m 5G -w -H \
   -s 0,hostbridge\
   -s 3,ahci-cd,./empty.iso\
   -s 4,ahci-hd,./06_win7bac4.img,sectorsize=512\
   -s 5,virtio-net,tap13\
   -s 7,passthru,5/1/0\
   -s 29,fbuf,tcp=192.168.16.109:5905,w=1024,h=800\
   -s 31,lpc -l com1,stdio\
   -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd 06win7bac4

any ideas, thanks

richard
 
I have seen devices with PCI address like that that did not work.
It seems to have to do with the middle PCI bus address component.
5/1/0
I had a device with a middle number like this and it would not work.
Most all PCI addresses use something more like this:
5/0/1
With the middle address always 0

I don't know 'how or why' but I thought I would leave you with that tidbit.
Do you have another PCI/PCIe network card you could try?
Is this Realtek onboard motherboard? Only one Realtek interface or more?
Multi-port ethernet card? If so post pciconf -lv for all ethernet ports so we can see addresses.
 
One more thought.
You are showing a bhyve config with virtio-net tap and passthru network. For passthru drop the tap.
Passthru acts like bare metal adapter. No tap, bridge or tun needed.
 
I already seen that error in my numerous attempts to get the passthru working. But can't recall for sure the cause.

Have you enabled VT-d in the BIOS of your machine?
 
Emrion said:
Have you enabled VT-d in the BIOS of your machine?
Click to expand...
yes, starting bhyve without '-s 7,passthru,5/1/0\' works fine

Phishfry said:
Plus if you use AMD there are additional settings needed
Click to expand...
no its intel

Phishfry said:
-s 7:1,passthru,5/1/0
Click to expand...
same behavier

I've put the external network card in an other slot, than it appears as 5/0/0, but same behavior
with that the output of pciconf -lv:

Code: 
root@main2rest2:/usrx4_tmp_winvm # pciconf -v -l
hostb0@pci0:0:0:0:      class=0x060000 card=0x85341043 chip=0x0c008086 rev=0x06 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '4th Gen Core Processor DRAM Controller'
    class      = bridge
    subclass   = HOST-PCI
pcib1@pci0:0:1:0:       class=0x060400 card=0x85341043 chip=0x0c018086 rev=0x06 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = 'Xeon E3-1200 v3/4th Gen Core Processor PCI Express x16 Controller'
    class      = bridge
    subclass   = PCI-PCI
vgapci0@pci0:0:2:0:     class=0x030000 card=0x85341043 chip=0x04128086 rev=0x06 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Xeon E3-1200 v3/4th Gen Core Processor Integrated Graphics Controller'
    class      = display
    subclass   = VGA
hdac0@pci0:0:3:0:       class=0x040300 card=0x85341043 chip=0x0c0c8086 rev=0x06 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = 'Xeon E3-1200 v3/4th Gen Core Processor HD Audio Controller'
    class      = multimedia
    subclass   = HDA
xhci0@pci0:0:20:0:      class=0x0c0330 card=0x85341043 chip=0x8cb18086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family USB xHCI Controller'
    class      = serial bus
    subclass   = USB
none0@pci0:0:22:0:      class=0x078000 card=0x85341043 chip=0x8cba8086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family ME Interface'
    class      = simple comms
ehci0@pci0:0:26:0:      class=0x0c0320 card=0x85341043 chip=0x8cad8086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family USB EHCI Controller'
    class      = serial bus
    subclass   = USB
hdac1@pci0:0:27:0:      class=0x040300 card=0x86161043 chip=0x8ca08086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family HD Audio Controller'
    class      = multimedia
    subclass   = HDA
pcib2@pci0:0:28:0:      class=0x060400 card=0x85341043 chip=0x8c908086 rev=0xd0 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family PCI Express Root Port 1'
    class      = bridge
    subclass   = PCI-PCI
pcib3@pci0:0:28:2:      class=0x060400 card=0x85341043 chip=0x8c948086 rev=0xd0 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family PCI Express Root Port 3'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:28:3:      class=0x060401 card=0x85341043 chip=0x244e8086 rev=0xd0 hdr=0x01
    vendor     = 'Intel Corporation'
    device     = '82801 PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
ehci1@pci0:0:29:0:      class=0x0c0320 card=0x85341043 chip=0x8ca68086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family USB EHCI Controller'
    class      = serial bus
    subclass   = USB
isab0@pci0:0:31:0:      class=0x060100 card=0x85341043 chip=0x8cc48086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family Z97 LPC Controller'
    class      = bridge
    subclass   = PCI-ISA
ahci0@pci0:0:31:2:      class=0x010601 card=0x85341043 chip=0x8c828086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family SATA Controller [AHCI Mode]'
    class      = mass storage
    subclass   = SATA
none1@pci0:0:31:3:      class=0x0c0500 card=0x85341043 chip=0x8ca28086 rev=0x00 hdr=0x00
    vendor     = 'Intel Corporation'
    device     = '9 Series Chipset Family SMBus Controller'
    class      = serial bus
    subclass   = SMBus
re0@pci0:3:0:0: class=0x020000 card=0x859e1043 chip=0x816810ec rev=0x11 hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet
pcib5@pci0:4:0:0:       class=0x060401 card=0x84891043 chip=0x10801b21 rev=0x04 hdr=0x01
    vendor     = 'ASMedia Technology Inc.'
    device     = 'ASM1083/1085 PCIe to PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
ppt0@pci0:5:0:0:        class=0x020000 card=0x813910ec chip=0x813910ec rev=0x10 hdr=0x00
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL-8100/8101L/8139 PCI Fast Ethernet Adapter'
    class      = network
    subclass   = ethernet
root@main2rest2:/usrx4_tmp_winvm #
 
Are you sure you have your device right? Looking at your pciconf -lv output you should be using 5/0/0 as the ppt0 device.
So adjust your bhyve to suit.
-s 7:0,passthru,5/0/0
 
yes, starting bhyve without '-s 7,passthru,5/1/0\' works fine
Click to expand...
Your answer is ambiguous. Have you really checked this BIOS setting?

With an Intel processor and if the system crashes when you passthru a device to a VM, you need to apply this patch: PR 229852.
 
it keeps strange,
i've tried several different pci-cards but the result ist the same

bhyve: PCI device at 5/0/0 is not using the ppt(4) driver

with

ppt0@pci0:5:0:0: class=0x028000 card=0x0300168c chip=0x002d168c rev=0x01 hdr=0x00
vendor = 'Qualcomm Atheros'
device = 'AR9227 Wireless Network Adapter'
class = network

is there a chance to get some more debugging information out of bhyve?

starting a vm without passthru is working.
 
You must log in or register to reply here.
Back
Top