Solved I'm looking for an external PCI-E extension or a parameter to allows bhyve to pass some USB devices to a VM if PCIe slots are full

ZioMario

ZioMario

Hello to everyone.

I'm trying to find an hardware component that can help me to assign some of my USB devices (like a mouse and a keyboard) from the host os to a guest / bhyve VM. The easier solution I can think at the moment is a "USB 3.0 PCI-E Express USB 3.0 Expansion Adapter". But the problem is that actually my two PCI-E buses are occupied by the two GPUs I have. I haven't any additional PCI-E line. I tried to attach the mouse and keyboard to use within the bhyve VM to the USB port which belong to one of my GPU,the 2080 ti infact has an USB 3.1 port,instead the other GPU,the 1060 does not have it. Unfortunately this solution does not work because I can't attach that GPU to the bhyve VM,because I need it elsewhere. I don't realize which hardware component can I buy to attach my USB devices to the bhyve VM. I know for sure that it should be a PCI-E device,because bhyve does not allow to pass the USB devices if they aren't attached to a PCI-E bus and I can't assign the main PCI-E bus of my PC to a bhyve VM !. So,maybe all around there is an something like an external PCI-E device that can be attached to a USB port ? Or I have thought about an HDMI to USB converter because my geforce 1060 has some additional HDMI ports that I don't use. And I've found the product that does it,but it's not for my case,it works only to bring the HDMI signal to a video device to another one. Someone has the right product for my scenario in mind ? thanks.
 
There are some "external PCI expansion" solutions available. I wouldn't touch them with someone else's stick but you asked so...

www.startech.com

PCI Express to 4 Slot PCI Expansion System - TAA

Add four external PCI expansion card slots to a desktop or laptop computer system
www.startech.com www.startech.com

esons.ch

www.esons.ch www.esons.ch
Note that those DVI, USB or whatever physical connection they use are merely to physically pass through the PCI lane(s). Hence these tend to "work" without any drivers.
 
Last edited:
Thanks. The first one is out of discussion because it costs more than one kidney. The second me seems to be interesting. But Im not sure if bhyve will recognize it as a PCI-e device and not as an USB device. In the latter case it will not work. And anyway,I don't understand where should I attach the second product. Maybe to an USB 3.1 port ?
 
ziomario said:
The first one is out of discussion because it costs more than one kidney.
Click to expand...
You might be surprised what a kidney actually costs...

ziomario said:
The second me seems to be interesting. But Im not sure if bhyve will recognize it as a PCI-e device and not as an USB device.
Click to expand...
Hence my explanation: It's not a USB device and it never will be. The USB cable & connectors are only used to physically pipe through a PCIe 1x lane.

In any case, you will find "barebone" solutions like the latter of my examples easily from other sources. Especially now that the crypto minig scene seems to dump hardware left and right you might be able to get some for dirt cheap as these are/were very popular to hook up more GPUs than you have physical PCIe slots available on your mainboard.
 
ziomario said:
Can you suggest some device of the same kind to chose from ?
Click to expand...
I don't understand what you expect. I already made suggestions. There are plenty of those out there. Just browse the usual places on the web for similar product names.

ziomario said:
I'm not experienced at all with the crypto mining.
Click to expand...
Neither am I but you don't have to be. Those kind of kits (like the 2nd one I linked) are very popular in GPU based mining rigs as they allow to connect four GPUs to one motherboard PCIe slot. You'll pay in performance (as you can see all of these are just 1x) but you get multiple slots in return. Now that the golden age of bitcoin mining seem to be over I assume that you can get those kits used for dirt cheap just like you can get entire mining rigs for dirt cheap.
What I understood from your opening post is that you'd like to add a PCIe card to your computer which provides an additional USB controller (and matching USB ports on the back) so you can pass-through that USB controller to a bhyve guest but that you don't have enough slots free on your machine. A product like the one I linked simply gives you four PCIe slots for the price of one (when just counting slots, again, these are usually just 1x but that should be plenty to pass through some USB HID devices as you mentioned in your opening post.

In any case, please bear in mind that this is a FreeBSD support forum and these products have nothing to do with FreeBSD. Not even driver wise as they are just a physical extension/split. Your OS won't know whether you're using one of these god awful abominations or not.
 
I asked it because it should work on FreeBSD,not in a different OS. I'm not sure that any product of that kind will work for sure without problems with FreeBSD.
 
Please re-read the last paragraph of my last post again.

Most of these are just a "dump" electrical thing. They don't do anything "communication" wise themselves. Some of them are at most comparable to a USB switch. Your OS doesn't know or care whether your USB memory stick is connected directly to a port on your motherboard or through a USB hub (Yes, the USB protocol itself carries this information but for different purposes). And a lot of these products are really just a bunch of fancy wires from a communication point of view. They use USB or DVI cables to connect but not because they use USB or DVI. It's just that these PCIe lanes have somewhat strict electrical requirements. And it just so happens that those requirements are similar to those of USB or DVI. Therefore, they can just sell you existing USB or DVI cables which are dirt cheap because they are produced by the billions instead of inventing a new cable/connector or using very expensive, rarely produced ones.
Check out the pictures they show you. The PCB that goes into your mainboard just has a USB connector on it and the traces on the PCB go from the PCIe connector directly to the USB connector. There is no fancy magic chip wizardry happening at all.
But because one cannot really route (enough) power through those USB or DVI connectors, the adapter piece at the other end (which ultimately provides the PCIe socket you plug your actual PCIe expansion cards in) have some power connector to help out with that.
 
See if "pciconf" reveals a USB hub not in use but physically accessible. You can then passthrough that PCI device.
For instance on one machine I get

$ pciconf -l | grep hci
xhci0@pci0:4:0:3: class=0x0c0330 rev=0x00 hdr=0x00 vendor=0x1022 device=0x15e0 subvendor=0x1d05 subdevice=0x1077
xhci1@pci0:4:0:4: class=0x0c0330 rev=0x00 hdr=0x00 vendor=0x1022 device=0x15e1 subvendor=0x1d05 subdevice=0x1077
ahci0@pci0:5:0:0: class=0x010601 rev=0x61 hdr=0x00 vendor=0x1022 device=0x7901 subvendor=0x1d05 subdevice=0x1077
 
bakul said:
See if "pciconf" reveals a USB hub not in use but physically accessible. You can then passthrough that PCI device.
Click to expand...
Unless I'm mistaken what you're looking at is not a USB hub but rather a USB controller. While you can pass through a USB controller you cannot pass through a USB Hub.
 
Code: 
$ pciconf -l | grep hci

xhci1@pci0:0:20:0:    class=0x0c0330 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa36d subvendor=0x1458 subdevice=0x5007
ahci0@pci0:0:23:0:    class=0x010601 rev=0x10 hdr=0x00 vendor=0x8086 device=0xa352 subvendor=0x1458 subdevice=0xb005

what are the negative consequence if I pass thru those pci addresses to a bhyve vm ? Will I lose the control of some devices ?
 
As you will be passing through the entire USB controller any USB device attached/connected to that controller will be passed through to the VM. The host essentially loses access to those USB devices.
But that is the war plan anyway. It's not like you have any other option. Hence you were asking for a solution to add another PCIe USB controller to your computer.
 
As far as I can tell ziomario knows that he can/must pass through a USB controller. Hence he is specifically asking for a hardware solution / product to hook up more PCIe extension cards to his computer without having enough empty PCIe slots on the mainboard.
 
jbo said:
As far as I can tell ziomario knows that he can/must pass through a USB controller. Hence he is specifically asking for a hardware solution / product to hook up more PCIe extension cards to his computer without having enough empty PCIe slots on the mainboard.
Click to expand...

exactly. what to do when the onboard pcie slots are all occupied ? I've also asked to a bhyve developer and he gave me a software solution,that I didn't know :

If it’s only mouse and keyboard you could use bhyves virtio-input emulation.
As it’s one of my bhyve versions it’s likely to include virtio-input support.
Check “man bhyve”.
I’ve committed virtio-input to 13.2 and 14.0 upstream.
So, when these versions are released, virtio-input will be supported.

From “man bhyve” (https://www.freebsd.org/cgi/man.cgi...FreeBSD+14.0-CURRENT&arch=default&format=html):

Virtio input device backends:

/dev/input/eventX

Send input events of /dev/input/eventX to guest

by VirtIO Input Interface.
Click to expand...

It sounds nice. Unfortunately I didn't understand how it works. I hope someone of you can explain to me how to use it. Should I add "/dev/input/eventX" parameter inside the bhyve parameters like this ?

Code: 
bhyve -S -c sockets=2,cores=2,threads=2 -m 8G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/dev/$vmdisk7,bootindex=1 \
-s 4,fbuf,tcp=0.0.0.0:5915,w=1600,h=950 \
-s 8:0,passthru,1/0/0 \
-s 8:1,passthru,1/0/1 \
-s 9,/dev/input/eventX \
-s 10,virtio-net,tap15 \
-s 11,virtio-9p,sharename=/ \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm15 < /dev/null & sleep 2 && vncviewer 0:15

but what's eventX ? how can I know what's the value to give to X ? thanks.
 
jbo said:
Those are your input devices.


I think that xinput(1) should be able to help here.
Click to expand...

not me because we are going into a lot of technicalities. I love technicalities when I don't see a compact list of parameters unrelated to practical examples. Unfortunately the man pages are formatted in the exact way that I'm not able to understand what to do.
 
it seems that the parameter to add is :

-s 9,virtio-input,/dev/input/eventX

what I didn't understand is if eventX should be left as is or if X should assume some value. And anyway,it captures the movements of the mouse and keyboard used on the host os ?
 
ziomario said:
not me because we are going into a lot of technicalities. I love technicalities when I don't see a compact list of parameters unrelated to practical examples. Unfortunately the man pages are formatted in the exact way that I'm not able to understand what to do.
Click to expand...
I don't mean to be rude or anything but this seems to be a reoccurring issue. I think you're best of learning how to read and use man pages first if you want to continue using FreeBSD.
 
jbo said:
I don't mean to be rude or anything but this seems to be a reoccurring issue. I think you're best of learning how to read and use man pages first if you want to continue using FreeBSD.
Click to expand...

I don't agree with that. Luckily until now I have reached all my achievements by asking (here and on reddit) and reading more focused tutorials found on internet and on youtube. I suggest you to don't fall as a victim of the "generalization" mental bias. The error that you make is to think that I'm not able to read the man pages. But I know that what is missing to me is a solid theory on top of my shoulders and experience. After all I've been trained as a psychologist,not as a system administrator. This is an hobby for me. I like it and I try to do th best as I can,but my knowledge gaps are many. Furthermore, I believe that there must be some sort of correspondence between how the human's mind processes informations and the form in which they are shown. Different forms correspond to differently effective ability of learning. Here is where you can see most where my skills lie,when we start talking about how the human's brain work.
 
It does not work :

Code: 
bhyve -S -c sockets=2,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 1,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/focal-stable-diffusion.img,bootindex=1 \
-s 2,virtio-blk,/dev/$vmdisk3 \
-s 8:0,passthru,1/0/0 \
-s 8:1,passthru,1/0/1 \
-s 9,virtio-input,/dev/input/eventX \
-s 11,virtio-net,tap20 \
-s 12,virtio-9p,sharename=/ \
-s 13,hda,play=/dev/dsp,rec=/dev/dsp \
-s 29,fbuf,tcp=0.0.0.0:5920,w=1600,h=950 \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm0:20 < /dev/null & sleep 2 && vncviewer 0:20

pci slot 0:9:0: unknown device "virtio-input"
 
You must log in or register to reply here.
Back
Top