bhyve My GPU RTX 2080 ti and bhyve's behaviors are unusual.

ZioMario

ZioMario

Hello to everyone.

My GPU RTX 2080 ti is experiencing a particular behavior that I'm not able to understand if it is good or not. First of all,the nvidia-smi utility does not detect it. It detects only the first GPU that's attached on the PCI BUS 1 :

Code: 
root@marietto:/home/marietto # nvidia-smi -L
GPU 0: NVIDIA GeForce GTX 1060 3GB

This is the GPU that I'm using on the host os. But my PC has another 2 GPUs. Let's focus now on the 2080 ti :

Code: 
00:02.0 Display controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] (rev 02)
02:00.0 VGA compatible controller: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti] (rev a1)
02:00.1 Audio device: NVIDIA Corporation TU102 High Definition Audio Controller (rev a1)
02:00.2 USB controller: NVIDIA Corporation TU102 USB 3.1 Host Controller (rev a1)
02:00.3 Serial bus controller: NVIDIA Corporation TU102 USB Type-C UCSI Controller (rev a1)

Because it is not detected neither by the utility nv-sglrun :

Code: 
root@marietto:/home/marietto # nv-sglrun nvidia-smi
shim init
Tue Jan 10 22:23:42 2023   
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: N/A      |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0  On |                  N/A |
| 54%   31C    P5    11W / 120W |    344MiB /  3072MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                            
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      8091      G   /usr/local/libexec/Xorg           213MiB |
|    0   N/A  N/A      8167      G   xfwm4                               2MiB |
|    0   N/A  N/A      8173      G   ...local/lib/firefox/firefox      125MiB |
+-----------------------------------------------------------------------------+

NOR BY nvidia-smi :

Code: 
root@marietto:/home/marietto # nvidia-smi
Tue Jan 10 22:24:17 2023   
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.60.02    Driver Version: 510.60.02    CUDA Version: N/A      |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0  On |                  N/A |
| 53%   30C    P8     9W / 120W |    364MiB /  3072MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                            
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      8091      G   /usr/local/libexec/Xorg           233MiB |
|    0   N/A  N/A      8167      G   xfwm4                               2MiB |
|    0   N/A  N/A      8173      G   ...local/lib/firefox/firefox      125MiB |
+-----------------------------------------------------------------------------+

And now the most unusual behavior :

In /boot/loader.conf, I have commented the pptdevs line !!!

Code: 
#pptdevs="0/2/0 2/0/0 2/0/1 2/0/2 2/0/3"

what I find odd is that I CAN pass through the 2/0/0 gpu (rtx 2080 ti) on a Linux / bhyve vm having commented the pptdevs argument. Here you can see my xorg.conf file :

Ubuntu Pastebin

pastebin.ubuntu.com pastebin.ubuntu.com

I don't think that xorg.conf kidnap my 2080,because if I want,I can start a bhyve / linux vm declaring its PCI addess between the bhyve parameters without modifying the /boot/loader.conf and without rebooting and it will be passed through the VM without problems. It never happened.
 
That's interesting. the ppt driver is attached to the all the gpus that usually I pass thru,but in /boot/loader.conf, I have commented on the line #pptdevs="0/2/0 2/0/0 2/0/1 2/0/2 2/0/3". How can this be possible ?

Code: 
pciconf -l

ppt4@pci0:0:2:0:        class=0x038000 rev=0x02 hdr=0x00 vendor=0x8086 device=0x3e98 subvendor=0x1458 subdevice=0xd000
ppt0@pci0:2:0:0:        class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1e04 subvendor=0x19da subdevice=0x2503
ppt1@pci0:2:0:1:        class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f7 subvendor=0x19da subdevice=0x2503
ppt2@pci0:2:0:2:        class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad6 subvendor=0x19da subdevice=0x2503
ppt3@pci0:2:0:3:        class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad7 subvendor=0x19da subdevice=0x2503

I've detached the ppt driver from the RTX 2080 ti PCI addresses like this :

Code: 
root@marietto:/home/marietto/bhyve # devctl detach pci0:2:0:0
root@marietto:/home/marietto/bhyve # devctl detach pci0:2:0:1
root@marietto:/home/marietto/bhyve # devctl detach pci0:2:0:2
root@marietto:/home/marietto/bhyve # devctl detach pci0:2:0:3

and,as you can see,it worked :

Code: 
root@marietto:/home/marietto/bhyve # pciconf -l

none3@pci0:2:0:0:       class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1e04 subvendor=0x19da subdevice=0x2503
none4@pci0:2:0:1:       class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x10f7 subvendor=0x19da subdevice=0x2503
none5@pci0:2:0:2:       class=0x0c0330 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad6 subvendor=0x19da subdevice=0x2503
none6@pci0:2:0:3:       class=0x0c8000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1ad7 subvendor=0x19da subdevice=0x2503

but the utility nvidia-smi is still unabled to detect it. It should do this,right ?

Code: 
root@marietto:/home/marietto/bhyve # nvidia-smi -L

GPU 0: NVIDIA GeForce GTX 1060 3GB (UUID: GPU-ff7130b0-97de-d6ba-dec8-7445a7edced8)
 
You must log in or register to reply here.
Back
Top