bhyve Understanding bhyve settings for VM

GlitchyDot

GlitchyDot

Trying to understand some settings and what to use;

CPU: cpu / cpu_socket / cpu_cores / cpu threads. Mainly all self explained but, i have dual cpu board, how it would use it ? cpu_socket states = sockets, which is emulated, but is there a way to use something like NUMA inside config without manual setup ? and how bhyve looks at the cpu? as in Proxmox you can choose emulation or host cpu instructions.
P.s. do i need to install mirocode inside bhyve or no ?

Loader: bhyveload / uefi / grub. bhyveload - is designed to use if VM is FreeBSD or there is no difference if i use uefi for FreeBSD and what would be difference for FreeBSD VM between these 2 ?

Graphics: graphics / graphics_res . I was trying to find some info about these settings but im suck at googling.
graphics="yes" , graphics_res="1920x1080" - this is only to set up graphics if i use package like VNCTiger etc. and res would be size of the screen inside ?

Disk: nvme / virtio-blk / file . My host is raidz2 with 4nvme`s and 2 ssd`s - so kinda mixed one, which option would be better to use ? nvme ( NVM Express (NVMe) controller. ) just in name ala emulations of nvme or if system resides on NVME drives ? file (disk0_dev=”file” ) - how to understand this ?
disk0_name=”disk0.img . disk0.img can be renamed ?

Misc: xhci_mouse. Is this to have mouse available inside VNCTiger etc ?
USB passtrough. I have mouse and keyboard - i have to use same tactics as passing gpu ? i want to pass specific ports only.
 
I can't answer your questions, but maybe this will help.
My host:
- processors
# dmidecode 3.6
# SMBIOS entry point at 0x6f4e1000
Found SMBIOS entry point in EFI, reading table from /dev/mem.
SMBIOS 3.2.1 present.

Handle 0x004D, DMI type 4, 48 bytes
Processor Information
Socket Designation: CPU1
Type: Central Processor
Family: Xeon
Manufacturer: Intel(R) Corporation
ID: 54 06 05 00 FF FB EB BF
Signature: Type 0, Family 6, Model 85, Stepping 4
Flags:
FPU (Floating-point unit on-chip)
VME (Virtual mode extension)
DE (Debugging extension)
PSE (Page size extension)
TSC (Time stamp counter)
MSR (Model specific registers)
PAE (Physical address extension)
MCE (Machine check exception)
CX8 (CMPXCHG8 instruction supported)
APIC (On-chip APIC hardware supported)
SEP (Fast system call)
MTRR (Memory type range registers)
PGE (Page global enable)
MCA (Machine check architecture)
CMOV (Conditional move instruction supported)
PAT (Page attribute table)
PSE-36 (36-bit page size extension)
CLFSH (CLFLUSH instruction supported)
DS (Debug store)
ACPI (ACPI supported)
MMX (MMX technology supported)
FXSR (FXSAVE and FXSTOR instructions supported)
SSE (Streaming SIMD extensions)
SSE2 (Streaming SIMD extensions 2)
SS (Self-snoop)
HTT (Multi-threading)
TM (Thermal monitor supported)
PBE (Pending break enabled)
Version: Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz
Voltage: 1.6 V
External Clock: 100 MHz
Max Speed: 4000 MHz
Current Speed: 2200 MHz
Status: Populated, Enabled
Upgrade: Socket LGA3647-1
L1 Cache Handle: 0x004A
L2 Cache Handle: 0x004B
L3 Cache Handle: 0x004C
Serial Number: Not Specified
Asset Tag: UNKNOWN
Part Number: Not Specified
Core Count: 14
Core Enabled: 14
Thread Count: 28
Characteristics:
64-bit capable
Multi-Core
Hardware Thread
Execute Protection
Enhanced Virtualization
Power/Performance Control

Handle 0x0051, DMI type 4, 48 bytes
Processor Information
Socket Designation: CPU2
Type: Central Processor
Family: Xeon
Manufacturer: Intel(R) Corporation
ID: 54 06 05 00 FF FB EB BF
Signature: Type 0, Family 6, Model 85, Stepping 4
Flags:
FPU (Floating-point unit on-chip)
VME (Virtual mode extension)
DE (Debugging extension)
PSE (Page size extension)
TSC (Time stamp counter)
MSR (Model specific registers)
PAE (Physical address extension)
MCE (Machine check exception)
CX8 (CMPXCHG8 instruction supported)
APIC (On-chip APIC hardware supported)
SEP (Fast system call)
MTRR (Memory type range registers)
PGE (Page global enable)
MCA (Machine check architecture)
CMOV (Conditional move instruction supported)
PAT (Page attribute table)
PSE-36 (36-bit page size extension)
CLFSH (CLFLUSH instruction supported)
DS (Debug store)
ACPI (ACPI supported)
MMX (MMX technology supported)
FXSR (FXSAVE and FXSTOR instructions supported)
SSE (Streaming SIMD extensions)
SSE2 (Streaming SIMD extensions 2)
SS (Self-snoop)
HTT (Multi-threading)
TM (Thermal monitor supported)
PBE (Pending break enabled)
Version: Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz
Voltage: 1.6 V
External Clock: 100 MHz
Max Speed: 4000 MHz
Current Speed: 2200 MHz
Status: Populated, Enabled
Upgrade: Socket LGA3647-1
L1 Cache Handle: 0x004E
L2 Cache Handle: 0x004F
L3 Cache Handle: 0x0050
Serial Number: Not Specified
Asset Tag: UNKNOWN
Part Number: Not Specified
Core Count: 14
Core Enabled: 14
Thread Count: 28
Characteristics:
64-bit capable
Multi-Core
Hardware Thread
Execute Protection
Enhanced Virtualization
Power/Performance Control

-motherboard
# dmidecode 3.6
# SMBIOS entry point at 0x6f4e1000
Found SMBIOS entry point in EFI, reading table from /dev/mem.
SMBIOS 3.2.1 present.

Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: ASUSTeK COMPUTER INC.
Product Name: WS-C621E-SAGE Series
Version: Rev 1.xx
Serial Number: ........................
Asset Tag: To Be Filled By O.E.M.
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: To Be Filled By O.E.M.
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0
...

Below are some of my VM configurations (I don't use FreeBSD as a VM, only Windows and Linux. And I don't use vm-bhyve, I use bhyve directly)
Bash: 
sudo bhyve -c 16,sockets=2,cores=2,threads=4 -m 32G -H -w -S \
  -s 0,hostbridge \
  -s 3,ahci-cd,/mirlo/iso/SSS_X64FREV_EN-US_DV9.iso \
  -s 4,ahci-hd,/mirlo/bsd/vms/ws2022_1.img \
  -s 5,ahci-cd,/mirlo/iso/virtio-win-0.1.240.iso \
  -s 6,virtio-net,tap0 \
  -s 7,passthru,94/0/0 \
  -s 8,hda,play=/dev/dsp,rec=/dev/dsp \
  -s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1200,wait \
  -s 30,xhci,tablet \
  -s 31,lpc \
  -l com1,stdio \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  ws2022_1

Bash: 
sudo bhyve -c 8,sockets=2,cores=2,threads=2 -m 16G -H -w -S \
  -s 0,hostbridge \
  -s 4,ahci-hd,/mirlo/bsd/vms/deb12.img \
  -s 5,virtio-net,tap0 \
  -s 7,passthru,94/0/0 \
  -s 8,hda,play=/dev/dsp,rec=/dev/dsp \
  -s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1200,wait \
  -s 30,xhci,tablet \
  -s 31,lpc \
  -l com1,stdio \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  deb12

Screen resolution 1920x1200 is the maximum supported by bhyve (bhyve(8))
Code: 
w=width and h=height
                   A display resolution, width and height, respec-
                   tively.    If not specified, a default resolution
                   of  1024x768 pixels will    be used.  Minimal sup-
                   ported resolution is 640x480 pixels, and     maxi-
                   mum is 1920x1200    pixels.

'cpu=sockets x cores x thteads'.

ws2022_vm.png


deb12_vm.png
 
You can generally get just specify cpu=X to choose the number of cpu cores for the guest which looks like X single core processors to the guest. The other options allow you to specify how the cpus appear to the guest (i.e appear as 2 quad core processors). I don't believe this has any real relation to the physical host, each guest cpu is just a separate thread on the host and could run on any physical processor. (It may be possible to pin virtual cpus to a physical core but I can't remember). Specifying dual socket with 8 cores each may be useful for example if you are running a Windows guest that is only licensed for dual cpu but up to 16 cores. bhyve doesn't have the option to specify the cpu type for the guest as far as i'm aware.

bhyveload was the first available when bhyve was developed with just FreeBSD support. UEFI support was added later to allow other guests to run such as Windows and now is often used for all guests, especially by management tools, as it provides a consistent process for all guests and a vnc console. I tend to use bhyveload for my FreeBSD guests as it loads the kernel directly and is very quick.

The graphics options relate only to the vnc console.

The disk options have nothing to do with the host disk type and only choose the type of emulation for the guest. The _dev option allows between using a regular file as the guest storage, a ZFS volume or a physical disk. bhyve just wants the path to something it can read/write and IIRC these options just tell vm-bhyve whether to prefix the given disk name with /path/to/guest, /dev/zvol/etc/etc, or whether to treat disk_name directly as a full path. The name of the file (e.g. disk0.img) can be changed.

xhci_mouse="yes" may not be supported by all guests (although probably is) and performs better than the default mouse. I seem to remember using the non-xhci mouse device would lag and not always line up with your physical mouse position very well in vnc.
 
usdmatt said:
xhci_mouse="yes" may not be supported by all guests (although probably is) and performs better than the default mouse.
Click to expand...
Yes, I once encountered such a problem -
A

Solved Thread 'mouse disappeared on Debian guest after kernel update in it'

Hi everyone!
Just updated kernel in Debian 12 VM (to 6.1.0-13-amd64).
After VM's reboot, mouse dissapeared in it.
bhyve VM startup script:
Bash: 
sudo bhyve -c 8,sockets=2,cores=2,threads=2 -m 16G -H -w -S  \
  -s 0,hostbridge \
  -s 4,ahci-hd,/uvero/bsd2/vms/deb12.img \
  -s 5,virtio-net,tap0 \
  -s 7,passthru,94/0/0 \
  -s 8,hda,play=/dev/dsp,rec=/dev/dsp \
  -s 29,fbuf,tcp=0.0.0.0:5900,w=1920,h=1200,wait \
  -s 30,xhci,tablet \
  -s 31,lpc \
  -l com1,stdio \
  -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
  deb12

Code: 
$ uname -aKU
FreeBSD freebsd2 13.2-RELEASE-p4 FreeBSD...
 
Well, my mouse does work inside VNC as i used during Linux install inside bhyve but as i dont need mouse , im fine i just needed to understand what that command does.
usdmatt said:
You can generally get just specify cpu=X to choose the number of cpu cores for the guest which looks like X single core processors to the guest. The other options allow you to specify how the cpus appear to the guest (i.e appear as 2 quad core processors). I don't believe this has any real relation to the physical host, each guest cpu is just a separate thread on the host and could run on any physical processor. (It may be possible to pin virtual cpus to a physical core but I can't remember). Specifying dual socket with 8 cores each may be useful for example if you are running a Windows guest that is only licensed for dual cpu but up to 16 cores. bhyve doesn't have the option to specify the cpu type for the guest as far as i'm aware.
Click to expand...
Ok, so no benefit then to emulate/use 2 cpu option for FreeBSD VM.
Yes, you can pin specific cores trough bhyve commands like Andrey Lanin config, but i use vm-bhyve and i could not find how to pin there , maybe because its not available.
I thought maybe there is other way.

usdmatt said:
bhyveload was the first available when bhyve was developed with just FreeBSD support. UEFI support was added later to allow other guests to run such as Windows and now is often used for all guests, especially by management tools, as it provides a consistent process for all guests and a vnc console. I tend to use bhyveload for my FreeBSD guests as it loads the kernel directly and is very quick.
Click to expand...
Good to know !

usdmatt said:
The disk options have nothing to do with the host disk type and only choose the type of emulation for the guest.
Click to expand...
So this is only emulation. Understood.
 
I don't see any config shown in this thread for pinning cpus but it looks like it can be done by specifying the -p option for each cpu.
For example -p 0:1 -p 2:3 to pin virtual cpus 0 & 2 to physical 1 & 3. You could do this with the bhyve_options vm-bhyve config variable which allows specifying arbitrary additional options to be passed to bhyve.
 
usdmatt said:
I don't see any config shown in this thread for pinning cpus but it looks like it can be done by specifying the -p option for each cpu.
For example -p 0:1 -p 2:3 to pin virtual cpus 0 & 2 to physical 1 & 3. You could do this with the bhyve_options vm-bhyve config variable which allows specifying arbitrary additional options to be passed to bhyve.
Click to expand...
Correct, config not shown but you can do it.
bhyve_options ? Nice. Thank You!
 
You must log in or register to reply here.
Back
Top