bhyve vm-bhyve Ubuntu 24.04 installed

NapoleonWils0n

NapoleonWils0n

Ubuntu 24.04 installed with vm-bhyve after a bit of work

github.com

cerberus/freebsd/vm-bhyve.org at master · NapoleonWils0n/cerberus

cerberus code library. Contribute to NapoleonWils0n/cerberus development by creating an account on GitHub.
github.com github.com

install packages

Code: 
doas pkg install bhyve-firmware grub2-bhyve edk2-bhyve vm-bhyve-devel

create the zfs datasets

Code: 
doas zfs create -o mountpoint=/vm zroot/vm
doas zfs create -o mountpoint=/vm/.iso zroot/vm/iso
doas zfs create -o mountpoint=/vm/.templates zroot/vm/templates

rc.conf

Code: 
doas sysrc kld_list+="vmm nmdm"
doas sysrc vm_enable=YES
doas sysrc vm_dir="zfs:zroot/vm"

kldload

only needed if you havent rebooted, and not needed after that

Code: 
doas kldload vmm nmdm

networking,
pinched the tips from vermaden

vermaden.wordpress.com

FreeBSD Bhyve Virtualization

The Bhyve FreeBSD hypervisor (called/spelled ‘beehive’ usually) was created almost 10 years ago. Right now it offers speed and features that other similar solutions provide – such…
vermaden.wordpress.com vermaden.wordpress.com

gateway enable

Code: 
doas sysrc gateway_enable=YES

sysctl

Code: 
doas sysctl net.link.tap.up_on_open=1
doas sysctl net.inet.ip.forwarding=1

set options permanently

Code: 
doas echo "net.link.tap.up_on_open=1" >> /etc/sysctl.conf
doas echo "net.inet.ip.forwarding=1" >> /etc/sysctl.conf

copy templates

Code: 
doas cp -a /usr/local/share/examples/vm-bhyve /vm/.templates

doas set up

edit doas.conf and add a rule for the vm command

Code: 
doas vi /usr/local/etc/doas.conf

change username to your username

Code: 
# vm-bhyve
permit nopass :username cmd vm

start the vm service
only needed if you havent rebooted, and not needed after that


Code: 
doas service vm start

vm init

Code: 
doas vm init

networking

change lagg0 to your network interface

Code: 
doas vm switch create public
doas vm switch add public lagg0

fetch the iso

i found the vm iso command stopped downloading the iso part way through,
so instead i used fetch to download the iso

Code: 
doas fetch -o /vm/.iso/ubuntu-24.04.1-live-server-amd64.iso 'https://releases.ubuntu.com/24.04.1/ubuntu-24.04.1-live-server-amd64.iso'

find the number of cpu's on your machine

Code: 
sysctl kern.sched.topology_spec

edit the ubuntu template

Code: 
doas vi /vm/.templates/ubuntu.conf

i have set the cpu to 6 as i have 12 cpus
and the memory to 8G because i have 16G

im need a bit of power for the apps im going to be running

Code: 
loader="grub"
cpu=6
memory=8G
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
grub_run_partition="2"
grub_run_dir="/grub"

this is the important part of the config

Code: 
grub_run_partition="2"
grub_run_dir="/grub"

if you dont have those two option you will be dumped into the grub prompt
and have to load some stuff manually which didnt work for me

create the vm

Code: 
doas vm create -t ubuntu -s 40G ubuntu

-t is the template name
-s is the size
ubuntu is the name of the vm

vm install

use the -f option to run the vm in the current terminal

Code: 
doas vm install -f ubuntu ubuntu-24.04.1-live-server-amd64.iso

just got everything installed
so i thought id share my notes so far
 
Thank you for sharing this.

NapoleonWils0n said:
if you dont have those two option you will be dumped into the grub prompt
and have to load some stuff manually which didnt work for me
Click to expand...
You should be able to prevent this issue just by using UEFI instead:
github.com

Ubuntu 24.04 guest requires the grub run partition setting · Issue #550 · churchers/vm-bhyve

Starting with the Ubuntu 24.04 server (amd64 ISO) the installation completes successfully. This is using the suggested "Debian and Ubuntu (Grub)" section on the Wiki page However when rebooting, th...
github.com github.com
 
cheers mate
ill try using uefi instead

still good to work out what the issue with grub is
im not going to be using graphics so grub will do for now

i want to pass through my gpu
and install the nvidia driver and cuda

however i only have 1 gpu

i know if you have 2 gpus you can "blacklist" one of them
and then pass it though to the vm

just wondering how to go about passing through the gpu
without "blacklisting" it on Freebsd

i guess you can use the vm passthru option

on the nvidia side on things

with jails you have to manually install the exact version of the nvidia driver used on Freebsd for it to work

just wondering if anyone has passed through a single gpu
and what the general set up is

i was hoping to install janus-pro

github.com

GitHub - deepseek-ai/Janus: Janus-Series: Unified Multimodal Understanding and Generation Models

Janus-Series: Unified Multimodal Understanding and Generation Models - deepseek-ai/Janus
github.com github.com

and whisperx

github.com

GitHub - m-bain/whisperX: WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)

WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization) - m-bain/whisperX
github.com github.com

googling for information these days is awful
so if anyone can give me a kick in the right direction that would be great
 
vm passthru

Code: 
doas vm passthru

Code: 
DEVICE     BHYVE ID     READY        DESCRIPTION
hostb0     0/0/0        No           8th Gen Core Processor Host Bridge/DRAM Registers
pcib1      0/1/0        No           6th-10th Gen Core Processor PCIe Controller (x16)
vgapci1    0/2/0        No           CoffeeLake-H GT2 [UHD Graphics 630]
none0      0/4/0        No           Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Thermal Subsystem
none1      0/8/0        No           Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model
pchtherm0  0/18/0       No           Cannon Lake PCH Thermal Controller
xhci0      0/20/0       No           Cannon Lake PCH USB 3.1 xHCI Host Controller
none2      0/20/2       No           Cannon Lake PCH Shared SRAM
ig4iic0    0/21/0       No           Cannon Lake PCH Serial IO I2C Controller
ig4iic1    0/21/1       No           Cannon Lake PCH Serial IO I2C Controller
none3      0/22/0       No           Cannon Lake PCH HECI Controller
ahci0      0/23/0       No           Cannon Lake Mobile PCH SATA AHCI Controller
pcib2      0/27/0       No           Cannon Lake PCH PCI Express Root Port
pcib7      0/28/0       No           Cannon Lake PCH PCI Express Root Port
pcib8      0/28/4       No           Cannon Lake PCH PCI Express Root Port
pcib9      0/29/0       No           Cannon Lake PCH PCI Express Root Port
isab0      0/31/0       No           Cannon Lake LPC Controller
hdac0      0/31/3       No           Cannon Lake PCH cAVS
ichsmb0    0/31/4       No           Cannon Lake PCH SMBus Controller
none4      0/31/5       No           Cannon Lake PCH SPI Controller
vgapci0    1/0/0        No           TU117M [GeForce GTX 1650 Mobile / Max-Q]
pcib3      2/0/0        No           JHL6340 Thunderbolt 3 Bridge (C step) [Alpine Ridge 2C 2016]
pcib4      3/0/0        No           JHL6340 Thunderbolt 3 Bridge (C step) [Alpine Ridge 2C 2016]
pcib5      3/1/0        No           JHL6340 Thunderbolt 3 Bridge (C step) [Alpine Ridge 2C 2016]
pcib6      3/2/0        No           JHL6340 Thunderbolt 3 Bridge (C step) [Alpine Ridge 2C 2016]
none5      4/0/0        No           JHL6340 Thunderbolt 3 NHI (C step) [Alpine Ridge 2C 2016]
xhci1      58/0/0       No           JHL6340 Thunderbolt 3 USB 3.1 Controller (C step) [Alpine Ridge 2C 2016]
iwlwifi0   59/0/0       No           Wi-Fi 6 AX200
rtsx0      60/0/0       No           RTS525A PCI Express Card Reader
nvme0      61/0/0       No           2300 NVMe SSD [Santana]

Code: 
nvidia-smi

Code: 
Wed Jan 29 15:34:16 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.127.05             Driver Version: 550.127.05     CUDA Version: N/A      |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce GTX 1650        Off |   00000000:01:00.0 Off |                  N/A |
| N/A   47C    P8              2W /   50W |       1MiB /   4096MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

so i guess this is the gpu

Code: 
vgapci0    1/0/0        No           TU117M [GeForce GTX 1650 Mobile / Max-Q]
 
the READY column lists devices that are ready to be passed through
and the gpu is listed as No because i havent blacklisted it

im sure someone on the forum mention its possible to use the gpu without blacklisting it
but my memory may be getting a bit fuzzy

just watching this video to see if it has any tips

 
no joy

ubuntu.conf

Code: 
loader="grub"
cpu=6
memory=8G
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
grub_run_partition="2"
grub_run_dir="/grub"

tried both these options
1/0/0 is the gpu on the freebsd host

1 - passthru0

Code: 
passthru0="1/0/0=7:0"

2 - bhyve_option

Code: 
bhyve_options="-s 2:0,passthru,1/0/0"

the vm wouldnt boot

i cant mask the gpu because i only have 1 gpu on the machine

monwarez mentioned the following
have to read up on that

Just need to implement the virtio-gpu with bhyve as host, using virglrenderer with the drm-renderers option.
Click to expand...

monwarez

Post in thread 'Ollama working with deepseek-r1, deepseek-coder, mistral, Nvidia gpu and emacs with gptel on Freebsd 14.2'

NapoleonWils0n said:
thinking out load

i wonder if torch will work in a linux byhve vm with my gpu

however i only have one gpu,
so i cant disable the gpu to pass through to the byhve vm
Click to expand...
Just need to implement the virtio-gpu with bhyve as host, using virglrenderer with the drm-renderers option.
  • Thanks
 
podman works

Code: 
sudo apt install podman podman-compose

Code: 
podman run quay.io/podman/hello

podman.png
 
You must log in or register to reply here.
Back
Top