Solved Network bridge set up for bhyve debian & ubuntu vm instances

Sivan!

Sivan!

In freebsd 13.1 release p1, I have installed bhyve following instructions from this page. a minimal version of debian, the installation went well except that the bridge specified did not work. I could finish installation, reboot the vm, and also could set up a script as outlined in Step 8 of the instructions from the same page, all that worked fine.

bhyve debian installation Screenshot_20220815_164704.png


When restarting I found these errors:

# /home/user/debianvm

the above is the location of the script file, containing

#!/bin/sh
ifconfig tap0 create
ifconfig bridge1 addm tap0
if ! kldstat | grep -w vmm.ko
then
kldload -v vmm
fi
if ! kldstat | grep -w nmdm.ko
then
kldload -v nmdm
fi
bhyve -c 1 -m 1G -w -H \
-s 0,hostbridge \
-s 4,virtio-blk,/dev/zvol/zroot/debianvm \
-s 5,virtio-net,tap0 \
-s 29,fbuf,tcp=0.0.0.0:5900,w=1024,h=768 \
-s 30,xhci,tablet \
-s 31,lpc -l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
debianvm
Click to expand...
vm starts and restarts without network, bridge0 / 1 tap0 and the links created do not work.
Click to expand...

I also downloaded a ubuntu and when I tried to install the ubuntu vm there are similar errors.

bhyve reboot errors Screenshot_20220815_180023.png

Kernel Security: I lowered kernel security to complete the bhyve / debian installation, but brought the security level back to 3, and after that kldstat shows vmm.ko. (The problem may not be due to kernel security, but probably related to network bridge, its name, and the tap0 creation.)

I don't understand the various errors, please help me fix these errors:

Thank you.
 
Bhyve is very picky about where it expects the EFI files to be in the virtual machine, if you don't want to define that file using the BIOS.

Debian installs them under /boot/efi/EFI/debian, but Bhyve wants them under /boot/efi/EFI/BOOT.

It also expects the main file to be named BOOTX64.EFI, not grubx64.efi as Debian calls it.

I mean it's explained in the instructions you were using, but the errors look exactly like either you didn't do it, or have some typo in it.
 
Thank you jmos I tried following the official guide, tried it again and again, everytime I faced some difficulty with some of the steps, as for instance now when I retried with recreated interfaces to follow the steps afresh:


# sysctl net.link.tap.up_on_open=1
net.link.tap.up_on_open: 1 -> 1
# ifconfig bridge0 create
ifconfig: interface bridge0 already exists
# ifconfig bridge0 addm re0 addm tap0
ifconfig: interface bridge0 does not exist # there is a conflict here, the previous command returned the message that bridge0 already exists, and this one says bridge0 does not exist.
# ifconfig bridge0 up
ifconfig: interface bridge0 does not exist

Like this, there were several issues, that I discussed in this freebsd forums page After all that, I tried this guide, which seemed to get my vm up and running, without network. Some of the errors that I described in this post are already fixed, but the problem with the network is persistent.
 
hardworkingnewbie said:
Bhyve is very picky about where it expects the EFI files to be in the virtual machine, if you don't want to define that file using the BIOS.

Debian installs them under /boot/efi/EFI/debian, but Bhyve wants them under /boot/efi/EFI/BOOT.

It also expects the main file to be named BOOTX64.EFI, not grubx64.efi as Debian calls it.

I mean it's explained in the instructions you were using, but the errors look exactly like either you didn't do it, or have some typo in it.
Click to expand...

Thank you hardworkingnewbie . This is very correct. I found a fix to the vm stuck on the shell prompt, created a file following a guide which instructed the vm the location of the .EFI file, and that fixed the boot problem. I still haven't resolved the network issue.

I have also installed ubuntu 18.04 and got past the UEFI problem, there is no network after reboot, both debian and ubuntu do not show the bridge interfaces. The bridge commands and configurations seem not to have taken effect.
 
The guide says:
Create a device.map that grub will use to map the virtual devices to the files on the host system:
(hd0) ./linux.img
(cd0) ./somelinux.iso
Click to expand...

How do I create a device map?

I carried out this step with the wrong syntax, so no device map was created, I continued with the unofficial guide, there is network in the ubuntu vm in bhyve environment. Everything works fine.

RESOLVED.
 
Last edited:
You must log in or register to reply here.
Back
Top