bhyve bhyve and Debian

ikevin8me

Active Member

Reaction score: 15
Messages: 198

I'm trying to install Debian on bhyve. I got stuck with "No common CD-ROM drive was detected" error. Obviously, the installation CD is provided to bhyve via "-s 3,ahci-hd,<debian.iso>" and it managed to boot up.

I cannot proceed with the installation as it says "The failing step is: Detect and mount CD-ROM"

Note: I already have FreeBSD and OpenBSD guests working perfectly.

What is the solution?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,118
Messages: 39,712

Use UEFI boot to boot the Debian VM.

Code:
Jul 29 18:14:29:  [bhyve options: -c 2 -m 4096M -Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -U 137ea5a5-58d0-11eb-bcdc-002590f15838 -u]
Jul 29 18:14:29:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/dev/zvol/stor10k/DATA/vm/debian/disk0 -s 5:0,virtio-net,tap14,mac=58:9c:fc:05:c8:d6 -s 6:0,fbuf,tcp=0.0.0.0:5900,wait]
Jul 29 18:14:29:  [bhyve console: -l com1,/dev/nmdm-debian.1A]
Jul 29 18:14:29:  [bhyve iso device: -s 3:0,ahci-cd,/vm/.iso/debian-10.7.0-amd64-netinst.iso,ro]
Jul 29 18:14:29: starting bhyve (run 1)
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

I need more clarifications. Here's my command:

bhyve -c 1 -m 2G -w -u -H \
-s 0,hostbridge \
-s 3,ahci-hd,/guests/install/os/debian-10.10.0-amd64-xfce-CD-1.iso \
-s 4,virtio-blk,/guests/vms/debian.img \
-s 11,virtio-net,tap11 \
-s 29,fbuf,tcp=127.0.0.1:10011,w=1200,h=1000 \
-s 30,xhci,tablet \
-s 31,lpc \
-l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
debian.img

I'm already using "BHYVE_UEFI".
I can't really tell what's crucial in your command which is needed to make it work. Could you please point it out?
 

jmos

Well-Known Member

Reaction score: 224
Messages: 290

ahci-hd -> ahci-cd

Actually I have five bhyve Debian VMs, and all of them worked with the explanation of the handbook of how to create a Linux guest (grub-bhyve & device.map).
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,118
Messages: 39,712

I would suggest using sysutils/vm-bhyve to create and manage your bhyve(8) VMs. That's will make using bhyve(8) much easier.

Code:
# vm configure debian
loader="uefi"
graphics="YES"
cpu="2"
memory="4096M"
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0"
disk0_dev="sparse-zvol"
grub_run_dir="/grub"
Code:
root@hosaka:~ # vm start debian
Starting debian
  * found guest in /storage/vm/debian
  * booting...
root@hosaka:~ # vm list | grep debian
debian          stor10k    uefi       2    4096M   0.0.0.0:5900  No         Running (33724)
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

After trying much...
changing to "cd" and adding "ro" make it work:
-s 3,ahci-hd,/guests/install/os/debian-10.10.0-amd64-xfce-CD-1.iso \
to
-s 3,ahci-cd,/guests/install/os/debian-10.10.0-amd64-xfce-CD-1.iso,ro \
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

Well, the installation problem has been resolved and I can boot Debian into a text-based environment.

However, after re-installing again with Xfce, I can't boot Debian into the graphical environment. It simply shows a blank screen with a blinking cursor at the top-left.

I've tried both (on separate occasions):
-l com1,stdio
-l com1,/dev/nmdm-debian.1A
... and it does not work.

I'm not sure whether the bootup process stalled, or it had switched into a different console/display.

Would appreciate some help! ...
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,118
Messages: 39,712

I've tried both (on separate occasions):
Those are serial consoles, you will never get any graphics output from those. Use a VNC client to connect to the VNC port of the VM.
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

That's correct. I'm already using VNC to connect to it. It switched to a blank screen.

I just tried a manual install by "apt install xfce ..." and it is still the same problem.
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

I already did that. It is both a bhyve and Debian question.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,118
Messages: 39,712

No, it's a Debian configuration issue. Bhyve does what it needs to do, open a graphical console through VNC. Now it's a matter of configuring the Debian VM to use the framebuffer.
 

jmos

Well-Known Member

Reaction score: 224
Messages: 290

I simply used the handbook to set up my Debian VMs; So I don't use bhyves builtin VNC server, instead I've installed and configured them on the Debian VM (but: I also don't need the whole X environment every time). Maybe this howto helps you.
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

I'm now trying NOT to use bhyve's built-in VNC. I tried to install it using "bhyve" command by removing:
-s 29,fbuf,tcp=127.0.0.1:10011,w=1200,h=1000 \
-s 30,xhci,tablet \
so the command becomes:
bhyve -c 1 -m 2G -w -u -H \
-s 0,hostbridge \
-s 3,ahci-hd,/guests/install/os/debian-10.10.0-amd64-xfce-CD-1.iso \
-s 4,virtio-blk,/guests/vms/debian.img \
-s 11,virtio-net,tap11 \
-s 31,lpc \
-l com1,stdio \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
debian.img
and the problem becomes Debian will switch to a different console.

When I do the same thing with OpenBSD, I have to type "set tty com0" to it uses the same console.
So, Question #1: How can do the same thing with Debian - to use the same console with the comnand "bhyve"?



I simply used the handbook to set up my Debian VMs; So I don't use bhyves builtin VNC server, instead I've installed and configured them on the Debian VM (but: I also don't need the whole X environment every time). Maybe this howto helps you.

Then, I tried follow jmos's advice in the howto and executed:
grub-bhyve -m device.map -r cd0 -M 1024M linuxguest
then:
which in my case showed only:
Note: those "(hd0) (cd0) (cd0,msdos1)" in the howto did not appear.

So, Questions #2: for the command "grub-bhyve", how do I point to the "guests/install/os/debian-10.10.0-amd64-xfce-CD-1.iso" and "debian.img" (created by truncate -s 16G debian.img)? Or, rather, how to I make those "(hd0) (cd0) (cd0,msdos1)" appear as described in the howto? There is no "man entry" for "grub-bhyve" and I can't find a man-page for "grub-bhyve" on the web, so if there is one, please point me to it. (this one does not seem be descriptive enough)
 

jmos

Well-Known Member

Reaction score: 224
Messages: 290

Using the debian-10.10.0-amd64-xfce-CD-1.iso image won't force you to type in anything in Grub; Instead you will see the usual menu, in which you're choosing the text install - then it exits. But: The VM is still running. Just execute the normal bhyve command afterwards. That's how it should look like in your directory:
user@machine ~/yourdir> ls
debian-10.10.0-amd64-xfce-CD-1.iso
device.map
linux.img
user@machine ~/yourdir> cat device.map
(hd0) ./linux.img
(cd0) ./debian-10.10.0-amd64-xfce-CD-1.iso
user@machine ~/yourdir> doas grub-bhyve -m device.map -r cd0 -M 1024M linuxguest

[… here we will automatic return to the shell, but the VM is still running …]

user@machine ~/yourdir> doas bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img -s 4:0,ahci-cd,./debian-10.10.0-amd64-xfce-CD-1.iso -l com1,stdio -c 4 -m 1024M linuxguest

Checked, works.
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

Using the debian-10.10.0-amd64-xfce-CD-1.iso image won't force you to type in anything in Grub; Instead you will see the usual menu, in which you're choosing the text install - then it exits. But: The VM is still running. Just execute the normal bhyve command afterwards. That's how it should look like in your directory:
user@machine ~/yourdir> ls
debian-10.10.0-amd64-xfce-CD-1.iso
device.map
linux.img
user@machine ~/yourdir> cat device.map
(hd0) ./linux.img
(cd0) ./debian-10.10.0-amd64-xfce-CD-1.iso
user@machine ~/yourdir> doas grub-bhyve -m device.map -r cd0 -M 1024M linuxguest

[… here we will automatic return to the shell, but the VM is still running …]

user@machine ~/yourdir> doas bhyve -A -H -P -s 0:0,hostbridge -s 1:0,lpc -s 2:0,virtio-net,tap0 -s 3:0,virtio-blk,./linux.img -s 4:0,ahci-cd,./debian-10.10.0-amd64-xfce-CD-1.iso -l com1,stdio -c 4 -m 1024M linuxguest

Checked, works.
OK. The above works! I managed to install Debian and boot it up. It is all text-based, as expected.

I actually need to run some graphical apps so I need to "see it" with a VNC. I'm thinking of 2 solutions:

Solution #1. type in something Grub to set the console, just like "set tty com0" as with OpenBSD.

jmos, you said:
Using the debian-10.10.0-amd64-xfce-CD-1.iso image won't force you to type in anything in Grub
Are you hinting me that if I download a different image, I can type something ito Grub? If so, which one should I download?

Being able to VNC into bhyve seems to be a more natural approach.

Solution #2. set up vncserver inside Debian OS.

If there is no Solution #1, then I'm thinking of setting up tightvnc inside Debian OS and access it via ssh. This approach should work but doesn't seem to be most elegant.

What is you advice?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,118
Messages: 39,712

Solution #3, use SSH with X-forwarding to start the application on the Debian VM and have the GUI appear on your FreeBSD desktop.
 

jmos

Well-Known Member

Reaction score: 224
Messages: 290

Are you hinting me that if I download a different image, I can type something ito Grub? If so, which one should I download?
"If the installation CD contains a grub.cfg, a menu will be displayed. If not, the vmlinuz and initrd files must be located and loaded manually"
That's from the handbook. Also my usage example is from the handbook ;)
Solution #2. set up vncserver inside Debian OS.
[…]
What is you advice?
As in post #13 said I've set up a VNC server on my Debian VMs (a howto link is also in post #13). Just starting X applications remote (like SirDice mentioned) may also be a solution (unkown by many). But note that there are applications that won't be usable via SSH, also you might get keyboard problems via VNC (and some desktops won't work via such a VNC server). Using bhyves builtin VNC also has its benefits. There is no "best solution", it depends on what works for you and your requirements. I'm using all three options (builtin VNC, installed VNC, SSH with X11 (as well es plain text SSH)) across my different VMs (there are some more than the five Debians).
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

I have noticed a strange problem. By using the installation method in the howto above (loading by grub-bhyve...) the Linux installation becomes different. First, instead of the network interface being "enp0s11", it has become "enp0s2". And, instead of the settings in /etc/network/interfaces, they can no longer be found (ie. the interfaces file does not contain the specific IP address) even though "# ip address" shows the setting is effective.
 
OP
I

ikevin8me

Active Member

Reaction score: 15
Messages: 198

-U 137ea5a5-58d0-11eb-bcdc-002590f15838
SirDice, I notice that you put the UUID in the bhyve parameters. I supposed this is generated from "uuidgen" and I'm wondering why do we need that. Does it make any different to the FreeBSD host, or within the guest OS? Do it improve security in any way?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 13,118
Messages: 39,712

I didn't put it there, it's generated when you create a VM from a template. I'm honestly not sure why it's there.

Code:
root@hosaka:~ # vm create -d stor10k -t debian -c 2 -m 4096M debian2
root@hosaka:~ # cat /storage/vm/debian2/debian2.conf
loader="grub"
cpu="2"
memory="4096M"
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0"
disk0_dev="sparse-zvol"
grub_run_dir="/grub"
uuid="cbafd042-f9ad-11eb-aa5e-002590f15838"
network0_mac="58:9c:fc:02:03:b9"
Code:
root@hosaka:~ # cat /vm/.templates/debian.conf
loader="grub"
cpu=1
memory=512M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0"
disk0_dev="sparse-zvol"
grub_run_dir="/grub"
 
Top