[Bhyve Virtualization] CentOS guest Display problem

I'm running FreeBSD 12.1-RELEASE-p1 and trying to host a CentOS VM (v8) with Bhyve.
When trying to do this with the MBR guide on the handbook bhyve page, I could not even get the VM working no matter what, I read that centos machines need UEFI booting with bhyve.
So I followed this guide UEFI bhyve to do so.
Now my problem is that I managed to get the VM shell when running it through a VNC. But my goal is to be able to access it using nmdm consoles(cf 21.7.7. Virtual Machine Consoles).
After installing the VM through VNC with the UEFI bhyve guide, I changed the booting code to this :
Code:
bhyve -c 2 -m 2G -w -H \
        -s 0,hostbridge \
        -s 4,ahci-hd,centos8.img \
        -s 5,virtio-net,tap0 \
        -s 31,lpc -l com1,/dev/nmdm0A \
        -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \
        vm1
Then I booted the vm and tried to connect to it from a terminal using this command cu -l /dev/nmdm0B but nothing is displayed.
Note : I managed to make this work(accessing with nmdm devices) for debian and ubuntu (while not using UEFI firmware pkg) but I can't seem to make it work for CentOS (using UEFI pkg because it didn't work otherwise).
 
I suspect you need to start the getty service on ttyS0 on the CentOS side; systemctl start getty@ttyS0. I suspect it's off by default and needs to be explicitly enabled.
 
You're right that does it thx!, is there anyway to enable this in the VM booting step with bhyve ? If I can avoid using UEFI-GOP (Graphical UEFI Framebuffer) and VNC altogether that would be great because I want to run a freebsd host machine without a windows display manager in the end to run this guest VM.
 
I have not tried Centos, yet. But, Debian and FreeBSD install fine, as per FreeBSD handbook - MBR or UEFI.

But now, I tried "vm-bhyve" to install OpenBSD-6.6, just for test.

I was using following instructions:

It worked OK too.

bhyve-obsd.png

Few things to note when using "vm-bhyve" utility after using basic "bhyve"

Firstly, you may need to comment out (as in my case) any references in /etc/rc.conf to a bridge and/or switch that were created for use with bhyve.
Code:
# cloned_interfaces="bridge0 tap0"
# ifconfig_bridge0="addm em0 addm tap0"

Follow vm-bhyve instructions on howto create switch, as in my case:
bhyve-ifconfig.png


Secondly, look inside the template configuration files (in .templates dir) to make sure that settings in those files match your guest's OS version and specs.
bhyve-fs.png
Thirdly, you can download your fav. guest OS disrto (iso or img) to any location and then copy or move it to the .iso or .img dirs of you vm for installation and set-up of the OS with vm-bhyve.

So, try vm-bhyve. It may work better for Centos.
 
If I can avoid using UEFI-GOP (Graphical UEFI Framebuffer) and VNC altogether that would be great because I want to run a freebsd host machine without a windows display manager in the end to run this guest VM.

Try the "*boot.iso" image from there. I think that's non-GUI installer.

ATM, I'm playing with plain "bhyve" and CentOS GUI installer as per:

"Support for the UEFI-GOP framebuffer may also be enabled with the -s 29,fbuf,tcp=0.0.0.0:5900 flags. The framebuffer resolution may be configured with w=800 and h=600, and bhyve can be instructed to wait for a VNC connection before booting the guest by adding wait. The framebuffer may be accessed from the host or over the network via the VNC protocol.
The resulting bhyve command would look like this:"

Code:
bhyve -AHP -s 0:0,hostbridge -s 31:0,lpc -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img -s 4:0,ahci-cd,./install.iso -c 4 -m 1024M -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd guest

Did you try this option with CentOS GUI installer?
 
Got to this point with Centos in bhyve. After installation and reboot, I'm stuck in grub because I cannot find the boot image on the virtual hd :(

bhyve-centos.png

These are my choices in grub, but I don't know where to go from here
Code:
grub> ls
(hd0) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0,msdos2) (host) (lvm/cl-root) (lvm/cl-home) (lvm/cl-swap)

grub> ls (hd0,gpt1) 
        Partition hd0,gpt1: Filesystem type fat, UUID BF26-D5E5 - Partition start at 2048 - Total size 1228800
sectors
 
Try the "*boot.iso" image from there. I think that's non-GUI installer.

ATM, I'm playing with plain "bhyve" and CentOS GUI installer as per:

"Support for the UEFI-GOP framebuffer may also be enabled with the -s 29,fbuf,tcp=0.0.0.0:5900 flags. The framebuffer resolution may be configured with w=800 and h=600, and bhyve can be instructed to wait for a VNC connection before booting the guest by adding wait. The framebuffer may be accessed from the host or over the network via the VNC protocol.
The resulting bhyve command would look like this:"

Code:
bhyve -AHP -s 0:0,hostbridge -s 31:0,lpc -s 2:0,virtio-net,tap1 -s 3:0,virtio-blk,./disk.img -s 4:0,ahci-cd,./install.iso -c 4 -m 1024M -s 29,fbuf,tcp=0.0.0.0:5900,w=800,h=600,wait -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd guest

Did you try this option with CentOS GUI installer?
I've already managed to run it with VNC but my end goal is to to be able to run the vm installation and booting on a machine without needing to install xorg+xfce or smthg else which I won't need. That is why I'm trying to find a way to do the whole installation in serial console so in a "text" mode.
 
Got to this point with Centos in bhyve. After installation and reboot, I'm stuck in grub because I cannot find the boot image on the virtual hd :(

View attachment 7251

These are my choices in grub, but I don't know where to go from here
Code:
grub> ls
(hd0) (hd0,gpt3) (hd0,gpt2) (hd0,gpt1) (cd0) (cd0,msdos2) (host) (lvm/cl-root) (lvm/cl-home) (lvm/cl-swap)

grub> ls (hd0,gpt1)
        Partition hd0,gpt1: Filesystem type fat, UUID BF26-D5E5 - Partition start at 2048 - Total size 1228800
sectors
If my memory serves me well I managed to boot from (hd0,gpt2) where the files vmlinuz-XXX and initramfs-XXX were available, just ls through the different partitions and u'll you'll find it I'm sure if ur your installation went on smoothly.
 
Got to this point with Centos in bhyve. After installation and reboot, I'm stuck in grub because I cannot find the boot image on the virtual hd :(

Update July 2020

I have been playing for a while with bhyve for centos7 and centos8 with and without RNC connection. Finally I could run the latest version of with CentOS-8.2.2004 using just the console (not uefi) :). The issue is how to select properly the grub partition for the linux kernel and root. Using vm-bhyve, this the template created for centos8:

./template/centos8.2.conf
Code:
loader="grub"
cpu=2
memory=2048M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"

# Kernel from ISO --> centos-8.2.2004
grub_install0="linux /isolinux/vmlinuz"
grub_install1="initrd /isolinux/initrd.img"

#Kernel from disk --> centos-8.2.2004
grub_run0="linux /vmlinuz-4.18.0-193.el8.x86_64 root=/dev/mapper/cl-root"
grub_run1="initrd /initramfs-4.18.0-193.el8.x86_64.img"
If you install different version of centos8.x, you have to match the version of the kernel with your grub file (always) !!
Code:
[root@centos8 ~]# cat /etc/redhat-release
CentOS Linux release 8.2.2004 (Core)
[root@centos8 ~]#
[root@centos8 ~]# neofetch
                 ..                    root@centos8
               .PLTJ.                  ------------
              <><><><>                 OS: CentOS Linux 8 (Core) x86_64
     KKSSV' 4KKK LJ KKKL.'VSSKK        Host: BHYVE 1.0
     KKV' 4KKKKK LJ KKKKAL 'VKK        Kernel: 4.18.0-193.el8.x86_64
     V' ' 'VKKKK LJ KKKKV' ' 'V        Uptime: 1 min
     .4MA.' 'VKK LJ KKV' '.4Mb.        Packages: 577 (rpm)
   . KKKKKA.' 'V LJ V' '.4KKKKK .      Shell: bash 4.4.19
.4D KKKKKKKA.'' LJ ''.4KKKKKKK FA.    Terminal: /dev/ttyS0
<QDD ++++++++++++  ++++++++++++ GFD>   CPU: Intel Pentium N3700 (2) @ 1.332GHz
'VD KKKKKKKK'.. LJ ..'KKKKKKKK FV     Memory: 135MiB / 1987MiB
   ' VKKKKK'. .4 LJ K. .'KKKKKV '
      'VK'. .4KK LJ KKA. .'KV'                                
     A. . .4KKKK LJ KKKKA. . .4                              
     KKA. 'KKKKK LJ KKKKK' .4KK
     KKSSA. VKKK LJ KKKV .4SSKK
              <><><><>
               'MKKM'
                 ''

[root@centos8 ~]#
 
For anyone else running into pain trying to get text-only CentOS/RHEL/RockyLinux 8 to work (as the GRUB-based install doesn't like the resulting XFS partitions):

Use UEFI and modify the installer boot line to use the com port.

Works great with vm-bhyve, the key steps (from the above link, here for convenience) during booting of the install CD:
  • Go to the line Install CentOS Linux 8
  • Press e to edit the command line
  • Modify the vmlinuz line to replace quiet by console=ttyS0,115200n8
  • Hit CTRL-X to boot.
This also gets away from having to deal with the GRUB boot loader which is a little less clean inside Bhyve than just letting UEFI take care of it. You also don't have to do a two-step of use an old installer to get up-and-running.
 
Back
Top