Solved Need help in creating a Rocky Linux Bhyve VM

byrnejb

byrnejb

FreeBSD-12.3
vm-bhyve-1.4.2
Rocky Linux 9.0

I am trying to install Rock Linux (RL) into a Bhyve VM. What I need help with is how do I determine what GRUB commands are required to get this to happen. Does someone have RL installed into a Bhyve VM and would care to share their configuration file settings; and, how these were determined?

I have looked at grub.cfg in the ISO image. From this I have set these values:

Code: 
loader="grub"
cpu=4
memory=6G
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
grub_install0="linux /images/pxeboot/vmlinuz inst.stage2=hd:LABEL=Rocky-9-0-x86_64-dvd quiet"
grub_install1="initrd /images/pxeboot/initrd.img"

running the vm and using vm console gets me to this:

Code: 
grub>
grub> ls
(hd0) (host)
grub> ls (hd0)/
Device hd0: No known filesystem detected - Total size 134217728 sectors
grub> ls (host)/
etc/ net/ bin/ usr/ boot/ lib/ COPYRIGHT rescue/ proc/ root/ home/ opt/ entropy
sys mnt/ zroot/ tmp/ libexec/ cogrub>
grub>
grub>

At this point I am stuck. What are the next grub commands to continue the installation? Evidently (host)/ refers to the host filesystem. I infer that (hd0) is the disk system for the vm but it is empty and has no partitions:
Code: 
grub> ls (hd0)/
error: unknown filesystem.

grub> ls (hd0,1)/
error: disk `hd0,1' not found.
 
Rocky Linux is basically a clone of RedHat. I would try to use the RHEL/CentOS 7 instructions.

github.com

Supported Guest Examples

Shell based, minimal dependency bhyve manager. Contribute to churchers/vm-bhyve development by creating an account on GitHub.
github.com github.com
 
I started from the CentOS.conf provided. Here is what I get from the example file for CentOS7 given at the reference above adjusted for the disk drive and network::

Code: 
guest="linux"
loader="grub"
cpu=4
memory=6G

network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"

grub_install0="linux /isolinux/vmlinuz"
grub_install1="initrd /isolinux/initrd.img"
grub_install2="boot"
grub_run_partition="msdos1"
grub_run_dir="/grub2"
[code]

When the vm is started the log file shows this:
[code]
Jul 29 11:02:23:  [bhyve options: -c 4 -m 6G -AHP -U b1da0526-0ea9-11ed-a5db-7085c2da8
84f -u]
Jul 29 11:02:23:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/zroot/v
m/vhp3k-01/disk0.img -s 5:0,virtio-net,tap0,mac=58:9c:fc:09:98:bc]
Jul 29 11:02:23:  [bhyve console: -l com1,/dev/nmdm-vhp3k-01.1A]
Jul 29 11:02:23: starting bhyve (run 1)
Jul 29 11:02:23: bhyve exited with status 3
Jul 29 11:02:23: destroying network device tap0
Jul 29 11:02:23: stopped
Jul 29 11:02:55: initialising
Jul 29 11:02:55:  [loader: grub]
Jul 29 11:02:55:  [cpu: 4]
Jul 29 11:02:55:  [memory: 6G]
Jul 29 11:02:55:  [hostbridge: standard]
Jul 29 11:02:55:  [com ports: com1]
Jul 29 11:02:55:  [uuid: 86a17176-0f4f-11ed-a5db-7085c2da884f]
Jul 29 11:02:55:  [utctime: yes]
Jul 29 11:02:55:  [debug mode: no]
Jul 29 11:02:55:  [primary disk: ]
Jul 29 11:02:55:  [primary disk dev: file]
Jul 29 11:02:55: booting
Jul 29 11:02:55: fatal; non-uefi loaders require a boot disk device
Jul 29 11:02:55: fatal; loader returned error 15
Jul 29 11:02:55: stopped
Jul 29 11:05:21: initialising
Jul 29 11:05:21:  [loader: grub]
Jul 29 11:05:21:  [cpu: 4]
Jul 29 11:05:21:  [memory: 6G]
Jul 29 11:05:21:  [hostbridge: standard]
Jul 29 11:05:21:  [com ports: com1]
Jul 29 11:05:21:  [uuid: 86a17176-0f4f-11ed-a5db-7085c2da884f]
Jul 29 11:05:21:  [utctime: yes]
Jul 29 11:05:21:  [debug mode: no]
Jul 29 11:05:21:  [primary disk: disk0.img]
Jul 29 11:05:21:  [primary disk dev: file]
Jul 29 11:05:21: generated static mac 58:9c:fc:0f:78:aa (based on 'vhp3k-01:0:16591071
21:0')
Jul 29 11:05:21: initialising network device tap0
Jul 29 11:05:21: adding tap0 -> vm-public (public addm)
Jul 29 11:05:21: bring up tap0 -> vm-public (public addm)
Jul 29 11:05:21: booting
Jul 29 11:05:21: create file /zroot/vm/vhp3k-01/device.map
Jul 29 11:05:21:  -> (hd0) /zroot/vm/vhp3k-01/disk0.img
Jul 29 11:05:21: /usr/local/sbin/grub-bhyve -c /dev/nmdm-vhp3k-01.1A -m /zroot/vm/vhp3
k-01/device.map -M 6G -r hd0,msdos1 -d /grub2 vhp3k-01

The vm starts and console show this:
Code: 
# vm console vhp3k-01
Connected
grub> ls
(host)

The is no hd0 to work with.
 
Try EUFI booting the VM. Some time ago I tested CentOS 8 Stream. That installed and worked without a hitch. But I honestly can't remember exactly how I configured it.
 
I extracted the iso file to disk and searched for grub.cfg. The only thing I found was this:
Code: 
find Rocky-9.0-x86_64-minimal -name grub.cfg 
Rocky-9.0-x86_64-minimal/EFI/BOOT/grub.cfg
Which contains this:
Code: 
cat Rocky-9.0-x86_64-minimal/EFI/BOOT/grub.cfg
set default="1"

function load_video {
  insmod efi_gop
  insmod efi_uga
  insmod video_bochs
  insmod video_cirrus
  insmod all_video
}

load_video
set gfxpayload=keep
insmod gzio
insmod part_gpt
insmod ext2

set timeout=60
### END /etc/grub.d/00_header ###

search --no-floppy --set=root -l 'Rocky-9-0-x86_64-dvd'

### BEGIN /etc/grub.d/10_linux ###
menuentry 'Install Rocky Linux 9.0' --class fedora --class gnu-linux --class gnu --class os {
    linuxefi /images/pxeboot/vmlinuz inst.stage2=hd:LABEL=Rocky-9-0-x86_64-dvd quiet
    initrdefi /images/pxeboot/initrd.img
}
menuentry 'Test this media & install Rocky Linux 9.0' --class fedora --class gnu-linux --class gnu --class os {
    linuxefi /images/pxeboot/vmlinuz inst.stage2=hd:LABEL=Rocky-9-0-x86_64-dvd rd.live.check quiet
    initrdefi /images/pxeboot/initrd.img
}
submenu 'Troubleshooting -->' {
    menuentry 'Install Rocky Linux 9.0 in text mode' --class fedora --class gnu-linux --class gnu --class os {
        linuxefi /images/pxeboot/vmlinuz inst.stage2=hd:LABEL=Rocky-9-0-x86_64-dvd inst.text quiet
        initrdefi /images/pxeboot/initrd.img
    }
    menuentry 'Rescue a Rocky Linux system' --class fedora --class gnu-linux --class gnu --class os {
        linuxefi /images/pxeboot/vmlinuz inst.stage2=hd:LABEL=Rocky-9-0-x86_64-dvd inst.rescue quiet
        initrdefi /images/pxeboot/initrd.img
    }
}

Searching for vmlinuz gives this:
Code: 
find Rocky-9.0-x86_64-minimal -name vmlinuz
Rocky-9.0-x86_64-minimal/isolinux/vmlinuz
Rocky-9.0-x86_64-minimal/images/pxeboot/vmlinuz
That matches the default configuration setting.

Looking for initrd.img gives this:
Code: 
find Rocky-9.0-x86_64-minimal -name initrd.img
Rocky-9.0-x86_64-minimal/isolinux/initrd.img
Rocky-9.0-x86_64-minimal/images/pxeboot/initrd.img
This also matches the default configuration.
 
I am making progress but still no joy:

Code: 
# Rocky Linux 8.6  2022-08-03
guest="linux"
loader="grub"
cpu=4
memory=6G

network0_type="virtio-net"
network0_switch="public"

disk0_type="virtio-blk"
disk0_name="disk0.img"

grub_install0="linux /isolinux/vmlinuz LANG=en_US.UTF-8 KEYTABLE=us console=ttyS0"
grub_install1="initrd /isolinux/initrd.img"
grub_install2="boot"

grub_run0="linux /vmlinuz-4.18.0-372.9.1.el8.x86_64 nomodeset console=ttyS0"
grub_run1="initrd /initramfs-4.18.0-372.9.1.el8.x86_64.img"
grub_run2="boot"

This gets me to here:
Code: 
vm console vhp3k-01

error: disk `hd0,1' not found.
error: you need to load the kernel first.

Followed by:
Code: 
                             GNU GRUB  version 2.00

 +--------------------------------------------------------------------------+
 |vhp3k-01 (bhyve run)                                                      |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 +--------------------------------------------------------------------------+

      Use the ^ and v keys to select which entry is highlighted.
      Press enter to boot the selected OS, `e' to edit the commands
      before booting or `c' for a command-line.

Entering e gets me to here:
Code: 
                             GNU GRUB  version 2.00

 +--------------------------------------------------------------------------+
 | setparams 'vhp3k-01 (bhyve run)'                                         |
 |                                                                          |
 |     root=hd0,1                                                           |
 |     linux /vmlinuz-4.18.0-372.9.1.el8.x86_64 nomodeset console=ttyS0     |
 |     initrd /initramfs-4.18.0-372.9.1.el8.x86_64.img                      |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 |                                                                          |
 +--------------------------------------------------------------------------+

      Minimum Emacs-like screen editing is supported. TAB lists
      completions. Press Ctrl-x or F10 to boot, Ctrl-c or F2 for
      a command-line or ESC to discard edits and return to the GRUB menu.
But going into the command line with F2 I get this:
Code: 
grub> ls
(hd0) (host)
grub> ls (hd0)/
error: unknown filesystem.

Also, when I am in edit mode I cannot move the cursor with the arrow keys. Nor can I use the EMACs C-n command. I am immediately returned to the bhyve run grup screen.
 
As an alternative I created a bhyve vm using loader="uefi" and graphis="yes". I used vm list to find the port and used vncviewer to connect:
Code: 
vm list
NAME      DATASTORE  LOADER  CPU  MEMORY  VNC           AUTOSTART  STATE
test      default    uefi    1    512M    -             No         Stopped
vhp3k-01  default    grub    4    6G      -             No         Stopped
vhp3k-02  default    grub    4    6G      -             No         Stopped
vhp3k-03  default    uefi    4    6G      0.0.0.0:5900  No         Running (23039)

vncviewer ::5900

This gets me into a graphical session. But I end up in a shell SHELL>.

The mapping table is:
Code: 
  BLK0: Alias(s)
             PciRoot(0x0)/Pci(0x3,0x0) /Sata (0x0,oxFFFF,0x0)
  BLK1: Alias(s):
             PciRoot (0x0) /Pci (0x4,0x0)

I have no idea where to go from here.
 
get rid of grub and install it with uefi. I just verified it and it works, my config:

Code: 
loader="uefi"
cpu=2
memory=4096M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
uuid="c30f0d70-bb5f-40fc-bee9-a6eaa2276452"
network0_mac="58:9c:fc:07:e1:a2"
graphics="yes"
graphics_listen="10.0.1.239"
graphics_port="5922"
graphics_vga=io
 
You must log in or register to reply here.
Back
Top