How-to: Install and run BlissOS (Android-x86) with bhyve

[Maxnix]

Hello folks!
In this tutorial I'll show you how to install and run BlissOS (an Android-x86 based OS) with bhyve.

Table of contents

• Reasoning
• Requirements
• Software used
• Installation
• Post-install steps
  
  • Setup a new installation
  • Migrate a VirtualBox VM
• Conclusion

PART 1

• Reasoning

Even if Android (in its x86-64 port) is supported by bhyve and it's simple to boot a live Android-x86 OS with bhyve, there are some details worthy of attention when installing it. That's why I'm writing this how-to, since I didn't find a unique source to address them entirely.​

• Requirements

First of all, your PC should support bhyve: check the wiki page for more informations.​

Then you need to download the iso of the version you want to install. From BlissOS site you can choose which one download. I'll use Bliss OS 15.9.x (x86_64-v2) with GApps in this tutorial because it's the newest that is not in beta stage.​

• Software used

I used sysutils/vm-bhyve as my VM manager, and net/tigervnc-viewer as VNC client.​

• Preliminary steps

If this is the first time you use vm-bhyve, there are some preliminary steps to enable and configure it. Let's start by enabling it (and bhyve support too) in /etc/rc.conf:​

vm_enable="YES"
vm_dir="<path to the directory/ZVOL which will host your VMs>"
vmm_load="YES"

This will set things up for vm-bhyve: needed directories will be created and required kernel modules will be loaded.​

# vm init

Now it's time to configure a virtual switch that will let the associated VMs to connect to the internet. And here we have two choices:​

1. Bridge
2. NAT

By default, bridging is used, BUT it won't work if your computer use a wireless interface. In that case NAT is required.​

​

We'll start by showing the bridge-based setup:​

# vm create switch public
# vm switch add public <your interface name>

Here we created a new virtual switch named public, and assigned an interface to it. You can create as much virtual switch you want, and name them as you please. It's just up to how you want things to be organized.​

​

Now let's see how to configure NAT: here things are more complex, since to use NAT we have to enable the host to act as a gateway, and a firewall will be responsible for the NAT itself. The wiki have instructions for both static IP configuration and for DHCP; I'll report here the steps described to set things up with a static IP address and using PF as firewall, since is what I use. To configure DHCP instead, you can check the wiki page. If you prefer another firewall and/or program for DHCP you will have to adapt the wiki instructions.​

​

Now, let's enable the host to act as a gateway and PF in /etc/rc.conf​

gateway_enable="YES"
pf_enable="YES"

​

We create a rule for NAT in /etc/pf.conf and one to permit traffic from the VMs to the internet; I choose the described 192.168.8.x network for my VM:​

nat on <interface name> from {192.168.8.0/24} to any -> (<interface name>)
pass in quick from {192.168.8.0/24} to any

Now, if you don't want to reboot, enable IP forwarding and PF this way:​

# sysctl net.inet.ip.forwarding=1
# service pf start

Note: net.inet.ip.forwarding will be enabled automatically on the next boot because we enabled the gateway functionalities; there is no need to put it in /etc/sysctl.conf​

​

Very well, now that the host is ready for NAT, let's setup vm-bhyve:​

# vm switch create -a 192.168.8.1/24 public     <-- if creating a new switch
# vm switch address public 192.168.8.1/24       <-- if you already have a switch you want to use
# vm switch list
NAME    TYPE      IDENT  ADDRESS         PRIVATE  MTU  VLAN  PORTS
public  standard  -      192.168.8.1/24  no       -    -     -

Done! We can now start the installation process!​

• Installation

First of all, let's create a new VM. I didn't use a specific template, just the default one. We'll modify it immediately afterwards.​

# vm create BlissOS
# vm configure BlissOS

And this is how my BlissOS.conf looks like:​

loader="uefi"
cpu=2
memory=4096M
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0.img"
uuid="<Here will be the unique id of your VM>"
network0_mac="<Here will be the MAC address of the VM network interface>"
graphics="yes"
graphics_port="5900"
graphics_res="<Put your VM resolution here (don't chose a resolution smaller than 640x480)>"
graphics_wait="yes"
xhci_mouse="yes"
bhyve_options="-s 10,hda,play=/dev/dsp,rec=/dev/dsp"

Now that the VM is ready installation can begin: let's boot this thing!​

# vm install BlissOS <path to the iso image file>
$ vncviewer 0:5900

Here the bootloader welcome us with the various boot options. What interest us is the 5th entry: "Installation":

However, we can't boot that entry directly: we must edit it.

We must add two options to the setparams line: nomodeset and VIRT_WIFI=1

This options will be automatically added to the GRUB entries when the OS will be installed; we won't need to add them manually.
This is how the boot process looks like:

It'll take some time detecting the hardware, then the installer will start:

 

[Maxnix]

PART 2
Here we are informed that the system used (the VM) uses UEFI, and that we need to select an EFI partition. However the drive is still empty:

Now let's create the partitioning scheme and the EFI and system partitions. Choose cgdisk for the task.

Just ignore the warning, since we know that the virtual disk is still empty:

We'll use GPT as partitioning scheme:

Now let's add the EFI partition first: choose [New] and accept the default alignement. As for the size, I choose 100M, then ef00 as partition type. If you want you can add a name to the partition. I named it EFI.

 

[Maxnix]

PART 3
It's time to create the system partition. Just add another partition, and accept all the defaults, this way all the remaining space will be used to create a partition for the system itself:

Commit the changes writing the partition table to the disk, the installer then will restart itself.
Now, select vda1 as ESP:

The installer will ask to format the ESP partition, but we choose to not re-format it; we'll be asked to format it later nonetheless.

Next step is to choose and format the system partition:

 

[Maxnix]

PART 4

The installer now will ask if we want to prepare for OTA update. Since this will double the system size (and being a VM probably won't work) I choose No.

Now it's really time to format the ESP partition:

Ok, time to put everything on the (virtual) disk:

Finally done! Congratulations, BlissOS has been installed!

However we have not finished yet: from the TigerVNC menu (F8) choose to send a Ctrl+Alt+Del sequence to the VM. This way the VM will restart and the VNC connection will be lost. But we can't just start the VM now, because it won't boot as things are now, so instead poweroff it:

# vm poweroff BlissOS

Since we have graphics_wait="YES" in our VM config file, the machine will be locked and not effectively powered off until we make a new VNC connection (that will inevitabily last less than a second ).

$ vncviewer 0:5900
$

• Post install

Why the machine will not boot if we start it? Because there is a "problem" with the EFI loader that bhyve need to boot the OS: some systems (and BlissOS makes no exception) install the EFI loader in a non standard location (always in the ESP partition). So we need to copy that binary in the correct place and rename it accordingly.
First we create a mountpoint for the ESP partition, then attach the virtual disk with mdconfig(8), mount the partition and copy the file in the path that bhyve expect. Finally, unmount the partition and destroy the /dev/md99 device created to attach the virtual disk.

# mkdir /tmp/ESP
# mdconfig -f /path/to/your/virtual/disk/disk0.img -u99
# mount -t msdosfs /dev/md0p1 /tmp/ESP
# cd /tmp/ESP
# ls
EFI
# cd EFI
# ls
BlissOS
# mkdir BOOT
# cp BlissOS/grubx64.efi BOOT/bootx64.efi
# cd /tmp
# umount /tmp/ESP
# rmdir /tmp/ESP
# mdconfig -d -u99

Everything done! Now our VM is really ready to boot!

The next step now depends on what you need/want: if you want to setup your VM from zero, we'll proceed to the configuration of BlissOS (just as you would do with every new android device you get), otherwise, if you already have a VirtualBox VM with Android you can move your system to the new bhyve VM.

• Setup a new installation

Setting up a new installation, as said, is pretty straightforward: just proceed as you would do with a smartphone/tablet. Just remember to choose to set up it offline; you'll be able to set up the virtual Wi-Fi interface later.

• Migrate a VirtualBox VM

Migrating an existing installation from VirtualBox is pretty easy. Fi#rst of all we must convert the disk to a raw image, in order to be able to attach it with mdconfig(8), then we attach the bhyve VM disk, check the ext4 filesystems with e2fsck from sysutils/e2fsprogs, mount them and finally transfer everything that matters from the VirtualBox disk to the bhyve one:

# cd /path/of/the/vm-bhyve/vm
# VBoxManage clonehd /path/to/your/disk rawimage.img --format raw
# mdconfig -f rawimage.img -u98
# mdconfig -f disk0.img -u99
# e2fsck /dev/md98s1
# e2fsck /dev/md99p2
# mkdir /tmp/old_sys /tmp/new_sys
# mount -t ext2fs /dev/md98 /tmp/old_sys
# mount -t ext2fs /dev/md99 /tmp/new_sys
# rm -r /tmp/new_sys/android-<YYYY-MM-DD>/*
# cd /tmp/new_sys/android-<YYYY-MM-DD>/
# tar cfC - /tmp/old_sys/android-<YYYY-MM-DD> . | tar xpvfC - /tmp/new_sys/android-<YYYY-MM-DD>
# cd -
# umount /tmp/old_sys
# umount /tmp/new_sys
# mdconfig -d -u98
# mdconfig -d -u99

All done! Your VM is now ready to be used!

• Static IP configuration

The last step is to configure the virtual Wi-Fi interface to use a static IP. An example is shown below, if you choose another IP range adjust accordingly.

• Conclusion

That's all! I hope I've been helpful and that you can enjoy your bhyve based Android VM!

 

[Emrion]

Note that there is no problem if the efi loader is in a non standard place. Normally, the installer writes the appropriate entry (including the loader path) in the efi vars.

I think it doesn't work here because you did not provide a file for storing the efi vars. Seems you need to set uefi_vars at true with vm-bhyve. See vm(8), paragraph GUEST CONFIGURATION FORMAT.

 

[Maxnix]

Note that there is no problem if the efi loader is in a non standard place. Normally, the installer writes the appropriate entry (including the loader path) in the efi vars.

I think it doesn't work here because you did not provide a file for storing the efi vars. Seems you need to set uefi_vars at true with vm-bhyve. See vm(8), paragraph GUEST CONFIGURATION FORMAT.

Thank you for pointing that out. As far as I knew, bhyve was unable to store UEFI variables set during the installation of a guest, so I didn't study in deep. I'll give a better look.

 

[ziomario]

Does Bliss supports Google Play ?

 

[Maxnix]

Does Bliss supports Google Play ?

Yes, of course. Just be carefull when you download your ISO: there are ISOs without Google apps (they have FOSS in their name) and the ones that have them installed instead (they have GApps in their name). I used the latter here.