Bhyve with Windows 10

rdunkle

rdunkle

Recent changes to Bhyve have allowed for a VNC console to complete
guest installs. This example will install Windows 10 as a guest.

For this example FreeBSD 12-CURRENT was installed:
FreeBSD 12.0-CURRENT #0 r302526

Edit: /boot/loader.conf
Add the following lines:
Code: 
vmm_load="YES"
nmdm_load="YES"
if_bridge_load="YES"
if_tap_load="YES"
Edit: /etc/rc.conf
Add the following lines
**Note -- change em0 to whatever your real network interface is
Code: 
cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="addm em0 addm tap0"

For simplicity -- this example -- all files are copied to a single directory
The MS Windows 10 install iso: win10.iso

The latest virtio iso file:
fetch https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.118-2/virtio-win-0.1.118.iso

Then install the latest Bhyve UEFI:
You can read more here:
https://wiki.freebsd.org/bhyve/UEFI

The basic steps:
Create a temporary directory to store the branch.
Change into that directory.
Execute these commands:
Code: 
# svnlite co http://svn.freebsd.org/base/projects/bhyve_graphics
# cd bhyve_graphics
# make BHYVE_SYSDIR=/usr/src/ -m /usr/src/share/mk
# cd /usr/ports/sysutils/uefi-edk2-bhyve
# make install

Copy the new uefi file into our Bhye directory with the rest of our files:
cp /usr/local/share/uefi-firmware/BHYVE_UEFI.fd the_directory_of_our Bhyve files

Create the empty Bhyve Win10 image:
truncate -s 70GB win_10.img

This will be the first boot to start the install:
bhyve -c 2 -s 0,hostbridge -s 3,ahci-cd,./win10.iso -s 4,ahci-hd,./win_10.img -s 10,virtio-net,tap0 -s 11,fbuf,tcp=0.0.0.0:5900,w=1600,h=900,wait -s 20,xhci,tablet -s 31,lpc -l bootrom,./BHYVE_UEFI_20160526.fd -m 2G -H -w win10

The install waits for the vnc connection.
Connect with vnc client (I used tsclient) to the Windows install console:
ip_address_of_bhyve_host:5900

Press Enter (on the vnc screen) to get the install started.
Let the installer copy all the files and do the first reboot

After the reboot. Bhyve exits normally.
Now create an empty install DVD. We no longer need the install iso, but
Bhyve tries to boot from a DVD.
touch null_dvd.iso

Boot Win10 a second time and let it continue the installation.
Notice how we use the null_dvd.iso.

Again, connect to the Windows console using VNC

bhyve -c 2 -s 0,hostbridge -s 3,ahci-cd,./null_dvd.iso -s 4,ahci-hd,./win_10.img -s 10,virtio-net,tap0 -s 11,fbuf,tcp=0.0.0.0:5900,w=1600,h=900,wait -s 20,xhci,tablet -s 31,lpc -l bootrom,./BHYVE_UEFI_20160526.fd -m 2G -H -w win10

The install will continue and then reboot again.

The third boot of Windows use the same command line to continue the installation:
bhyve -c 2 -s 0,hostbridge -s 3,ahci-cd,./null_dvd.iso -s 4,ahci-hd,./win_10.img -s 10,virtio-net,tap0 -s 11,fbuf,tcp=0.0.0.0:5900,w=1600,h=900,wait -s 20,xhci,tablet -s 31,lpc -l bootrom,./BHYVE_UEFI_20160526.fd -m 2G -H -w win10

After you get the Windows desktop there will be no network.
Need to install the virtio network driver.
Shutdown Windows 10.

Boot this time with the virtio net drivers from the CD:
bhyve -c 2 -s 0,hostbridge -s 3,ahci-cd,./virtio-win-0.1.118.iso -s 4,ahci-hd,./win_10.img -s 10,virtio-net,tap0 -s 11,fbuf,tcp=0.0.0.0:5900,w=1600,h=900,wait -s 20,xhci,tablet -s 31,lpc -l bootrom,./BHYVE_UEFI_20160526.fd -m 2G -H -w win10

Login to Windows.
Look for the CD drive. Open this folder to install the network driver:
NetKVM\w10\amd64
Right-click "netkvm" and select install
Networking should start to install and configure.
That should be all needed for Windows guest install.

You can configure RDP or use VNC to connect.
Use this command to start the Windows guest:
bhyve -c 2 -s 0,hostbridge -s 3,ahci-cd,./virtio-win-0.1.118.iso -s 4,ahci-hd,./win_10.img -s 10,virtio-net,tap0 -s 11,fbuf,tcp=0.0.0.0:5900,w=1600,h=900,wait -s 20,xhci,tablet -s 31,lpc -l bootrom,./BHYVE_UEFI_20160526.fd -m 2G -H -w win10
 
If you're running 12-CURRENT, you don't need to build the bhyve_graphics project branch - it's already been merged into head. In fact it's also in the 11 beta so both of these can run UEFI with a VNC console out of the box. There was a push to get this code into head before the 11-stable fork.

The uefi port is also a bit of a pain to build as it depends on gcc48. I'd recommend using the package from the latest pkg branch if you can, or downloading it from @grehan's FreeBSD page.

It's also made a lot easier using one of the available bhyve tools such as (sorry, shameless plug) sysutils/vm-bhyve, or sysutils/iohyve. Michael Dexter's vmrc has also been heavily updated in the last few days although I haven't had a chance to see what that's like yet.
 
Oh wait, one thing I'm not sure about is which AMD processors have unrestricted guest support which would be needed for UEFI (or whether it's even available/needed on AMD)...
 
I have no idea only i tried with FreeBSD 10 (on FreeNAS) and bhyve was not supporting UEFI. I only can find that my processor should be working but all information that i found is about FreeBSD 11 so maybe i have to just try it out.... and install FreeBSD 11. My AMD processor is the latest generation, so it should be working... I only can find that it must support POPCNT and NRIPS and it has both features...
 
dmesg.boot:
Code: 
CPU: AMD A8-7600 Radeon R7, 10 Compute Cores 4C+6G (3094.43-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x630f01 Family=0x15 Model=0x30 Stepping=1
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36
,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x3e98320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,POPCNT,AESNI,XSAVE,OSX
SAVE,AVX,F16C>
AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
AMD Features2=0xfebbfff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,XOP,SKINIT
,WDT,LWP,FMA4,TCE,NodeId,TBM,Topology,PCXC,PNXC,<b25>,DBE,PTSC>
Structured Extended Features=0x9<FSGSBASE,BMI1>
XSAVE Features=0x1<XSAVEOPT>
SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=65536
TSC: P-state invariant, performance statistics
real memory = 34846277632 (33232 MB)
avail memory = 33202843648 (31664 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <ALASKA A M I>
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
cpu0 (BSP): APIC ID: 16
cpu1 (AP): APIC ID: 17
cpu2 (AP): APIC ID: 18
cpu3 (AP): APIC ID: 19

so it should be working
 
>which AMD processors have unrestricted guest support

It's available on all AMD CPUs that bhyve supports - on AMD, it was never a separate feature.
 
Ah, good to know that all the AMD processors support it. I got my hands on a Xeon E5504 machine the other day thinking it would make a good test system and it doesn't have UG :(
 
help needed! new to FreeBSD coding... i'm running FreeBSD 12 and want to install Windows in bhyve (just to test and make clear it is working on my AMD).
First thing... fetch the virtio iso does not work??

Code: 
fetch https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.126-2/virtio-win-0.1.126.iso

Certificate verification failed for /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA

34374374744:error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed:/usr/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1264:

fetch: https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.126-2/virtio-win-0.1.126.iso: Authentication error


Next i have the windows iso on a usb flash drive how to get it into FreeBSD?
mount /dev/[usb drive] ?
 
Hi,

Just tried this and everything seems to work besides networking. Installation pass like a charm, however, during first install I have had to wait for timeout and to get an console from EFI, than I was able to chose installation media.

I've used pure FreeBSD11 + p5, I've setuped tap1 -> bridge0 <- wlan0 from my atheros, and also I've tried another physical device, unfortunately, when W10 boots up (I've installed VirtIO from fedora KVM project) there was no flow on tap1 interface, neither on bridge. I'm thinking that virtio-net in FreeBSD11 + Bhyve probably does impact on that behavior (I've tried few previous drivers but they didn't solve networking issues).

Can any one confirm my assumption that bhyve in FreeBSD11 is still out of date enough to be not capable to bring networking in Windows10 ?
 
I'm not sure what other interface you tried, but lots of people report problems using wlan interfaces for virtual guests. I haven't bothered looking into the details but I don't think wireless access points like seeing traffic for multiple MAC addresses from one client. Usually with wireless you have to resort to using NAT.
 
It works like a charm with 0.1.118_2. It seems that driver impact on the wrong behavior. I was able to get it work on win7 and win10 with FreeBSD11+p5.

usdmatt: Everything work. Here is my setup: em0 (internet + nat), ath0+wlan0 for home purpose, tap1-bridge0-wlan0 intentionally I've used wlan0 because I'm to lazy to change my PF setup xD. So I've used working one and according to above the only issue was with the virtio driver for windows.
 
it is an certificate eerror but i downloaded it via firefox in XFCE.
Have it up and running only need some tweeking, also i can not use another VNC viewer than tightVNC which is not the nicest... that is very strange... also got no internet connection i think i did something wrong with the bridge setup...
 
I run into problems... on my machine i tried first on Intel machine with FreeBSD 12 it worked with 2 cpu's and 2Gb ram now i tried on my AMD and it freezes the installation of Windows 10 with all the same settings. When i made the VM 4G Ram it freezes later and with 8Gb of ram it freezes again but again later. What am i doing wrong??

EDIT: Think a memory error... ?
 
it only works on AMD A8 with FreeBSD 12 with 1 cpu core

EDIT: It could be a broken memory problem.
 
Last edited:
You must log in or register to reply here.
Back
Top