Solved BSD hypervisor

They may peacefully coexist but you really should learn what you are doing instead of relying on crutches.
Crutches are best when you have too many VM's to manage.
I am not sure both can coexist when both need vmm module.
They also allow you to create VM's without any idea what you are doing. That will not help you at all long term.
 
Since I use several bhyve virtual machines at the same time,I use this script screated by me day by day with the help of the wonderful experienced users of the freebsd-forums :

Code: 
setxkbmap it

bhyvectl --vm=vm7 --force-reset

bhyvectl --vm=vm7 --destroy


vmdisk=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (NM13N4CZ)/ && d{print d}'`

echo "Seagate M3 Portable 1.8 TB ; $vmdisk"


vmdisk0=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2022285F1175)/ && d{print d}'`

echo "CT1000P1SSD8 ; $vmdisk0"


vmdisk1=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (2015020204055E)/ && d{print d}'`

echo "TOSHIBA External USB 3.0 1.8 TB ; $vmdisk1"


vmdisk2=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (20130506005976F)/ && d{print d}'`

echo "TOSHIBA External USB 3.0 932G ; $vmdisk2"


vmdisk3=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (BE0191500218)/ && d{print d}'`

echo "G-DRIVE USB ; $vmdisk3"


vmdisk4=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (38434B4237354B45)/ && d{print d}'`

echo "Elements 25A3 ; $vmdisk4"


vmdisk5=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (WD-WCAV2X597309)/ && d{print d}'`

echo "WDC WD3200AAJS-00L7A0-298 GB ; $vmdisk5"


vmdisk6=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (0774911DDC4200A6)/ && d{print d}'`

echo "SanDisk Cruzer-15GB ; $vmdisk6"


vmdisk7=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (60A44C4138D8F311190A0149)/ && d{print d}'`

echo "Kingston DataTraveler 2.0 ; $vmdisk7"


vmdisk8=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (20140100006C)/ && d{print d}'`

echo "Corsair Force 3 SSD ; $vmdisk8"


vmdisk9=`geom disk list | awk '/^Geom name: /{d=$NF} /^ *ident: (1924E20B2AE5)/ && d{print d}'`

echo "CT500MX500SSD4 ; $vmdisk9"


mount -t ufs /dev/$vmdisk1'p2' /mnt/$vmdisk1'p2'


bhyve -S -c sockets=2,cores=2,threads=2 -m 8G -w -H -A \
-s 0,hostbridge \
-s 1,virtio-blk,/mnt/$vmdisk1'p2'/bhyve/img/Linux/ubuntu2204-cuda510.img,bootindex=1 \
-s 2,nvme,/dev/$vmdisk0 \
-s 3,virtio-blk,/dev/$vmdisk2 \
-s 2,virtio-blk,/dev/$vmdisk3 \
-s 4,virtio-blk,/dev/$vmdisk4 \
-s 5,virtio-blk,/dev/$vmdisk5 \
-s 6,virtio-blk,/dev/$vmdisk6 \
-s 7,passthru,1/0/0 \
-s 8:0,passthru,2/0/0 \
-s 8:1,passthru,2/0/1 \
-s 8:2,passthru,2/0/2 \
-s 8:3,passthru,2/0/3 \
-s 10,virtio-net,tap7 \
-s 11,virtio-9p,sharename=/ \
-s 29,fbuf,tcp=0.0.0.0:5907,w=1600,h=950,wait \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm7 < /dev/null & sleep 2 && vncviewer 0:7

Using a script like this you can :

1) use more virtual machines at the same time because I gave an unique number for each vm that runs.
2) as soon as you ran the script,the vnc viewer will start automatically
3) you will pass thru your USB controller and your GPU
4) you will use a lot of disks without passing them through the USB controller
5) each disk is identified with a precise number
6) if previously you ran the same vm but it hasn't been closed correctly,it will do
7) as soon as you run the script,you will see the model of the disks attached to the VM
8) boot an OS that you have previously installed physically to a disk
9) mount in advance the data partition of one UFS2 disk where previously you have stored your VM file
10) start a virtio-9p channel between the host and the guest if the guest is Linux (Windows does not have the virtio-9p client)
11) make a coffee.
 
Believe me you will need to run this script from time to time,because one of your vms will be not closed correcly and you need to know what is it. To save some time,you can run the script below to reset every vm that could be still loaded inside the memory :

Code: 
setxkbmap it
bhyvectl --vm=vm0 --force-reset
bhyvectl --vm=vm0 --destroy
bhyvectl --vm=vm1 --force-reset
bhyvectl --vm=vm1 --destroy
bhyvectl --vm=vm2 --force-reset
bhyvectl --vm=vm2 --destroy
bhyvectl --vm=vm3 --force-reset
bhyvectl --vm=vm3 --destroy
bhyvectl --vm=vm4 --force-reset
bhyvectl --vm=vm4 --destroy
bhyvectl --vm=vm5 --force-reset
bhyvectl --vm=vm5 --destroy
bhyvectl --vm=vm6 --force-reset
bhyvectl --vm=vm6 --destroy
bhyvectl --vm=vm7 --force-reset
bhyvectl --vm=vm7 --destroy
bhyvectl --vm=vm8 --force-reset
bhyvectl --vm=vm8 --destroy
bhyvectl --vm=vm9 --force-reset
bhyvectl --vm=vm9 --destroy
bhyvectl --vm=vm10 --force-reset
bhyvectl --vm=vm10 --destroy
bhyvectl --vm=vm11 --force-reset
bhyvectl --vm=vm11 --destroy
bhyvectl --vm=vm12 --force-reset
bhyvectl --vm=vm12 --destroy
bhyvectl --vm=vm13 --force-reset
bhyvectl --vm=vm13 --destroy
bhyvectl --vm=vm14 --force-reset
bhyvectl --vm=vm14 --destroy
bhyvectl --vm=vm15 --force-reset
bhyvectl --vm=vm15 --destroy
bhyvectl --vm=vm16 --force-reset
bhyvectl --vm=vm16 --destroy
bhyvectl --vm=vm17 --force-reset
bhyvectl --vm=vm17 --destroy
bhyvectl --vm=vm18 --force-reset
bhyvectl --vm=vm18 --destroy
bhyvectl --vm=vm19 --force-reset
bhyvectl --vm=vm19 --destroy
bhyvectl --vm=vm20 --force-reset
bhyvectl --vm=vm20 --destroy
 
Sivan! said:
When I start with the first command to prepare, I get this error: kldload: can't load vmm: Operation not permitted
Click to expand...
The first thing to check is that your CPU can run VM's

From the handbook:
The easiest way to tell if a processor supports bhyve is to run dmesg or look in /var/run/dmesg.boot for the POPCNT processor feature flag on the Features2 line for AMD® processors or EPT and UG on the VT-x line for Intel® processors.
Click to expand...

Have you confirmed this?

You are not running FreeBSD as a Virtual Machine correct? (ie. vmware,kvm,virtualbox)
You need to be running FreeBSD bare metal.
 
Phishfry said:
The first thing to check is that your CPU can run VM's

From the handbook: [The easiest way to tell if a processor supports bhyve is to run dmesg or look in /var/run/dmesg.boot for the POPCNT processor feature flag on the Features2 line for AMD® processors ...] Have you confirmed this?
Click to expand...
Click to expand...

Checked now. The processor has POPCNT feature, do I have to do something to make sure that the feature is 'enabled' or otherwise active?
Code: 
CPU: AMD Ryzen 3 3200G with Radeon Vega Graphics     (3593.35-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0x810f81  Family=0x17  Model=0x18  Stepping=1
 Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
 Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,[B]POPCNT[/B],AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
  Structured Extended Features=0x209c01a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,RDSEED,ADX,SMAP,CLFLUSHOPT,SHA>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
  SVM: (disabled in BIOS) NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 8589934592 (8192 MB)
avail memory = 6072229888 (5790 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
Phishfry said:
You are not running FreeBSD as a Virtual Machine correct? (ie. vmware,kvm,virtualbox)
You need to be running FreeBSD bare metal.
Click to expand...

It is 13.1 on bare metal. Thank you.
 
Last edited by a moderator:
Phishfry said:
Check your security level and let us know what this reads:

sysctl -n kern.securelevel
Click to expand...

3

Thank you Phishfry


/etc/rc.conf
Code: 
syslogd_flags="-ss -vv"
sendmail_enable="NONE"
sendmail_submint_enable="NONE"
sendmail_outbound_enable="NONE"
sendmail_msp_queue_enable="NONE"
hostname="subdomain.domainname.TLD"
ddclient_enable="YES"
vm_enable="YES"
vm_dir="zfs:zroot/bhyvedata"
keymap="us.kbd"
ifconfig_re0="DHCP"
ifconfig_re0_ipv6="inet6 accept_rtadv"
defaultroute_delay="1"  # Don't wait for a default route in the foreground
moused_enable="YES"
moused_port="/dev/psm0"
moused_type="auto"
moused_enable="YES"
ntpd_enable="YES"
#Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"
dbus_enable="YES"
kdm5_enable="YES"
mysql_enable="yes"
sshd_enable="YES"
apache24_enable="YES"
nginx_enable="NO"
#cloned_interfaces="bridge0 tap0"
#ifconfig_bridge0="addm igb0 addm tap0"
kld_list="amdgpu"
#kld_list="nmdm vmm"
firewall_enable="YES"
firewall_type="client"
firewall_myservices="22/tcp 80/tcp"
firewall_allowservices="any"
sddm_enable="yes"
clear_tmp_enable="YES"                                                                           
syslogd_flags="-ss"                                                                              
pf_enable="YES"                                                                                  
pflog_enable="YES"                                                                               
pflog_logfile="/var/log/pflog"  # where pflogd should store the logfile                          
pflog_flags=""                  # additional flags for pflogd startup                            
nfs_reserved_port_only="NO"                                                                      
linux_enable="YES"                                                                               
kern_securelevel_enable="YES"                                                                    
kern_securelevel="3"                                                                             
#altlog_proglist="named"                                                                         
#kern.elf64.aslr.enable=1                                                                        
#kern.elf32.aslr.enable=1                                                                        
named_enable="YES"                                                                               
# named_enable (bool):              Run named, the DNS server (or NO).                           
# named_program (str):              Path to named, if you want a different one.                  
# named_conf (str):                 Path to the configuration file                               
# named_flags (str):                Use this for flags OTHER than -u and -c
# named_uid (str):                  User to run named as
# named_chrootdir (str):            Chroot directory (or "" not to auto-chroot it)
#                                   Historically, was /var/named
# named_chroot_autoupdate (bool):   Automatically install/update chrooted
#                                   components of named.
# named_symlink_enable (bool):      Symlink the chrooted pid file
# named_wait (bool):                Wait for working name service before exiting
# named_wait_host (str):            Hostname to check if named_wait is enabled
# named_auto_forward (str):         Set up forwarders from /etc/resolv.conf
# named_auto_forward_only (str):    Do "forward only" instead of "forward first"

/etc/sysctl.conf
Code: 
# $FreeBSD$
#
#  This file is read when going to multi-user and its contents piped thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for details.
#

# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
security.bsd.see_other_uids=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.unprivileged_proc_debug=0
vfs.zfs.min_auto_ashift=12
kern.ipc.shm_allow_removed=1
kern.evdev.rcpt_mask=6
security.bsd.see_other_gids=0
security.bsd.see_jail_proc=0
kern.randompid=1
net.link.tap.up_on_open=1
net.inet.ip.fw.enable=1
net.inet.ip.fw.verbose=1
net.inet.ip.fw.verbose_limit=5

/boot/loader.conf
Code: 
kern.vty=vt
kern.geom.label.disk_ident.enable="0"
kern.geom.label.gptid.enable="0"
cryptodev_load="YES"
zfs_load="YES"
sysctlbyname_improved_load="YES"
security.bsd.allow_destructive_dtrace=0
autoboot_delay=1       # OPTION '-1' MEANS NO WAIT AND 'NO' MEANS INFINITE WAIT
fuse_load="YES"
cfumass_load="YES"
/etc/ttys
Code: 
console none                            unknown off insecure
#
ttyv0   "/usr/libexec/getty Pc"         xterm   onifexists secure
# Virtual terminals
ttyv1   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv2   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv3   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv4   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv5   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv6   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv7   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv8   "/usr/libexec/getty Pc"         xterm   onifexists secure
# Serial terminals
# The 'dialup' keyword identifies dialin lines to login, fingerd etc.
ttyu0   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
ttyu1   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
ttyu2   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
ttyu3   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
# Dumb console
dcons   "/usr/libexec/getty std.9600"   vt100   off secure
# Xen Virtual console
xc0     "/usr/libexec/getty Pc"         xterm   onifconsole secure
# RISC-V HTIF console
rcons   "/usr/libexec/getty std.9600"   vt100   onifconsole secure

# cat /rc.conf.hardening # output differs from etc rc conf, don't know how to merge.
Code: 
clear_tmp_enable="YES"
syslogd_flags="-ss"
sendmail_enable="NONE"

# cat /sysctl.conf.hardening # output differs from etc sysctl conf, don't know how to merge.
Code: 
security.bsd.see_other_uids=0
security.bsd.see_other_gids=0
security.bsd.see_jail_proc=0
security.bsd.unprivileged_read_msgbuf=0
security.bsd.unprivileged_proc_debug=0
kern.randompid=1
kern.elf32.aslr.enable=1
kern.elf32.aslr.pie_enable=1
kern.elf32.aslr.honor_sbrk=0
kern.elf64.aslr.enable=1
kern.elf64.aslr.pie_enable=1
kern.elf64.aslr.honor_sbrk=0

/ttys.hardening
Code: 
console none                            unknown off insecure
#
ttyv0   "/usr/libexec/getty Pc"         xterm   onifexists secure
# Virtual terminals
ttyv1   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv2   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv3   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv4   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv5   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv6   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv7   "/usr/libexec/getty Pc"         xterm   onifexists secure
ttyv8   "/usr/libexec/getty Pc"         xterm   onifexists secure
# Serial terminals
# The 'dialup' keyword identifies dialin lines to login, fingerd etc.
ttyu0   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
ttyu1   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
ttyu2   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
ttyu3   "/usr/libexec/getty 3wire"      vt100   onifconsole secure
# Dumb console
dcons   "/usr/libexec/getty std.9600"   vt100   off secure
# Xen Virtual console
xc0     "/usr/libexec/getty Pc"         xterm   onifconsole secure
# RISC-V HTIF console
rcons   "/usr/libexec/getty std.9600"   vt100   onifconsole secure

If it is pertinent, I have a similar problem with mysql80 that I posted in a relevant thread, though it had already been marked "solved"
 
Last edited by a moderator:
Secure Level 3 will not allow you to hand load modules.

Put the setting in your /boot/loader.conf instead.
vmm_load="YES"
 
Phishfry said:
Security Level 3 will not allow you to hand load modules.

Put the setting in your /boot/loader.conf instead.
vmm_load="YES"
Click to expand...

Added that, rebooted, #kldload vmm still returns Operation not permitted error. Then uncommented kld_list="nmdm vmm" in /etc/rc.conf still the same error.
 
It looks to me like you are running HardenedBSD. That could be troublesome.
You have hardened so well you can't run anything without knowing how to unharden.
You should really try this on bone stock FreeBSD.
 
Generally its a good idea to get your 'appliance' running before hardening. That is my philosophy.
That way you can tell when you have hardened too much when your appliance no longer works.

By 'appliance' I mean 'bhyve server' in your particular case.
 
What is the proper method of de-hardening (for the length of time that it takes to install Bhyve, mysql etc) and to set it to bone stock condition?

Run /usr/libexec/bsdinstall/hardening with all the options unchecked ? And, comment out the additional entries in rc, sys, loader conf and conf dot hardening files?

more important, vmm is probably running. Even as the kldload command returns an operation not permitted error, the following commands show results

# sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d guest.img guestname # I proceeded ahead to reach this step#
*** vmm.ko is not loaded
Click to expand...
# kldload vmm.ko
kldload: can't load vmm.ko: Operation not permitted
Click to expand...
# kldload vmm
kldload: can't load vmm: Operation not permitted
Click to expand...
# kldstat -v -m vmm
kldstat: can't find module vmm: No such file or directory
Click to expand...
# kldstat -v -n vmm # -n instead of -m
Id Refs Address Size Name
3 1 0xffffffff82135000 582d68 vmm.ko (/boot/kernel/vmm.ko)
Contains modules:
Id Name
5 acpi/ivhd
4 pci/amdiommu
3 pci/ppt
Click to expand...

No response requested on the following as the following is resolved, as updated in the next post:

I had remembered that something strange happened when I commented
#cloned_interfaces="bridge0 tap0" #ifconfig_bridge0="addm igb0 addm tap0"

I tried un-commenting these lines in rc conf, the result is that display resolution on the next boot dropped to 1024x800 and there are some strange issues, there is a terminal running at the background that overlaps with the browser / kolsole, as seen in the attached screenshot.

terminal overlap IMG_20220712_231115.jpgterminal overlap IMG_20220712_231333.jpg

There seems to be something running now. The screenshots also include screenshots taken by camera as I am typing this message:

terminal overlap IMG_20220713_003117.jpgteminal overlap IMG_20220713_003636.jpg

(I commented out those two lines in rc conf again, tried rebooting, powering off, restarting, display resolution remains low, the overlapping terminal window persists.)


Thank you
 
With help from a friend:

/boot/loader.conf
commente out vmm_load


/etc/rc.conf

commented out
Code: 
#vm_enable="YES"
#vm_dir="zfs:zroot/bhyvedata"#vm_enable="YES"
#vm_dir="zfs:zroot/bhyvedata"
#kld_list="nmdm vmm"

there are two conflicting entries for firewall in rc.conf
Code: 
firewall_enable="YES"
firewall_type="client"
firewall_myservices="22/tcp 80/tcp"
firewall_allowservices="any"
and also this
Code: 
pf_enable="YES"
pflog_enable="YES"
pflog_logfile="/var/log/pflog"  # where pflogd should store the logfile
pflog_flags=""
(I recall, but not sure) apache24 required one
nginx required another four entries

commented out firewall* entries

After reboot, screen resolution is fine. The terminal does not overlap.
 
Sivan! said:
#kld_list="nmdm vmm"
Click to expand...

Lots of random entries.
I know it can be tough being new to FreeBSD.

Let me explain one little part.
Sivan! said:
/boot/loader.conf
commente out vmm_load
Click to expand...
So you were loading vmm twice.
That was a good catch by your friend. I did not notice you were already loading it in /etc/rc.conf

So now you have not enabled VMM at all. How the heck is bhyve even working?
Well Bhyve is smart and loads the VMM module for you.

But there is case for loading VMM in /boot/loader.conf.
That is PCI pass-thru to VM's
You see loader.conf comes early in bootup. You want PCI pass-thru early in boot process.
So you load the VMM module in /boot/loader.conf
I usually add the nmdm device as well although that is not really needed early in the boot process like pass-thru.
 
Phishfry said:
They may peacefully coexist but you really should learn what you are doing instead of relying on crutches.
Crutches are best when you have too many VM's to manage.
I am not sure both can coexist when both need vmm module.
They also allow you to create VM's without any idea what you are doing. That will not help you at all long term.
Click to expand...
You are wrong and this sort of nonsense is unhelpful. Do something the easy way then do it more in depth, this is the best method. The only time it doesn't work is if someone doesn't realise there is more depth and only uses the easy way, Sivan! is not one of these people.

His first post was about wanting to try bhyve and you came back stating to use jails. Pointless. But it looks like Phishfry is being slightly less "know it all" now.

I see the documentation at FreeBSD is still ridiculously insecure, in the emotional sense, and doesn't mention that Linux KVM/libvirt can host FreeBSD (no, I'm not talking about in depth instructions just a mention that it can).
 
If learning fundamentals are wrong than I don't want to be right.
I have seen your caustic replys and I laught at you.
 
Don't use wrappers. They don't help you to understand how the parameters work together. At the end I've enriched the raw parameters with all the comforts I need without using wrappers.
 
I really want to repond to the critism. I am not thick skinned so I got mad.
I do not consider myself a know it all. There is alot I don't know.
I don't hesitate to ask when I have questions.
The original poster here reached out in a private conversation and I tried to steer them back to the original post.

I do know about bhyve.
I feel that a machine with low system resources is not an ideal candidate for bhyve.
Jails on the otherhand are much better suited to machines with limited resources.
That is just my opinion. I did not mean to come off as condescending.
You have a tool that does 80% of what a VM can do with dramatically less system resources.
That is why I recommended Jails.

It may not have answered the question but if someone is just starting off I feel its best to advise of a similar more efficient path (In my Opinion).
 
Apologies, if my questions here have unwittingly caused any differences here.

I have lowered the kernel security level, tried bhyve installation again, kldload vmm as the first command did not return errors. However when I reached this step there is an error

sh /usr/share/examples/bhyve/vmrun.sh -c 4 -m 1024M -t tap0 -d guest.img -i -I FreeBSD-13.0-RELEASE-amd64-bootonly.iso guestname
*** vmm.ko is not loaded
Click to expand...
# kldload vmm
kldload: can't load vmm: module already loaded or in kernel
Click to expand...

This is a desktop computer with rhyzen4 has the required processor features.
# kldstat -m vmm
kldstat: can't find module vmm: No such file or directory
Click to expand...

# kldstat -n vmm
Click to expand...
Id Refs Address Size Name
40 1 0xffffffff83400000 53d438 vmm.ko
Click to expand...

All this, while working on alternate solutions.

Thank you.
 
Newer AMD processors like Ryzen4 might need an extra setting.

First check to see if amdvi is working: sysctl hw.vmm.amdvi

If this returns no entries then try setting:
/boot/loader.conf
hw.vmm.amdvi.enable=1

Should look like this in dmesg when working:
ivhd0: <AMD-Vi/IOMMU ivhd with EFR> on acpi0
 
Phishfry said:
Newer AMD processors like Ryzen4 might need an extra setting.

First check to see if amdvi is working: sysctl hw.vmm.amdvi

If this returns no entries then try setting:
/boot/loader.conf
hw.vmm.amdvi.enable=1

Should look like this in dmesg when working:
ivhd0: <AMD-Vi/IOMMU ivhd with EFR> on acpi0
Click to expand...

Yes it is working:

# sysctl hw.vmm.amdvi
hw.vmm.amdvi.domain_id: 0
hw.vmm.amdvi.disable_io_fault: 0
hw.vmm.amdvi.ptp_level: 4
hw.vmm.amdvi.host_ptp: 1
hw.vmm.amdvi.enable: 0
hw.vmm.amdvi.count: 1
Click to expand...

dmesg | grep AMD
CPU: AMD Ryzen 3 3200G with Radeon Vega Graphics (3593.42-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x810f81 Family=0x17 Model=0x18 Stepping=1
AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
hdac1: <AMD Raven HDA Controller> mem 0xfcb80000-0xfcb87fff irq 54 at device 0.6 on pci8
ahci1: <AMD KERNCZ AHCI SATA controller> mem 0xfce00000-0xfce007ff irq 54 at device 0.0 on pci9
ugen0.1: <AMD XHCI root HUB> at usbus0
ugen1.1: <AMD XHCI root HUB> at usbus1
ugen2.1: <AMD XHCI root HUB> at usbus2
uhub0: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub1: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2
uhub2: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
intsmb0: <AMD FCH SMBus Controller> at device 20.0 on pci0
WARNING acrtc_attach->pflip_status != AMDGPU_FLIP_NONE failed at /wrkdirs/usr/ports/graphics/drm-fbsd13-kmod/work/drm-kmod-drm_v5.4.191_1/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c:5931
WARNING acrtc_attach->pflip_status != AMDGPU_FLIP_NONE failed at /wrkdirs/usr/ports/graphics/drm-fbsd13-kmod/work/drm-kmod-drm_v5.4.191_1/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c:5931
CPU: AMD Ryzen 3 3200G with Radeon Vega Graphics (3593.37-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x810f81 Family=0x17 Model=0x18 Stepping=1
AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
AMD Features2=0x35c233ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX>
AMD Extended Feature Extensions ID EBX=0x1007<CLZERO,IRPerf,XSaveErPtr,IBPB>
hdac1: <AMD Raven HDA Controller> mem 0xfcb80000-0xfcb87fff irq 54 at device 0.6 on pci8
ahci1: <AMD KERNCZ AHCI SATA controller> mem 0xfce00000-0xfce007ff irq 54 at device 0.0 on pci9
ugen2.1: <AMD XHCI root HUB> at usbus2
ugen0.1: <AMD XHCI root HUB> at usbus0
ugen1.1: <AMD XHCI root HUB> at usbus1
uhub1: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub2: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus1
uhub0: <AMD XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus2
intsmb0: <AMD FCH SMBus Controller> at device 20.0 on pci0
AMD-Vi: IVRS Info VAsize = 64 PAsize = 48 GVAsize = 2 flags:0
ivhd0: <AMD-Vi/IOMMU ivhd with EFR> on acpi0
Click to expand...

The last line confirms to what you said it should look like.

Thank you.
 
Sivan! said:
I don't see SVM mode in my BIOS, but someone helped me try Virtualization in this machine before. It will work. Thank you.
Click to expand...
Whether there's an SVM setting in the BIOS or not is immaterial. All the setting does is enable a bit in the MSR, which can be enabled by FreeBSD in svm_modinit(), by calling svm_enable(), anyway.

But this is not the only feature bhyve requires. It requires the popcnt instruction. Older Intel and AMD processors don't support it. AMD started supporting the popcnt instruction in Phenom II. You can tell if your processor supports bhyve by checking dmesg | egrep 'VT-x|SVM|POPCNT'. For Intel you should see VT-x and POPCNT. For AMD you should see SVM and POPCNT.
 
You must log in or register to reply here.
Back
Top