bhyve Graceful shutdown of bhyve guests

ikevin8me

Active Member

Reaction score: 12
Messages: 198

I did some testing and executed a "# poweroff" on the host system. Judging on the speed that the machine shutdown, I don't think the bhyve VMs went through a graceful shutdown.

Is there any way to tell bhyve VMs to gracefully shutdown?

(I think other virtualizers such as VirtualBox has something called "ACPI shutdown". Is there similar way for bhyve?)
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,292
Messages: 38,805

Judging on the speed that the machine shutdown, I don't think the bhyve VMs went through a graceful shutdown.
You probably just created a small script to start the VM. Then the process will just get killed when you shutdown the machine. If you create a proper rc(8) script the shutdown will send a proper stop to the service ( service .... stop), allowing for a graceful shutdown of the VM.

I can highly recommend using sysutils/vm-bhyve to manage your VMs. Then you can simply use vm stop .... It will also issue a stop to the VMs if you shutdown the host too.
 

Phishfry

Beastie's Twin

Reaction score: 2,667
Messages: 5,587

I use the /etc/rc.shutdown.local script facility on the host to shutdown the VM's.
Sending a sigterm to the VM's via bhyve.
Code:
#!/bin/sh
pkill bhyve
sleep 10
So copy the above into a new file named /etc/rc.shutdown.local
Increase sleep as needed for graceful VM shutdown.
Here is how to check.
 

Phishfry

Beastie's Twin

Reaction score: 2,667
Messages: 5,587

I use tmux on my other virt box and I notice some differences.
/etc/rc.shutdown.local
Code:
#!/bin/sh
pkill bhyve
sleep 10
bhyvectl --destroy --vm=freebsd1
bhyvectl --destroy --vm=freebsd2
bhyvectl --destroy --vm=freebsd3
bhyvectl --destroy --vm=freebsd4
#bhyvectl --destroy --vm=freebsd5
bhyvectl --vm=freebsd-i386a --destroy
tmux kill-server
This box uses legacy BIOS VM for i386.

Running bhyve under tmux is a pretty descent method. It does make for a messy /etc/rc.local startup script.
Code:
#!/bin/sh
sleep 6
/usr/local/bin/tmux new-session -d -s freebsd1 'exec bhyve -S -c 4 -m 4G -AHP -u -s 0:0,hostbridge -s 1:0,lpc -s 2:0,ahci-hd,/vm/freebsd/freebsd1.img -s 3:0,ahci-hd,/dev/da0 -s 7:0,passthru,6/0/0 -s 30:0,xhci,tablet -l com1,stdio -l bootrom,/vm/freebsd/BHYVE_UEFI.fd freebsd1'
sleep 6
/usr/local/bin/tmux new-session -d -s freebsd2 'exec bhyve -S -c 4 -m 4G -AHP -u -s 0:0,hostbridge -s 1:0,lpc -s 2:0,ahci-hd,/vm/freebsd/freebsd2.img -s 7:0,passthru,6/0/1 -s 30:0,xhci,tablet -l com1,stdio -l bootrom,/vm/freebsd/BHYVE_UEFI.fd freebsd2'
sleep 5
/usr/local/bin/tmux new-session -d -s freebsd3 'exec bhyve -S -c 4 -m 4G -AHP -u -s 0:0,hostbridge -s 1:0,lpc -s 2:0,ahci-hd,/vm/freebsd/freebsd3.img -s 7:0,passthru,7/0/0 -s 30:0,xhci,tablet -l com1,stdio -l bootrom,/vm/freebsd/BHYVE_UEFI.fd freebsd3'
sleep 5
/usr/local/bin/tmux new-session -d -s freebsd4 'exec bhyve -S -c 4 -m 4G -AHP -u -s 0:0,hostbridge -s 1:0,lpc -s 2:0,ahci-hd,/vm/freebsd/freebsd4.img -s 7:0,passthru,7/0/1 -s 30:0,xhci,tablet -l com1,stdio -l bootrom,/vm/freebsd/BHYVE_UEFI.fd freebsd4'
bhyveload -S -m 4G -c /dev/nmdm0A -d /vm/freebsd/freebsd-i386a.img freebsd-i386a &
sleep 15
bhyve -S -c 8 -m 4G -AHP -s 0:0,hostbridge -s 1:0,lpc -s 5:0,ahci-hd,/vm/freebsd/freebsd-i386a.img -s 7:0,passthru,15/0/0 -l com1,/dev/nmdm0A freebsd-i386a &
 
OP
I

ikevin8me

Active Member

Reaction score: 12
Messages: 198

I'm not using tmux (at this moment).

I tried "pkill bhyve" and it seems to work. However, I think your allowance of 10 secs is too short.

And, I have a question regarding "rc.shutdown.local" This file does not exist by FreeBSD default install. I googled this and can't find any info. Do I just create this file and it will be called when I execute "poweroff"? (I just to make sure because it is bad to kill off all the VMs abruptly if it doesn't work.)
 

Phishfry

Beastie's Twin

Reaction score: 2,667
Messages: 5,587

I think your allowance of 10 secs is too short.
Yes if you have applications that take time to shutdown increase as needed.
My VM's are mostly compile instances so nothing running usually.

Do I just create this file and it will be called when I execute "poweroff"
Yes create the file. It will be called at shutdown -p now.
If you need to reboot I use shutdown -r just to make sure the script fires.
 
OP
I

ikevin8me

Active Member

Reaction score: 12
Messages: 198

I notice that executing "reboot" does not get the "rc.shutdown.local" called. Is there another file for reboot or a workaround?
 

Phishfry

Beastie's Twin

Reaction score: 2,667
Messages: 5,587

Like I said above. shutdown -r now on host for reboots, shutting down the VM's gracefully.
 
OP
I

ikevin8me

Active Member

Reaction score: 12
Messages: 198

Oh I see. I missed that. (I usually just execute "reboot"). That was helpful! Thanks!
 
Top