Solved Bhyve - Where to start?

gh_origin

Member

Reaction score: 22
Messages: 97

Today I tried to build a cross compiler for Linux on FreeBSD with GCC. It failed with many well known errors but Google doesn't give any definite answer, they if not worked at all or very vague or only applicable for Linux. The GCC build script sucks! So I gave up and think about a virtual machine.

I'm more familiar with VirtualBox but I would like to try the native solution, Bhyve. But, where to start? Are there any graphical front end like virt-manager? Thanks.
 

usdmatt

Daemon

Reaction score: 551
Messages: 1,465

scottro Note that bhyve-firmware is a meta package to install both uefi firmware. You don’t need to explicitly specify the uefi firmware ports as well.
 

jmos

Member

Reaction score: 51
Messages: 73

Where to start is easy - the FreeBSD handbook explains how to set up a Linux client on bhyve:


In my opinion that is superior to any GUI tool etc.: You learn a little bit more about what's going on, and maybe your able to solve problems later on your own.

The first impression of the handbooks page might be a little bit complicated, but it isn't. Really. I've set up one little script afterwards to start my VMs, and added that to my desktop start menu - so everything is clickable.

…and I wouldn't waste another thought on VirtualBox today. Bhyve rocks.
 

diego

Member

Reaction score: 16
Messages: 53

Glad to hear this subject , that I asked by myself around a month ago when I moved all my VM's from Virtualbox to Bhyve :)
I will try to sum up my experience....

1) First place ---> official documentation 21.7. FreeBSD as a Host with bhyve --> (100% agreed with jmos)
2) I would recommend to learn the syntax from command line --> bash script (networking, configuration, etc.)
3) vm-bhyve for managing virtual machines from shell --> easier and faster
4) Graphical management virtual machines --> have a look this post (at the moment vm-bhyve its enough for me :) )

My deployments using bhyve......
- Ubuntu / debian / alpine servers --> better performance that Virtualbox
- Centos7 and CentOS8 servers --> better performance too
- Using docker and docker swarm on debian / centos7 --> better performance
- Linux Mint 19 and Windows10 --> graphical environment using RNC --> Virtualbox is better that Bhyve :( in my laptop

Hope this is helpful
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

P.S. GCC is rubbish. Give LLVM/CLANG a try.
I don't know why you dislike GCC that much. The GCC configure/build script is buggy but at least it's very clear how to build it for me. Regarding LLVM/Clang, even after I searched for hours I still have no idea how to build a cross compiler with it. If you know how, please let me know. Thanks.
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

I have a small page on vm-bhyve (with a Windows guest) that might help a bit. (In my example, I have -t for type, not always necessary, but was for Windows.) http://srobb.net/vm-bhyve.html
Hi. Thank you for your guide. It's really help me get started! But I found a small error on your guide:

Instead of:

cp /usr/local/share/examples/vm-bhyve/* /vm

It is:

cp /usr/local/share/examples/vm-bhyve/* /vm/.templates
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

Glad to hear this subject , that I asked by myself around a month ago when I moved all my VM's from Virtualbox to Bhyve :)
I will try to sum up my experience....

1) First place ---> official documentation 21.7. FreeBSD as a Host with bhyve --> (100% agreed with jmos)
2) I would recommend to learn the syntax from command line --> bash script (networking, configuration, etc.)
3) vm-bhyve for managing virtual machines from shell --> easier and faster
4) Graphical management virtual machines --> have a look this post (at the moment vm-bhyve its enough for me :) )

My deployments using bhyve......
- Ubuntu / debian / alpine servers --> better performance that Virtualbox
- Centos7 and CentOS8 servers --> better performance too
- Using docker and docker swarm on debian / centos7 --> better performance
- Linux Mint 19 and Windows10 --> graphical environment using RNC --> Virtualbox is better that Bhyve :( in my laptop

Hope this is helpful
Err. I intended to run a desktop Linux with this VM. I have setup vm-bhyve but not yet install the OS because I found your post. I have used VirtualBox on FreeBSD for a while and the writing performance of the VM is very bad, especially with big writing job (e.g: apt full upgrade). It's bad for either a ZVOL backed by a RAM VMDK or a dynamic VDI (I don't use fixed VDI, as I need the snapshot ability, via either ZFS or VBOX but there must be snapshot support). I'm now trying to go another route with a Linux jail.
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

Sorry everyone, thanks for your very kind helps but I'm going back to VirtualBox.

The ease of use of Bhyve for me even with vm-bhyve = ZERO

Someone curious about what I have experienced could download this ISO image and try installing it yourselves: https://osdn.net/dl/sparkylinux/sparkylinux-2020.06-x86_64-minimalcli.iso

The console is the worst experience for me. After being stuck because having no idea of what to do next (summary: grub want root to be specified with boot), I also have no idea how to quit that damn console other than... reboot.
 

diego

Member

Reaction score: 16
Messages: 53

The console is the worst experience for me. After being stuck because having no idea of what to do next (summary: grub want root to be specified with boot), I also have no idea how to quit that damn console other than... reboot
Sorry to hear that you gave up too soon, but you would need to read more Bhyve and understand their commands, specific configuration, templates, networking etc.

If the console its "the worst experience for you", you should avoid the command line and go back to Virtualbox, but I said in this post, in my experience, the performance in servers like Ubuntu, Debian, CentOS (including FreeBSD ;) ) is much better than Virtualbox.

Happy to hear if someone runs with Bhyve graphical environments (Windows , Linux) with same or better performance that Virtualbox.
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

If the console its "the worst experience for you", you should avoid the command line and go back to Virtualbox
Let me clarify the console vs the terminal. I'm fine with the terminal for hours but not with that console. That console is just particularly bad. No words could get away from this fact. It's like something from the 80s or 90s (just take the years from someone on this forums, I'm not yet born until the year HK be back to Communism China). The developers designed that console have their minds stuck in the past. They must fix it. Not I adapt myself to work with it. Just that.

BTW, utilizing grub to boot OSes that way always a bad idea for me. It looked like a patchy work, not professional at all. (Please don't said to me that professionals used it, they can use whatever they want, I don't care. The fact is I saw it looks unprofessional so I said it's unprofessional. Just that.)

p/s: I'm not Chinese. The HK event just to show a year. If I'm HK-er, my English would not that bad.
 

usdmatt

Daemon

Reaction score: 551
Messages: 1,465

I’m not sure why grub would be particularly “unprofessional”, other than you appearing to prefer things that look fancier or just work without having to actually do any configuration.

Grub was used by bhyve because is was a fairly straight forward way of supporting many additional operating systems. It’s very configurable and can boot a lot more than just Linux. It isn’t the easiest thing to use though, I will agree there.

These days I would suggest booting most Linux guests in bhyve using the UEFI firmware. That way you can get direct access to the guest console using vnc and install using the standard graphical installer.

Grub is more lightweight though and quicker to boot. I run all my FreeBSD guests via bhyeload even though it supports UEFI/VNC as it’s almost instant, and avoids having to go through an entire UEFI firmware just to load the OS. It also means I can access the guest from boot to multiuser entirely via ssh from anywhere without having to mess about with vnc access to the vga console. I personally find that far more “professional” that relying on nice looking guis because I’m not comfortable in a basic shell.
 

diego

Member

Reaction score: 16
Messages: 53

That console is just particularly bad. No words could get away from this fact.
That's your opinion........however you should focus in the point of this post.

1) You asked for help "Bhyve: where to start" ---> good feedback has been provided ( in my opinion)
2) Bhyve requires a learning curve if you are willing to learn --> (no shortcuts, I'm afraid ;) )
3) People here really appreciate technical questions. Please provide your scripts, logs, etc.
4) This is not a chat about grub, grub2, LILO, etc --> open new post for that
5) Neither console vs terminal vs gui

The bottom line is "Bhyve: where to start". If you want to talk about the rest of your "no technical comments", I would recommend to try other room, IRC, facebook, etc (take a pick)
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

I’m not sure why grub would be particularly “unprofessional”, other than you appearing to prefer things that look fancier or just work without having to actually do any configuration.

Grub was used by bhyve because is was a fairly straight forward way of supporting many additional operating systems. It’s very configurable and can boot a lot more than just Linux. It isn’t the easiest thing to use though, I will agree there.

These days I would suggest booting most Linux guests in bhyve using the UEFI firmware. That way you can get direct access to the guest console using vnc and install using the standard graphical installer.

Grub is more lightweight though and quicker to boot. I run all my FreeBSD guests via bhyeload even though it supports UEFI/VNC as it’s almost instant, and avoids having to go through an entire UEFI firmware just to load the OS. It also means I can access the guest from boot to multiuser entirely via ssh from anywhere without having to mess about with vnc access to the vga console. I personally find that far more “professional” that relying on nice looking guis because I’m not comfortable in a basic shell.
Professional = Instant working + No complicated configuration steps (A nice GUI is fine, but not needed).

Perhaps I will try with bhyveload sometime in the future. This time I will use just the handbook but not the vm-bhyve wrapper.
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

That's your opinion........however you should focus in the point of this post.

1) You asked for help "Bhyve: where to start" ---> good feedback has been provided ( in my opinion)
2) Bhyve requires a learning curve if you are willing to learn --> (no shortcuts, I'm afraid ;) )
3) People here really appreciate technical questions. Please provide your scripts, logs, etc.
4) This is not a chat about grub, grub2, LILO, etc --> open new post for that
5) Neither console vs terminal vs gui

The bottom line is "Bhyve: where to start". If you want to talk about the rest of your "no technical comments", I would recommend to try other room, IRC, facebook, etc (take a pick)
That's your opinion, too. I don't care. That console is bad, this is a fact. No way you could said it's a just an opinion of me to get away from this fact!

"Bhyve: where to start" archived. I will mark the thread as SOLVED and quit this discussion. Bhyve is a no go for me now.

p/s: By console, I mean the console opened by vm console or cu -l. Not the console/virtual terminal that used to display dmesg and I typed my user name and password before startx, not the virtual terminal provided by xfce4-terminal, too. I call all of them as the terminal, distinguish from that console.
 

jmos

Member

Reaction score: 51
Messages: 73

Someone curious about what I have experienced could download this ISO image and try installing it yourselves: https://osdn.net/dl/sparkylinux/sparkylinux-2020.06-x86_64-minimalcli.iso
Checked. For this image you should use the "Graphical UEFI Framebuffer for bhyve Guests" from the handbook (and not the classic "Linux way"). I couldn't choose anything from Grubs menu (no arrow key worked), and had to disconnect my VNC session two times to get the display updated - but that's not really relevant as this image only boots a live system and ends up loged in as user with a running bash. That's it. startx isn't available, and as its a "minimal cli"…

As you expected some comfort…: Is a CLI image really what you wanted to try? There are many other including different desktop environments…
 

kpedersen

Daemon

Reaction score: 825
Messages: 1,827

Professional = Instant working + No complicated configuration steps (A nice GUI is fine, but not needed).
Haha. Unfortunately this isn't really the case. For example compare a home "hub" with an enterprise standard Cisco Switch.

The latter requires an entire series of certified courses (CCNA) in order to configure the darn things ;)
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

Checked. For this image you should use the "Graphical UEFI Framebuffer for bhyve Guests" from the handbook (and not the classic "Linux way"). I couldn't choose anything from Grubs menu (no arrow key worked), and had to disconnect my VNC session two times to get the display updated - but that's not really relevant as this image only boots a live system and ends up loged in as user with a running bash. That's it. startx isn't available, and as its a "minimal cli"…

As you expected some comfort…: Is a CLI image really what you wanted to try? There are many other including different desktop environments…
Yeah. I use the sparkylinux-2020.06-x86_64-minimalcli.iso image because I want to make sure that it would work. I heard people here said bhyve is not fine with graphical so I try cli. Is it wrong?

sparkylinux-2020.06-x86_64-minimalcli.iso is the stable version. I just want to make sure everything would work as expected. The later time I tried, I use the rolling sparkylinux-5.12-x86_64-minimalcli.iso.

p/s: you could install it with sudo sparky-installer, it will let you setup a graphical environment, too.
 
OP
G

gh_origin

Member

Reaction score: 22
Messages: 97

Haha. Unfortunately this isn't really the case. For example compare a home "hub" with an enterprise standard Cisco Switch.

The latter requires an entire series of certified courses (CCNA) in order to configure the darn things ;)
Then the usage of grub is acceptable ;) I just found I could use tmux instead of nmdm console. It's hidden deep inside the wiki of vm-bhyve so I didn't found it on the first time I tried: https://github.com/churchers/vm-bhyve/wiki/Using-tmux

Bhyve developers, please remove that awful nmdm console. It blocked a lot of new comers and potential users :rolleyes:
 
Top