bhyve bridge networking does not work

fnoyanisi

fnoyanisi

Hi there,

I have a FreeBSD-RELEASE host machine (hostname = beastie) and I would like to run a FreeBSD-CURRENT on a bhyve VM (hostname = patch). The host was configured as described in the relevant section of the handbook (Section 21.7)

I am using iohyve as a bhyve wraper and I think association of the tap0 interface with VM is okay
Code: 
root@beastie:~/vm # iohyve taplist
Listing all network taps:
zroot/iohyve/freebsd-current......tap0

My re0 interface is assigned a static IP from the 192.168.1.0/24 pool and below is my bridge0 and tap0 configurations
Code: 
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 02:86:c2:97:9d:00
        inet 192.168.1.64 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000000
        member: re0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        groups: bridge
        nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=8903<UP,BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80000<LINKSTATE>
        ether 00:bd:5a:90:26:00
        groups: tap
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
The IP address of bridge0 is assigned via DHCP ( dhclient bridge0).

On the VM side, I have vtnet0 interface along with lo0. I added 8.8.8.8 as the default nameserver.

This is where I am stuck...I want to configure internet access for the VM.

1 ) I followed the steps mentioned in the Bridged Network section of this tutorial but it does not work. I cannot set my default gateway hence getting no DHCPOFFERS (I tried setting the defaultgateway as 192.168.1.254, which is the gateway for the host, and IP address of bridge0, both failed). I used
Code: 
route add default 192.168.1.254
2 ) Seems like I need some PF rules but cannot figure it out, any help with this would be appreciated.

Thanks
 
No pf is needed in configuration like this. Please post your /etc/rc.conf and /etc/sysctl.conf.

Also a iohyve getall of your VM and the content of /etc/rc.conf inside this VM.
 
Did you add the sysctl entry like the handboot shows?
net.link.tap.up_on_open=1
www.freebsd.org

Chapter 24. Virtualization

Virtualization software allows multiple operating systems to run simultaneously on the same computer
www.freebsd.org www.freebsd.org
The reason I ask is because I see 'no carrier' on your tap.
 
I also feel that using an external program for bhyve is counter-productive.
For only a single VM you should really use the manual method so you get to know how things work.
Tools like iohyve and vm-bhyve are fine when you have many VM's but for a small number of VM's I feel its best to get a solid foundation. The handbook is very relevant and with the UEFI VM's it really is simple. Download the UEFI bhyve firmware and start the VM. I use /etc/rc.local to start my 5 VM's on bootup.
I also think bridge and tap are not the best route. Using a separate network interface for each VM is ideal. Pass them through to VM.
Currently I use Intel 4 port gigabit ethernet cards for my VM's.
Do you have any old ethernet cards laying around? That will simplify things in my opinon.
 
Phishfry said:
I also feel that using an external program for bhyve is counter-productive.
For only a single VM you should really use the manual method so you get to know how things work.
Tools like iohyve and vm-bhyve are fine when you have many VM's but for a small number of VM's I feel its best to get a solid foundation. The handbook is very relevant and with the UEFI VM's it really is simple. Download the UEFI bhyve firmware and start the VM. I use /etc/rc.local to start my 5 VM's on bootup.
I also think bridge and tap are not the best route. Using a separate network interface for each VM is ideal. Pass them through to VM.
Currently I use Intel 4 port gigabit ethernet cards for my VM's.
Do you have any old ethernet cards laying around? That will simplify things in my opinon.
Click to expand...

sysutils/iohyve works very well for me. I'm very pleased with that one. So that isn't the problem.
 
Hi. Thanks for the replies Emrion and Phishfry

Below is my bhyve VM configuration along with the contents of /etc/sysctl.conf and /etc/rc.conf files (this is the host)
Code: 
root@beastie:~ # iohyve getall freebsd-current
Getting freebsd-current iohyve properties...
bargs            -A_-H_-P
bhyve_path       /usr/sbin/bhyve
boot             0
con              nmdm0
cpu              1
description      FreeBSD-CURRENT
install          no
loader           bhyveload
name             freebsd-current
os               FreeBSD
persist          1
ram              1024M
size             35GB
tap              tap0
template         NO
utc              YES
vnc              NO
vnc_h            600
vnc_ip           127.0.0.1
vnc_tablet       NO
vnc_w            800
vnc_wait         NO
root@beastie:~ # sysctl net.link.tap.up_on_open
net.link.tap.up_on_open: 1
root@beastie:~ # cat /etc/sysctl.conf
# $FreeBSD: releng/12.1/sbin/sysctl/sysctl.conf 337624 2018-08-11 13:28:03Z brd $
#
#  This file is read when going to multi-user and its contents piped thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for details.
#

# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0
vfs.zfs.min_auto_ashift=12

net.link.tap.up_on_open=1
root@beastie:~ # cat /etc/rc.conf
clear_tmp_enable="YES"
sendmail_enable="NONE"
hostname="beastie"
sshd_enable="YES"
powerd_enable="YES"

# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"
zfs_enable="YES"

# static ip
# tap0 ip is for bhyve
defaultrouter="192.168.1.254"
ifconfig_re0="inet 192.168.1.59 netmask 255.255.255.0"

# Server load monitoring
netdata_enable="YES"

# web server
nginx_enable="YES"

# jails and byhve
# bridge0 and tap0 are required for bhyve
iocage_enable="YES"
cloned_interfaces="tap0 lo1"

pf_enable="YES"
pflog_enable="YES"
gateway_enable="YES"

# bhyve / iohyve
iohyve_enable="YES"
iohyve_flags="kmod=1 net=em0"
kld_list="nmdm vmm"

Nothing in fancy in VM's /etc/rc.conf file
Code: 
root@patch:~ # cat /etc/rc.conf
clear_tmp_enable="YES"
sendmail_enable="NONE"
hostname="patch"
ifconfig_vtnet0="DHCP"
sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="AUTO"

root@patch:~ #

Even without any PF rules, I am not able to acquire any DHCP leases (I have an DHCP assigned IP for bridge0 and I try to acquire a new IP from within the VM)
 
Holy moly! Rebooting the guest without changing any settings solved the problem....

I spent all night trying to figure out this.

For future reference, the "Bridged Networking" option mentioned in this article works fine

empt1e.blogspot.com

Bhyve Networking Options

Once in a while people on the #bhyve IRC channel on freenode ask questions about bhyve networking configuration, i.e. how to configure thin...
empt1e.blogspot.com empt1e.blogspot.com
 
Phishfry said:
I also feel that using an external program for bhyve is counter-productive.
...
Click to expand...
Not only that, but I also noticed that vm-bhyve, for example, makes things easy only with generic configuration -- when all you need is just get bhyve up and running.
The project's very goal seems to be exactly that: get that extra "load" from the shoulders of some user who is not going to use advanced options "in the very near future". Which seems to apply to a good deal of bhyve users -- ppl who resort to FreeBSD because of bhyve.
Which shows, of course, the superiority of bhyve in some ways over other VM engines, that's true.???

BTW, I DO have 2 nic in my machine... (Realtek, yes, but better than nothing). However, at my workplace I only have one cable, alas :) :). So I'll have to use the virtual tap network. But for me it's just about having a working MS Word for format compatibility. (Shame, too, because LibreOffice is superior in SO many ways actually. But "the world" doesn't want to consider that... sliding off topic here, sorry).
 
You must log in or register to reply here.
Back
Top