bhyve vm cannot access internet

yhq_34

Member

Thanks: 1
Messages: 60

#1
Hi,

I create a bridge network for bhyve vm host, but this host network has problem, please see below conf file, anyone can help?
Code:
#ifconfig
wlan0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 90:48:9a:aa:f5:09
        hwaddr 90:48:9a:aa:f5:09
        inet6 fe80::9248:9aff:feaa:f509%wlan0 prefixlen 64 scopeid 0x3
        inet 192.168.2.102 netmask 0xffffff00 broadcast 192.168.2.255
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
        media: IEEE 802.11 Wireless Ethernet MCS mode 11ng
        status: associated
        ssid 123 channel 2 (2417 MHz 11g ht/20) bssid c8:3a:35:1d:8a:d0
        regdomain 96 indoor ecm authmode WPA2/802.11i privacy ON
        deftxkey UNDEF TKIP 2:128-bit txpower 20 bmiss 7 scanvalid 60
        protmode CTS ampdulimit 64k ampdudensity 8 shortgi -stbctx stbcrx wme
        burst roaming MANUAL
        groups: wlan
vm-public: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        ether 72:15:41:59:b5:d1
        nd6 options=1<PERFORMNUD>
        groups: bridge vm-switch viid-4c918@
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000000
        member: wlan0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 3 priority 128 path cost 66666
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: vmnet-freebsd-dns-0-public
        options=80000<LINKSTATE>
        ether 00:bd:e6:12:f8:00
        hwaddr 00:bd:e6:12:f8:00
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        groups: tap vm-port
        Opened by PID 1714

# vm switch info public
------------------------
Virtual Switch: public
------------------------
  type: auto
  ident: vm-public
  vlan: -
  physical-ports: wlan0
  bytes-in: 10394 (10.150K)
  bytes-out: 1798784 (1.715M)

  virtual-port
    device: tap0
    vm: freebsd
 

ikbendeman

Well-Known Member

Thanks: 17
Messages: 355

#2
Not necessarily the person to help you here but can you post your netstat -r for host and guest? (Never setup bHyve, myself)
 
OP
OP
Y

yhq_34

Member

Thanks: 1
Messages: 60

#3
host
Code:
#netstat -r
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.2.1        UGS       wlan0
localhost          link#2             UH          lo0
192.168.2.0/24     link#3             U         wlan0
192.168.2.102      link#3             UHS         lo0

Internet6:
Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
localhost          link#2             UH          lo0
::ffff:0.0.0.0/96  localhost          UGRS        lo0
fe80::/10          localhost          UGRS        lo0
fe80::%lo0/64      link#2             U           lo0
fe80::1%lo0        link#2             UHS         lo0
fe80::%wlan0/64    link#3             U         wlan0
fe80::9248:9aff:fe link#3             UHS         lo0
ff02::/16          localhost          UGRS        lo0
guest
Code:
#netstat -r 
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.2.1        UGS      vtnet0
localhost          link#2             UH          lo0
192.168.2.0/24     link#1             U        vtnet0
192.168.2.51       link#1             UHS         lo0

Internet6:
Destination        Gateway            Flags     Netif Expire
::/96              localhost          UGRS        lo0
localhost          link#2             UH          lo0
::ffff:0.0.0.0/96  localhost          UGRS        lo0
fe80::/10          localhost          UGRS        lo0
fe80::%vtnet0/64   link#1             U        vtnet0
fe80::5a9c:fcff:fe link#1             UHS         lo0
fe80::%lo0/64      link#2             U           lo0
fe80::1%lo0        link#2             UHS         lo0
ff02::/16          localhost          UGRS        lo0
 

ikbendeman

Well-Known Member

Thanks: 17
Messages: 355

#6
Why are you using dhcp in bHyve? Is that a necessity for you for some reason? You'll probably have to run a DHCP server on that tap0, from host that is, if you must have it. Otherwise, make sure you don't have pf, ipfw, or some other hinderance on the passing of packets. I think the tap method is a NAT method for the vm, where you would require shared. I'm sure it's in the man page but I would expect you'd need another loopback, as well. Again, I've not set this up personally, but yeah... my best 2 cents.
 

ikbendeman

Well-Known Member

Thanks: 17
Messages: 355

#7
From Serverfault :
Code:
 4
down vote

You likely need to do:

ifconfig tap0 up

and/or add net.link.tap.up_on_open=1 to /etc/sysctl.conf to make it auto-up.

Normally, the tap interface doesn't come up until you give it an IP.

This should make the bridge work.

To configure the bridge in rc.conf:

cloned_interfaces="bridge0 tap0"
ifconfig_bridge0="addm igb0 addm tap0 up"

You also need:

if_bridge_load="YES"
if_tap_load="YES"

in /boot/loader.conf so that tap can be created on the fly.
FreeBSD Handbook's Guide

Sorry, I didn't parse it.
 
OP
OP
Y

yhq_34

Member

Thanks: 1
Messages: 60

#8
Guest machine need to access internet.
I have created bridge interface (public), normally it should OK. I doubted host using WIFI that cause the problem.
Code:
vm switch create public
vm switch add public wlan0
 

ikbendeman

Well-Known Member

Thanks: 17
Messages: 355

#9
Do you have the sysctl variables set? vmm.ko loaded? Any indications from dmesg? Everything set properly in rc.conf? Have you read the man pages?

Wifi bridges can cause certain problems within VM's. It's been a recurring theme among many different virtualization technologies that, supposedly has gotten better. I don't even have enough data from you question to try to diagnose it myself. You'll have to set these things properly in the host as well as the guest but right now I can't even tell if your problem is coming from the host or guest. Is your host machine your gateway or do you have a gateway elsewhere on your network?
 

ikbendeman

Well-Known Member

Thanks: 17
Messages: 355

#11
Add the lines from above; replace igb0 with wlan0. If you're using wlan0 you could also add an apbridge device. Setting the devices on monitor mode will help. You're going to have to read the ifconfig(8) page and bhyve(8), for starters.
 
Top