Solved Bhyve windows guest network access?

S

Solaris

I have windows server 2016 installed as a guest on bhyve FreeBSD 11.1 Pre-release.
It managed to get local lan access but failed to get nat for accessing the internet through igb0.

The PF already include this only rule :
Code: 
nat on igb0 from {172.16.0.0/24} to any -> (igb0)

ifconfig
Code: 
ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:e2:a5:dc
        hwaddr a0:36:9f:e2:a5:dc
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:e2:a5:de
        hwaddr a0:36:9f:e2:a5:de
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 18:66:da:8e:52:9d
        hwaddr 18:66:da:8e:52:9d
        inet 163.245.253.15 netmask 0xffffff00 broadcast 163.172.253.255 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 18:66:da:8e:52:9e
        hwaddr 18:66:da:8e:52:9e
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo 
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
        groups: pflog 
pfsync0: flags=0<> metric 0 mtu 1500
        groups: pfsync 
        syncpeer: 0.0.0.0 maxupd: 128 defer: off
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: vm-public
        ether 02:5a:71:50:7c:00
        inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255 
        nd6 options=1<PERFORMNUD>
        groups: bridge 
        id 00:00:00:00:00:00 priority 0 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0

Code: 
# vm switch info public
------------------------
Virtual Switch: public
------------------------
  type: auto
  ident: bridge0
  vlan: -
  nat: yes
  physical-ports: 
  bytes-in: 14295962 (13.633M)
  bytes-out: 110889522 (105.752M)

Code: 
# vm list
NAME            DATASTORE       LOADER      CPU    MEMORY    VNC                  AUTOSTART    STATE
jenkins         default         uefi        16     16G       -                    No           Stopped

How to proceed with my setup?
 
When the VM is running, the tap0 being created and i have already restarted the PF,

Code: 
ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:e2:a5:dc
        hwaddr a0:36:9f:e2:a5:dc
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
ix1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether a0:36:9f:e2:a5:de
        hwaddr a0:36:9f:e2:a5:de
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 18:66:da:8e:52:9d
        hwaddr 18:66:da:8e:52:9d
        inet 163.245.253.15 netmask 0xffffff00 broadcast 163.172.253.255 
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
igb1: flags=8c02<BROADCAST,OACTIVE,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=6403bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 18:66:da:8e:52:9e
        hwaddr 18:66:da:8e:52:9e
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: no carrier
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128 
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5 
        inet 127.0.0.1 netmask 0xff000000 
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo 
pflog0: flags=141<UP,RUNNING,PROMISC> metric 0 mtu 33160
        groups: pflog 
pfsync0: flags=0<> metric 0 mtu 1500
        groups: pfsync 
        syncpeer: 0.0.0.0 maxupd: 128 defer: off
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: vm-public
        ether 02:5a:71:50:7c:00
        inet 172.16.0.1 netmask 0xffffff00 broadcast 172.16.0.255 
        nd6 options=1<PERFORMNUD>
        groups: bridge 
        id 00:00:00:00:00:00 priority 0 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 0 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 9 priority 128 path cost 2000000
tap0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: vmnet-jenkins-0-public
        options=80000<LINKSTATE>
        ether 00:bd:3a:42:f9:00
        hwaddr 00:bd:3a:42:f9:00
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet autoselect
        status: active
        groups: tap 
        Opened by PID 1556

and vm switch is changed :
Code: 
# vm switch info public
------------------------
Virtual Switch: public
------------------------
  type: auto
  ident: bridge0
  vlan: -
  nat: yes
  physical-ports: 
  bytes-in: 318956 (311.480K)
  bytes-out: 1895239 (1.807M)

  virtual-port
    device: tap0
    vm: jenkins

dnsmasq already configured and running and the VM able to obtain the LAN IP from the dnsmasq, of course virtio NETKVM already installed on windows.
 
In my opinion, it's wrong assumption to use NAT with bridge LAN in this case but I could mistake.
I have different configuration where NAT is used by one of my guest (firewall guest). So I pass traffic through the guest firewall and then NAT again to proper guests.

Firstly, to be sure PF rules are correct, pass all network traffic. I mean to use
Code: 
pass all log
in /etc/pf.conf.
Secondly, run
Code: 
ping 8.8.8.8
on windows guest machine.
Thirdly, try to use host (bridge0) IP instead of the LAN address
Code: 
nat on igb0 from 172.16.0.1 to any -> (igb0)
. Then on the guest (Windows in this case) use the address 172.16.0.1 as gateway.
Anyway, if everything is correct, you should be able to ping 172.16.0.1 from the guest at first.

By the way. Is the forwarding enabled in system configuration?
Code: 
user@vm:~ % sysctl net.inet.ip.forwarding
net.inet.ip.forwarding: 1
 
Done all above, and fixed with dnsmasq wrong settings, minimizing the dnsmasq config into

Code: 
# vm-bhyve dhcp
#port=0
#domain-needed
#no-resolv
#except-interface=lo0
#bind-interfaces
#local-service
#dhcp-authoritative

interface=bridge0
dhcp-range=172.16.0.10,172.16.0.254
 
You must log in or register to reply here.
Back
Top