bhyve, if_bridge and real IP for VM

[SemFLY]

Server on Hosting VDS/VPS
HOST SYSTEM FreeBSD 11 Release (last update - use subversion)
Provider give IP addresses 200.100.200.100/29 (example adresses)
Gateway 200.100.200.97
DNS 8.8.8.8




Host SYSTEM

# uname -simnr
FreeBSD srv-vds 11.0-RELEASE-p8 amd64 SRV-VDS

Use bhyve
[CODE]# kldload vmm

# ifconfig
em0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
ether 00:50:57:7d:f7:f2
inet 200.100.200.98 netmask 0xfffffff8 broadcast 200.100.200.255
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect (1000baseT <full-duplex>)
status: active

# ifconfig tap0 create
# ifconfig bridge0 create
# ifconfig bridge0 addm em0 addm tap0 up
# ifconfig
tap0: flags=8902<BROADCAST,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80000<LINKSTATE>
ether 00:bd:ee:6e:fa:00
nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
media: Ethernet autoselect
status: no carrier
groups: tap
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 02:ba:05:cb:3e:00
nd6 options=9<PERFORMNUD,IFDISABLED>
groups: bridge
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: em0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 1 priority 128 path cost 20000
member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
ifmaxaddr 0 port 6 priority 128 path cost 2000000

# sysctl -a | grep net.link.bridge
net.link.bridge.ipfw: 0
net.link.bridge.allow_llz_overlap: 0
net.link.bridge.inherit_mac: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 0
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_bridge: 0
net.link.bridge.pfil_onlyip: 0[/CODE]

When guest boot and open shell
On the HOST enter commands

# ifconfig tap0 up
# ifconfig bridge0 up

Guest FreeBSD 11.0

# cat bhyve_freebsd
#!/bin/sh

bhyve -c 1 -s 7,fbuf,tcp=200.100.200.98:5920,wait \
-s 0,hostbridge \
-s 3,ahci-hd,/vmdata/virtual.hdd \
-s 4,ahci-cd,/vmdata/FreeBSD-11.0-RELEASE-amd64-disc1.iso \
-s 10,virtio-net,tap0 \
-s 31,lpc -l bootrom,/vmdata/BHYVE_UEFI.fd \
-m 2G -H -w fbsd

Guest boot and open shell

# ifconfig vtnet 200.100.200.101 255.255.255.248
# ifconfig
vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        ether 00:a0:98:a1:89:eb
        inet 200.100.200.101 netmask 0xfffffff8 broadcast 200.100.200.255
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        media: Ethernet 10Gbase-T <full-duplex>
        status: active
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
        groups: lo

Yes, gateway for VM will be host and use this address (when try use gateway 200.100.200.97 no ping too)

# route add default 200.100.200.98

Guest system

# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
^C
--- 8.8.8.8 ping statistics ---
33 packets transmitted, 0 packets received, 100.0% packet loss

no ping.....


Maybe I'm missing something or Hosting VDS/VPS must turn on some options ?
Help please

 

[aragats]

Yes, gateway for VM will be host and use this address (when try use gateway 200.100.200.97 no ping too)

If you're using a bridge, the gateway is the same as for the host.
If you want to use your host as a gateway, you must use NAT instead of a bridge.
In a case when your host network interface is wireless that's the only way (well, almost ;-) ).

 

[usdmatt]

Is there a specific reason for using the host as the gateway? Traffic will go out via VM -> Host -> Router but will come back Router -> VM ("physically" through the host of course). Seems an odd setup.

It's entirely possible that the hosting provider have restricted this is some way, although they usually just disable VT-x in the host. (Edited: thought host was a real server but it's a vm?)

You don't have any firewall rules that may affect the traffic (or you've already tested with the firewall off) right?
Do you have connectivity between the guest and host?

 

[SemFLY]

If you're using a bridge, the gateway is the same as for the host.
If you want to use your host as a gateway, you must use NAT instead of a bridge.
In a case when your host network interface is wireless that's the only way (well, almost ;-) ).

Yes, i try gateway is the same as for the host, but does't work ))) and i will try host for gateway ))

Is there a specific reason for using the host as the gateway? Traffic will go out via VM -> Host -> Router but will come back Router -> VM ("physically" through the host of course). Seems an odd setup.

It's entirely possible that the hosting provider have restricted this is some way, although they usually just disable VT-x in the host. (Edited: thought host was a real server but it's a vm?)

You don't have any firewall rules that may affect the traffic (or you've already tested with the firewall off) right?
Do you have connectivity between the guest and host?

Yes, on VDS/VPS usually turn off VT-x.
But i send email to support and for me turn on VT-x technology on my VDS.

Firewall is off (ipwf and pf - no in kernel)
From host ping it's ok

From guest on host ping - yes
But in internet - no

 

[aragats]

vtnet0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
ether 00:a0:98:a1:89:eb inet 200.100.200.101 netmask 0xfffffff8 broadcast 200.100.200.255

Does it mean that the guest receives its IP address from a DHCP server?
If yes, that means the bridge is working correctly and you should be able to access the gateway.

 

[SemFLY]

Does it mean that the guest receives its IP address from a DHCP server?
If yes, that means the bridge is working correctly and you should be able to access the gateway.

No... on VDS doesn't use DHCP server, only manual set IP

Host machine - tcpdump
Look at this, ping from guest to internet
no answer to VM guest

# tcpdump -i em0 -v -p icmp
tcpdump: listening on em0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:52:26.491722 IP (tos 0x0, ttl 64, id 36008, offset 0, flags [none], proto ICMP (1), length 84)
    200.100.200.101 > google-public-dns-a.google.com: ICMP echo request, id 20738, seq 0, length 64
19:52:27.555201 IP (tos 0x0, ttl 64, id 36009, offset 0, flags [none], proto ICMP (1), length 84)
    200.100.200.101 > google-public-dns-a.google.com: ICMP echo request, id 20738, seq 1, length 64
19:52:28.618975 IP (tos 0x0, ttl 64, id 36010, offset 0, flags [none], proto ICMP (1), length 84)
    200.100.200.101 > google-public-dns-a.google.com: ICMP echo request, id 20738, seq 2, length 64
19:52:29.680158 IP (tos 0x0, ttl 64, id 36011, offset 0, flags [none], proto ICMP (1), length 84)

 

[SemFLY]

what do you think ?
may be some filtering on hosting provider(router hardware - example mac control) VDS or no, because this only my VDS area ?