Bhyve, Public IP on guest visible to all. How?

aqqaqq

New Member


Messages: 6

Hello,
I want guest VM to be fully accessible to the internet.
Freebsd 12 with Bhyve, installed windows as guest. Set up static IP. Internet is working. Online tools report good IP (the one from guest, not host)
Guest gateway is set to host IP.
When i set gateway to the one i got from Hetzner hosting company i get no internet.
I cloned host MAC address on windows guest, but internet stopped working at all (-s 5,virtio-net,tap10,mac=70:85:c2:fd:c5:ae )

Guest can ping host.
Guest can ping internet.
Host can not ping guest.
Guest not visible to outside world.
I have tried to configure it after searching forums but no luck. Can anybody help me with that?



1588581620517.png

rc.conf:
1588581653431.png

pf.conf
1588581884184.png

Windows:
1588584440309.png
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,679
Messages: 39,278

Try moving the IP address from igb0 to bridge0 (so no addresses are assigned to igb0).
 
OP
A

aqqaqq

New Member


Messages: 6

Igb0 must have it's main IP assigned (xxx.202.114.92) or it will not be accessible.
I have added guest IP to bridge0 (xxx.202.114.98)
xxx.202.114.98 is pingable from host
xxx.202.114.98 is pingable from internet
Guest lost internet
I am probably pinging just interface and packets are not being passed to VM guest server - i can not being able to connect to VM service like web server, RDP.

Removed xxx.202.114.98 from bridge0 and guest has again internet
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,679
Messages: 39,278

Igb0 must have it's main IP assigned (xxx.202.114.92) or it will not be accessible.
It's attached to the bridge0, that's how it's connected.

I have added guest IP to bridge0 (xxx.202.114.98)
Don't add the guest's IP address to the bridge, that's certainly not going to work.


Bridging works well but you cannot have an IP address on the bridged physical interface (igb0 in your case), that will cause weird issues due to the way a bridge(4) hooks in the IP stack.
 
OP
A

aqqaqq

New Member


Messages: 6

I removed main server IP from igb0 and added it to bridge0.
Ifconfig showed ip properly added. But server lost any connectivity. Reverted back.
 
OP
A

aqqaqq

New Member


Messages: 6

I am fighting again to solve this issue, i have tried in many ways but it is not working. Current situation:

Configured Bridge0 with igb0 and tap10 interfaces as members.
Guest OS configured with public, external IP assigned by hosting company. Internet is working on Guest machine.
Guest OS is not visible form internet.

vm switch:
1591735756881.png


Tried to create vm switch with both bridge0 and igb0. Guest OS had internet but no connectivity from outside.
 

Lamia

Aspiring Daemon

Reaction score: 212
Messages: 775

Tried to create vm switch with both bridge0 and igb0. Guest OS had internet but no connectivity from outside.
"Guest OS had Internet" means you can ping an IP address from it. RIght? And perhaps, you can ping a URL if you have already configured its DNS. If there is no connectivity from outside, that means traffic is not routed into the vm.
You need adjust
1) your PF rules - nat the private IP_block (e.g. 192.168.60.0/24) used in VM, set skip interface for bridge{0,1..} (including vm-public),
2) adjust your routing table - the vm-public is your gateway for the VM-bhyves. The private IP_block (e.g. 192.168.60.0/24) and vm-public IP addr (e.g. 192.168.60.1/32) should have the ether/mac address for vm-public as their gateway and vm-public as their interface (netif) in the routing table.
3)
 

Lamia

Aspiring Daemon

Reaction score: 212
Messages: 775

3) If you are NOT using a passthru "AND" using a public/static IP, I guess you will have to set static route in rc.conf and perhaps use the public/static IP as an alias for the ethernet interface - more like binding several static IP addresses to one interface. (1) & (2) might need to be slightly revised. The work best for private IP addresses with no passthru.

You will not be have this problem if you chose a passthru for one of your network interfaces. The interface would then be directly communicating on the Internet while the other interface would be for your host (FreeBSD) machine.
 
OP
A

aqqaqq

New Member


Messages: 6

I am not using passhtru. I have only 1 NIC so that would not work, as main NIC would be hidden from host OS, right ?
I have added PUBLIC IP to bridge0 interface. How do I configure static route?
I have tried
route add PUBLIC IP -interface bridge0
route add PUBLIC IP -interface tap10
route add PUBLIC IP GATEWAY IP
Guest machine has internet all the time, i can browse web. Pages like whatismyip.com report correct public ip. Guest is not pingable from internet and from host.
A
 

Lamia

Aspiring Daemon

Reaction score: 212
Messages: 775

I am not using passhtru. I have only 1 NIC so that would not work, as main NIC would be hidden from host OS, right ?
I have added PUBLIC IP to bridge0 interface. How do I configure static route?
I have tried
route add PUBLIC IP -interface bridge0
route add PUBLIC IP -interface tap10
route add PUBLIC IP GATEWAY IP
Guest machine has internet all the time, i can browse web. Pages like whatismyip.com report correct public ip. Guest is not pingable from internet and from host.
A
Follow all I suggested except the whole of (3).
 
Top