Networking Structure for VM Host


I'm currently running a server based on FreeBSD 10.3. I'm using jails to separate all the running services. Since I only have one public IPv4 address, I use PF and a nginx-proxy to redirect to the specific jails inside the NAT.

My plan is to use 11.0 and it's enhanced bhyve features to build my own virtual machine host. So far I've successfully installed a FreeBSD guest using the vm-bhyve tool. It runs inside of a NAT using dnsmasq and the vm-bhyve switch features.

My problem now is:

How do I set up the network if I want jail1 on guest1 to be able to connect to connect to guest2 (linux). My hosting provider provides me with one public IPv4 address and a IPv6 subnet. My first thought was to run all guest-vms inside a NAT and configure bridge interfaces for all jails on guest1.

2. Question:

Is it possible to install an OpenBSD guest to use as a firewall for the NAT or even better, the NAT and the host?

Thanks in advance,


I never actually tried it, but it should be as easy as using the bridge for the (additional) ips of your jails.
So if you connect your bhyve-VMs to bridge0, also use bridge0 for your jails.

For internal communication between VMs and jails I would use a seperate bridge (or "private switch" in vm-bhyve terms). This bridge could also be connected to an vpn-tunnel (tun or tap interface) for administration, so ssh and other management-related services don't need to be exposed on your external ip.

To connect jails inside a VM directly to another VM i'd try this:
- Host: give each VM a new virtual NIC, connect both to the same (new) bridge
- FreeBSD-VM: create a bridge with the new NIC as a member, then connect the jails to this bridge
- linux-VM: the new interface should now be in the same layer2-network as the bridge to which you connected the jails. Assign an IP within the same subnet and you should be fine.

The host could tap into this network by assigning an ip on the bridge. This way the bridge can also be used for routing to other (virtual) subnets or the outside world.

As long as no vlans are involved (or any interfaces sharing the same mac-address), bridges can connect pretty much any devices together and at the same time act as an entry-point by assigning one or multiple ip addresses to them. I'm using bridges to connect a multi-site telephone system. The bridges at each gateway connect their end of the vpn-tunnel to the physical interface. At one site the subnet is accessible for the management-software through the bridge and SIP-connection to the outside world is provided by siproxd listening on the bridge as well.

Make sure to check the manpage for if_bridge and adjust the sysctl variables if needed (e.g. for filtering, arp)