Hello,
I am trying go setup a FreeBSD 11.1 server in a way our old Linux Server behaves.
We have an interface with an external ip-address and an internal bridge where all the containers and KVMs are connected (acting like a switch). http-port from the external ip-address is passed through to one container with NAT, and this container (nginx) does the routing of locations and subdomains which reside on other containers or KVMs. Thats also the setup which works for us with OpenBSD/vmd and virtual machines.
So far, we had a great joy setting up jails with ezjail and everything works as expected. There is a bridge0 with a tap0 (with the hosts 192.168.1.1 ip-address) and a tap1 (with all jails + ip-addresses attached to).
However, we struggle with getting bhyve to run with a setup like this. I have tried bare bhyve, vm-bhyve and iohyve. With iohyve we managed to get a connection if e.g. we have a VM which is connected to tap10 (inside bridge1) and the hosts sets the ip-address (we tried to seperate jails from VMs and used the 192.168.2.0/24 network) to 192.168.2.1 on this interface. But setting up stuff like this on 10+ VMs seems too dirty. If we put tap11 to bridge1 and set an appropriate ip-address (like the setup with jails) it does not work. (ifconfig states "no carrier" which is not a problem for the jails setup)
We searched the internet, stumbled upon VIMAGE, but read that it's not stable yet producing kernel panics so this is not a choice for us. I might be on the complete wrong path and my understanding how bridges/tap-interfaces work under FreeBSD, so any enlightenment would be highly appreciated.
I am trying go setup a FreeBSD 11.1 server in a way our old Linux Server behaves.
We have an interface with an external ip-address and an internal bridge where all the containers and KVMs are connected (acting like a switch). http-port from the external ip-address is passed through to one container with NAT, and this container (nginx) does the routing of locations and subdomains which reside on other containers or KVMs. Thats also the setup which works for us with OpenBSD/vmd and virtual machines.
So far, we had a great joy setting up jails with ezjail and everything works as expected. There is a bridge0 with a tap0 (with the hosts 192.168.1.1 ip-address) and a tap1 (with all jails + ip-addresses attached to).
However, we struggle with getting bhyve to run with a setup like this. I have tried bare bhyve, vm-bhyve and iohyve. With iohyve we managed to get a connection if e.g. we have a VM which is connected to tap10 (inside bridge1) and the hosts sets the ip-address (we tried to seperate jails from VMs and used the 192.168.2.0/24 network) to 192.168.2.1 on this interface. But setting up stuff like this on 10+ VMs seems too dirty. If we put tap11 to bridge1 and set an appropriate ip-address (like the setup with jails) it does not work. (ifconfig states "no carrier" which is not a problem for the jails setup)
We searched the internet, stumbled upon VIMAGE, but read that it's not stable yet producing kernel panics so this is not a choice for us. I might be on the complete wrong path and my understanding how bridges/tap-interfaces work under FreeBSD, so any enlightenment would be highly appreciated.