We finished moving all our servers over to FreeBSD a couple of weeks ago (we really love FreeBSD) and would now like to leverage Jails to reduce our administration overhead if it fits in with what we are trying to achieve. We currently use Xen for virtualization but the overhead of managing an operating system for each part of our application consumes a heap of time not to mention the waste of server resources. 7 of our servers run nginx and one runs a database.
Our application requires eight different servers, which are duplicated and load balanced to deal with the load. What I would like to achieve is to have these 8 servers running in a jail on a single piece of hardware. My test machine is an Dell R710 with 96GB of RAM, 2 x 6 core processors and 4 network interfaces.
I have come up with the following design:
For my first tests I used Ezjail but I don't really mind what I use to achieve what we would like to. I used Ezjail because having flavours makes things much easier to replicate (and we will be replicating each server multiple times). I also like it because it mounts ports via nullfs and I would like to utilise nullfs mounting for each servers nginx files.
What I would like to achieve in essence is 8 separate servers just doing their thing, similar to having 8 virtualised instances of FreeBSD on Xen but without the huge admin overhead.
Hopefully the above gives enough background on what I would like to achieve. Now onto my questions.
When I create a jail with Ezjail and go into its console, running ifconfig shows all interfaces, even if they have not been allocated to the jail. Is this expected behavior or have I misconfigured something?
I have configured the base system to have a different WAN IP (in the same subnet) on each of the interfaces. I have then configured 3 jails which each using a different IP/interface. At this point I like to understand best practice for routing/firewall configuration. Is best practice to allow all traffic on each Jail interface on the base system (being careful to change listen addresses on the base system) and then configure IPFW on each of the Jails or something else?
Should I be using Ezjail or creating each jail the traditional way? I get the feeling that Ezjail may be restrictive in the future but I like many of its features.
I noticed many of the tutorials on the web use PF instead of IPFW. Are there features available in PF that make Jails easier to administer?
If I use IPs on different subnets I assume I will need to use setfib on the base system. Are there any gotcha's in regards to doing this?
Thanks in advance for any help!
Our application requires eight different servers, which are duplicated and load balanced to deal with the load. What I would like to achieve is to have these 8 servers running in a jail on a single piece of hardware. My test machine is an Dell R710 with 96GB of RAM, 2 x 6 core processors and 4 network interfaces.
I have come up with the following design:
- 1 x network interface for management of the base system (probably LAN).
- 3 x network interfaces for WAN connectivity. Different jails will be allocated different interfaces based on their expected traffic profile.
For my first tests I used Ezjail but I don't really mind what I use to achieve what we would like to. I used Ezjail because having flavours makes things much easier to replicate (and we will be replicating each server multiple times). I also like it because it mounts ports via nullfs and I would like to utilise nullfs mounting for each servers nginx files.
What I would like to achieve in essence is 8 separate servers just doing their thing, similar to having 8 virtualised instances of FreeBSD on Xen but without the huge admin overhead.
Hopefully the above gives enough background on what I would like to achieve. Now onto my questions.
When I create a jail with Ezjail and go into its console, running ifconfig shows all interfaces, even if they have not been allocated to the jail. Is this expected behavior or have I misconfigured something?
I have configured the base system to have a different WAN IP (in the same subnet) on each of the interfaces. I have then configured 3 jails which each using a different IP/interface. At this point I like to understand best practice for routing/firewall configuration. Is best practice to allow all traffic on each Jail interface on the base system (being careful to change listen addresses on the base system) and then configure IPFW on each of the Jails or something else?
Should I be using Ezjail or creating each jail the traditional way? I get the feeling that Ezjail may be restrictive in the future but I like many of its features.
I noticed many of the tutorials on the web use PF instead of IPFW. Are there features available in PF that make Jails easier to administer?
If I use IPs on different subnets I assume I will need to use setfib on the base system. Are there any gotcha's in regards to doing this?
Thanks in advance for any help!
