jails Am I overlooking anything with my basic jail layout here?

[puppydog]

So I've never done much with jails but recently I've been wanting to dive into them more and use them as part of my standard workflow on my freebsd server. I believe what I've done is basically correct, but I thought I would make a post here to see if someone with more jail experience than me can glance over it and say if I'm overlooking something or doing something in a totally wrong and insecure way.

The server in question is a dedicated server with several publicly routable IP addresses. The network and jails portion of rc.conf on the host is as follows (the IP addresses are placeholders here but are correct on the actual configurations):

ifconfig_igb0="x.x.x.10 netmask 255.255.255.192"
defaultrouter="x.x.x.1"
ifconfig_igb0_alias0="x.x.x.11 netmask 255.255.255.255"
ifconfig_igb0_alias1="x.x.x.12 netmask 255.255.255.255"
jail_enable="YES"
jail_list="jail1 jail2"
jail_parallel_start="YES"

jail.conf is as follows:

mount.devfs;
exec.clean;
exec.start="sh /etc/rc";
exec.stop="sh /etc/rc.shutdown";

jail1 {
host.hostname="jail1";
ip4.addr="x.x.x.11";
path="/jails/jail1";
}

jail2 {
host.hostname="jail2";
ip4.addr="x.x.x.12";
path="/jails/jail2";
}

* the only network-facing service on the host is sshd, which I've configured with (among other things) ListenAddress x.x.x.10
* Each jail has sshd configured with ListenAddress set for its particular IP, and resolv.conf set with nameservers.
* There is a jails ZFS volume containing a volume for each jail. e.g., pool/jails, pool/jails/jail1, pool/jails/jail2, each set with an appropriate quota property.
* When all work is complete, each jail will contain a different network-facing service on its own IP. A mix of webservers and other services, on whatever port it needs to use.

And that's pretty much it. With this layout, I am able to ssh into the host or the jails as desired from a remote host, and once I begin building out the contents of the jail, I will just connect via jexec and get to work from within the jail.

Am I missing anything with all this? Did I overlook some critical security component here? Or is everything basically perfect?

 

[im]

Jails will not have their loopback interface.
So everything that want to listen on 127.0.0.1 inside a jail will listen on jail's IP. For your case it will be a public IP.
Start your jails and review the list of open ports on jail's IPs.
I prefer to disable by firewall@host any external connections from untrusted IPs to some jailed services like MySQL.
You may want to disable some services like syslog (syslogd_flags="-ss") and sendmail (sendmail_enable="NONE") inside the jails.
You may want to disable some default periodic tasks inside the jails.

There were few rare issues with jexec for running some tasks inside a jail, because jexec does not assign tty to your login session as expected by some software. In case of any issues with running something by jexec use ssh instead.

 

[puppydog]

Jails will not have their loopback interface.
So everything that want to listen on 127.0.0.1 inside a jail will listen on jail's IP. For your case it will be a public IP.
Start your jails and review the list of open ports on jail's IPs.

This point in particular is one I hadn't considered. I'm not 100% sure on what I need to do here. Should I add a new loopback address to lo0 on the host for each jail just like I did with the public IP, and simply assign that to the jail's lo0? Or do I need to actually do cloned interfaces for the loopback? A lot of what I'm seeing with Google does it this way, but that seems to be mostly in conjunction with using pf on the host to route traffic, so I'm not sure that's necessary in my configuration.

Edit:
After doing some testing, I don't seem to actually need to change anything in /etc/rc.conf at all, for the loopback. I modified the jail.conf as follows:

ip4.addr="lo0|127.0.1.1/32";
ip4.addr+="igb0|x.x.x.11";

changing it to 127.0.1.2/32 and so forth for the other jails, and after starting the jail service the IPs I specified suddenly appear on the host's lo0 interface, and the jail itself has it assigned to its own loopback. So that seems to be all that was needed for this at least. Stopping the jail service removes the extra IPs from the host's lo0, so I guess the jail is just handling all this on its own.

 

[cmoerz]

If you want a fully separate loopback, you could also go for a vnet based jail (see https://wiki.freebsd.org/Jails).
You'll need an epair() interface-pair to connect the jail with the underlying host however.

Then again, the way you resolved that is actually quite elegant, I think. Unless you want to run separate pf, fully separate network stack with possibly different routing table and so on, vnet might not be worth the effort.

 

[im]

I use jails for web-hosting more than 10years and I had no case when I required a separate loopback interface inside a jail.
Try to configure your jails with single interface, but keep in mind that some software can listen on the external IP because of no loopback.
In my opinion, you should not configure loopback interface for jails, because of most common use cases don't require it.
It may be a good idea to setup a firewall for jail's IP with a rule like 'deny by default' and allow only required external traffic.
For my tasks I use lo1 interface with some private network addresses for jails, and use http/tcp proxy or 'nat portforwarding' to have any external communication with the jails. So I don't care about any software on jail's loopback.
But it is my own experience, someone may have any other thoughts.