jails with nullfs on host base install

A

An-tonio

Hi all,

this is my first post here. First thanks for this great system. Im a long term linux guy (principally debian), but after linux move to systemd I started to look on another direction...

First an introduction before my question. I'm building an automated provisioning system with ansible for web servers with php. I would like to have different php versions running at the same time and the way pkgs are build (I prefer to stay with binary builds for now) make impossible to mix different php versions, so I started to investigate "jails".

With jails I could install all php version in a clean manner and use php-fpm. But the problem I get is that on all doc/howto/info I can search, all jails are created from a base jail that is independent from host base install (like ezjail) or completely independent jails with his own base install. I get here two problems:

1. For each new server I have to update and take care of a minimum of two base systems (host + base jail). This can complicate things with a lot of servers (yes, automation can help, but...)

2. Disk space. With dedicated servers or relatively big cloud/vps servers you get enough disk but with small cloud offers, 1G~ lost for a jail base template can be a lot. I know about zfs deduplication, compression and clones. But if I can avoid spending 1G, better.

So finally my question :)

I know some jails systems (iocell/iocage maybe others) use nullfs filesystems in read-only over the base/template jail and let var etc and usr/local on it's own jail. I did some test creating jails manually with nullfs over the host base system and it work ok. So in theory this solve my "problems": only one base system to update and not waste disk space (or minimal). But I'm just starting to play with freebsd so maybe I miss something because I can't found info about creating jails on this manner... why ?, security problems ? any other issue ?

Any comment is welcome,
Thanks!
 
Other programs (and I haven't been keeping up with this lately, so I can't offer any suggestions about which is best) are ezjail and qjail. I believe one was based on the other. They both use nullfs.
If I'm not mistaken, iocell is based on the old iocage, which was also written in shell but is now using python. I've found those two the easiest to use. I haven't used them with ansible. I'm too tired to google this, but if I'm wrong, someone will hopefully correct this, but I I think that iocage and iocell require zfs, I don't believe ezjail and qjail do. I _do_ know that the original iocage required zfs.
 
Thanks for your response. I already tested near all jail solutions. Now I was testing iocell (forked from shell version of iocage) that is nice, and it use nullfs. But my objective is to create a jail with nullfs to the host base system, not to another jail/template. I already did it following this fantastic howto: https://clinta.github.io/freebsd-jails-the-hard-way/ but with some modifications, in fstab I did:
Code: 
/bin /usr/local/jails/test1/bin nullfs ro 0 0
/boot /usr/local/jails/test1/boot nullfs ro 0 0
/lib /usr/local/jails/test1/lib nullfs ro 0 0
/libexec /usr/local/jails/test1/libexec nullfs ro 0 0
/rescue /usr/local/jails/test1/rescue nullfs ro 0 0
/sbin /usr/local/jails/test1/sbin nullfs ro 0 0
/usr/bin /usr/local/jails/test1/usr/bin nullfs ro 0 0
/usr/include /usr/local/jails/test1/usr/include nullfs ro 0 0
/usr/lib /usr/local/jails/test1/usr/lib nullfs ro 0 0
/usr/libexec /usr/local/jails/test1/usr/libexec nullfs ro 0 0
/usr/ports /usr/local/jails/test1/usr/ports nullfs ro 0 0
/usr/sbin /usr/local/jails/test1/usr/sbin nullfs ro 0 0
/usr/share /usr/local/jails/test1/usr/share nullfs ro 0 0
/usr/src /usr/local/jails/test1/usr/src nullfs ro 0 0
/usr/libdata /usr/local/jails/test1/usr/libdata nullfs ro 0 0
/usr/lib32 /usr/local/jails/test1/usr/lib32 nullfs ro 0 0
And in /etc/jail.conf
Code: 
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;

test1 {
    host.hostname = "test1";
    path = "/usr/local/jails/test1";
    interface = "vtnet0";
    ip4.addr = 10.0.0.15;
}
And it seem to work ok. But my question is: why all jails normally use a template jail for nullfs and not the host ?

thanks
 
An-tonio said:
But my question is: why all jails normally use a template jail for nullfs and not the host ?
Click to expand...
Because the jail can run a different (lower) version. You can, for example, run a 10.3-RELEASE jail on a 11.0-RELEASE host (the other way around is not possible). Another reason is typically isolation, an issue on the host won't cause the same issue on the jail. A jail usually also has different settings in /etc/ and /usr/local/etc/, and has different packages installed.

As a suggestion, move the test1 mounts to /etc/fstab.test1 and add mount.fstab to your jail.conf. That way the filesystems will get mounted when the jail starts and unmounted when it stops.
 
Ok, thanks a lot. So my proposal is right, I will always use the same release on host and jails, and the idea with jails is only to separate services that can't be installed with different versions on same host (aka differents php-fpm versions) I will not mount jail/etc and jail/usr/local/etc from host (nor var), each jail will have his own and minimal with only the essential for the specific service.

Thank for the mount.fstab tip, it help on automation process.

I'm loving Freebsd and his true unix approach.
 
You must log in or register to reply here.
Back
Top