What's the best place to find jail management instructions?

Toolforger

Toolforger

Hi all,

TL;DR: I have a single FreeBSD host with a few datasets, each dataset has a few jails, what's the best way to set up and manage those jails and where to I get the docs to do that? (Each dataset is one protection domain.)
I believe that ezjail is the way to go, I believe that I don't need VIMAGE but I'm not sure, I don't know if there are additional features I should check.

Details:

I've been checking handbook and various tips&tricks pages about jails, and things are getting more confusing by the day - there are just too many details, many pages seem to be outdated (but not mention the FreeBSD version, grr), and I don't want to spend days on an approach, just to find that that it was outdated and possibly misguided. And I'm seeing references to VIMAGE and don't know what the status is, or if it is useful - probably no but who knows? And I don't know if there are more such features that may or may not be applicable, and it's hard to google for things that you don't even know the name of, or if they exist at all.

So... what's the most useful set of instructions for organizing jails?

I am aware that the answer may depend on the use case, so here is mine, just to give you an idea of the level of (non-)complexity I need to handle:

1. The jails run inside a single FreeBSD host (i.e. no clustering or failover, just good old ZFS backup to a separate machine).

2. There are multiple ZFS datasets (two for now).
Each dataset is operated upon by a couple of jails (one to retrieve data from the LAN; one to offer a read-only view of the data (sambad or rsyncd); maybe a webserver to display statistics and such).
There is currently no use case that requires that a jail accesses multiple datasets.

3. There are no end users on the FreeBSD host.
(I will likely assign a separate user id for each dataset, as an extra layer of protection. I don't know how much that will buy me.)

4. If there's a trade-off between reliability and something else, I will always favor reliability.
 
I too recommend iocage.
I don't recall problems with the non devel version. But my only use case is service isolation anyway.
 
rigoletto@ said:
The easiest way to manage Jails is using management tool and there are plenty: sysutils/iocage[1], sysutils/bastille, sysutils/cbsd etc.

[1] I would recommend the -devel version, I had some issue (I don't remember now) which is fixed on -devel.
Click to expand...
sysutils/iocage is (was?) serioulsy bugged if you apply an update with iocage update. It somehow loose its devfs_ruleset and the jail doesn't restart until you fix this number at what it should be (4 in general).

I updated sysutils/iocage yesterday (by pkg on LATEST repo) and don't know whether if this bug persists or not. But there is another annoying bug which makes iocage unworkable if LANG is set to something else than xx_xx.UFT-8. I found how to fix it (just added one line of code), made a bug report but nothing was corrected in this new version.

I will try the devel version if possible. I was looking at sysutils/iocell but it seems complicated at least to keep jails created by iocage. That being said, iocell has none of the two bugs I mentioned concerning iocage.
 
Emrion said:
sysutils/iocage is (was?) serioulsy bugged if you apply an update with iocage update. It somehow loose its devfs_ruleset and the jail doesn't restart until you fix this number at what it should be (4 in general).

I updated sysutils/iocage yesterday (by pkg on LATEST repo) and don't know whether if this bug persists or not. But there is another annoying bug which makes iocage unworkable if LANG is set to something else than xx_xx.UFT-8. I found how to fix it (just added one line of code), made a bug report but nothing was corrected in this new version.

I will try the devel version if possible. I was looking at sysutils/iocell but it seems complicated at least to keep jails created by iocage. That being said, iocell has none of the two bugs I mentioned concerning iocage.
Click to expand...

Until recently (at least) you couldn't update the jails at all with sysutils/iocell because FreeBSD was made some change in the files (removed one) and sysutils/iocell can't fetch it and then fail. Thus, I would say sysutils/iocell is Abandonware now.
 
rigoletto@ said:
Until recently (at least) you couldn't update the jails at all with sysutils/iocell because FreeBSD was made some change in the files (removed one) and sysutils/iocell can't fetch it and then fail. Thus, I would say sysutils/iocell is Abandonware now.
Click to expand...
Good to know.

After some more tests, I found that the last version of iocage is running ok for updating jails. And also I can use iocage created jails with the devel version. Furthermore the iocage-devel has none of the bugs I was speaking of, not even the "LANG" one. Thanks.
 
Thanks to all.
Seeing that nobody recommends ezjail - does ezjail have issues?
I mean - the iocage reports sound like it's somewhat flaky and near the borderline to abandonware, with no idea on what side of the border it is - if that's the best recommendation, paint me worried.
 
Toolforger said:
Thanks to all.
Seeing that nobody recommends ezjail - does ezjail have issues?
I mean - the iocage reports sound like it's somewhat flaky and near the borderline to abandonware, with no idea on what side of the border it is - if that's the best recommendation, paint me worried.
Click to expand...

I think sysutils/ezjail is abandonware now. Welcome to the OSS world!

sysutils/iocage-devel is being developed, had an update recently but just not a new release yet; however I would give a shot to Bastille if I had not several jails yet.
 
I went the other way from iocage to just editing my own jail.conf and pulling down base. Iocage broke on my more than enough times to give up on. Now I just keep snapshots of a base jail, clone it then edit /etc/jail.conf. Hasn't broken yet and no more difficult than using iocage.
 
rigoletto@ said:
The easiest way to manage Jails is using management tool and there are plenty: sysutils/iocage, sysutils/bastille, sysutils/cbsd etc.
Click to expand...

Bastille looks nice, but it seems to insist that you manually assign an IP address to each jail.
I'd really prefer if it allowed me to use DHCP/SLAAC for that. I could manage the addresses manually, but anything that you need to manage once every two years means that you either have to re-read all the details, or overlook some crucial detail and get held up.
 
rigoletto@ said:
The easiest way to manage Jails is using management tool and there are plenty: sysutils/iocage, sysutils/bastille, sysutils/cbsd etc.
Click to expand...

cbsd seems to do far too many things, including web management consoles.

This is certainly useful, but not for my use case: My FreeBSD box is supposed to be the trusted management anchor for the LAN, so it doesn't have sshd or anything else that allows anybody to manage it via the network. Just a console and a keyboard.
cbsd also does a lot of stuff I don't need (bhyve support, network consoles, and such), and since I want to keep the attack surface small, cbsd isn't usable.

With ezjail abandoned, iocage not really well-updated, bastille almost-good and cbsd not fitting - are there other candidates in the "etc." category worth consideration?
 
michael_hackson said:
I always go by the guide from scottro.

https://www.scottro.net/nullfsjail.html

That is the best hands on guide I have found, it has made jail management so much easier.
Click to expand...

This tutorial is outdated. Also I see mounting fs from host to jail as a kind of dependency. I like THICK jails. Disk space is cheap nowdays. They are more portable between hosts.

Mounting fs from host can be partly avoided by using for example nginx cache proxy in another jail as source for pkg, ports, jail update/upgrade process.

Tutorial which I linked in my previous post is no ideal, but is a good and actual base for vanilla jail managemet.
 
Toolforger said:
2. There are multiple ZFS datasets (two for now).
Each dataset is operated upon by a couple of jails (one to retrieve data from the LAN; one to offer a read-only view of the data (sambad or rsyncd); maybe a webserver to display statistics and such).
There is currently no use case that requires that a jail accesses multiple datasets.
Click to expand...

In my opinion each jail should have separate zfs dataset.

Jail can have more mounted datasets. For example:
1) Jail with mysql should have extra one or more datasets for db files (according to http://open-zfs.org/wiki/Performance_tuning#MySQL)

2) jail for ZFS replication backup can have a lot datasets. Such jail have to be prepared to manage of the dataset from within a jail (see Jails section of https://www.freebsd.org/cgi/man.cgi?zfs(8) ).
 
zomar said:
In my opinion each jail should have separate zfs dataset.

Jail can have more mounted datasets. For example:
1) Jail with mysql should have extra one or more datasets for db files (according to http://open-zfs.org/wiki/Performance_tuning#MySQL)

2) jail for ZFS replication backup can have a lot datasets. Such jail have to be prepared to manage of the dataset from within a jail (see Jails section of https://www.freebsd.org/cgi/man.cgi?zfs(8) ).
Click to expand...

Let me rephrase the core point I was trying to make:

I'd agree in general, but I want the one writer process in a separate jail than the reader processes, for security reasons.

Sure, that fits for the general case.

However, I want the system to be designed so that it is possible to identify all scripts that may write to a dataset, while retaining the ability to add reader scripts without having to worry about mistakes.
Having the dataset mounted r/w for one and only one jail should nicely achieve that. Plus, I could add reader jails at will, experiment with different tools or different tool versions, without any possibility that I may impact the writer process. (I could probably do similar by mounting the same dataset into a separate mountpoint for each process. This wouldn't allow me to play with the software I install at the "system" level inside each jail, so I believe jails would still be the best option.)
 
Last edited:
Toolforger said:
Let me rephrase the core point I was trying to make:

I'd agree in general, but I want the one writer process in a separate jail than the reader processes, for security reasons.
Click to expand...

I do not understand your goals.

General tips:
1) Do not try to over complicating zfs/jail configuration. Do not push technology to its limits. Think about that there will be a day when you will need to recreate envirioment. Especialy if you are going to use third party jail managemt tools.

2) Mayby there are other technologies, services or packages to achieve goal. If yes think about zfs/jail as a virtualization platform and all adventages of it.

Good luck!
 
You must log in or register to reply here.
Back
Top