jails Do you like it thick or do you prefer it thin?

[jbo@]

For your average small server hosting a couple of websites, databases, media servers and similar (i.e. neither performance critical nor heavily cpu/memory/storage limited), do you generally use thick or thin jails?
Please explain your reasoning.

Poll above.

Personally, I tend to default to thick jails as the performance/storage overhead is mostly negligible for what I'm dealing with.
I'm dealing with < 100 jails total. Sometimes I do wonder whether I'd have a bit more spare-spare-time if I were to selectively use thin jails for groups of "similar" jails (i.e. one for all webservers, one for all nextcloud instances, ...).
Thick jails just also seem to be more resilient towards "oopsie" mistakes.

 

[robroy]

I'm using thick jails 'cause they smell more like normal FreeBSD hosts; I can use the same familiar patching process, I can ssh in if I want, etc.

 

[jbo@]

You can SSH into a thin jail too, no?

 

[Jose]

I don't use many jails yet (I know, I know.) The attraction of thin to me is how they put the Docker "layers" nonsense to shame.

 

[Emrion]

Once I knew that existed, I always prefered thin jail. I was fascinated by ezjail.
I find that the update and the upgrade is faster and somewhat simpler (and creation too).
Clearly, I don't want anymore to update n jails n times, once per jail.

 

[kpedersen]

I tend to go with thick jails. My reasoning:

• Simpler to set up, just extract base tarball and done.
• An attacker can always add more files to a thin jail. Dubious protection benefits by stripping them.
• I don't like "riced" systems containing inconsistent user-land. They always have that broken Linux feeling.
• Updates are generally smoother due to expected files existing.

 

[gotnull]

I can't vote because I really use both:
_ thin jail for testing purpose, convenient because they are quick to deploy, take no space, generally I create a jail do my tests and then destroy the jail when I am done.
_ thick jail for something I want to keep around for a long period of time.

I am not an expert so I can be wrong but to me thick jails are more robust and trustful than thin jails when it comes to upgrade/update and security.
But in the end I use thick as well as thin jails because they both have their advantages and inconveniences depending on the situation.

 

[degauss4]

This is my great dillema as well I am learning and just trying things out. I dont know if I like it thick or thin

• Thick: Better for security and isolation and keeping something long term. Less complexity but more time in administrating.
• Thin: Faster to deploy, simpler to manage. More complexity but less time.

The drawback comes in when it's time to update. As Emrion pointed out, doing a manual update for each thick jail will be a drawback, but it may also be a positive since you need to pay attention to the packages installed on the thick jail, and it might be easier to rollback as well. I will go through when I see version 14.2 around

I can't vote because I really use both:
_ thin jail for testing purpose, convenient because they are quick to deploy, take no space, generally I create a jail do my tests and then destroy the jail when I am done.
_ thick jail for something I want to keep around for a long period of time.

I am not an expert so I can be wrong but to me thick jails are more robust and trustful than thin jails when it comes to upgrade/update and security.
But in the end I use thick as well as thin jails because they both have their advantages and inconveniences depending on the situation.

Reading this though, I think I might do the same. Thin jails for testing around, automating with ansible, destroying a messed up jail and starting again with a clean template, basically a development environment. After I am confident, release to production on a thick jail and add it to the boot up jail list.

 

[BaronBS]

I don't like "riced" systems containing inconsistent user-land. They always have that broken Linux feeling.

Me neither, but what this have to do with jails? Are you saying that a riced FBSD system is unstable like linux userland?

 

[kpedersen]

Me neither, but what this have to do with jails? Are you saying that a riced FBSD system is unstable like linux userland?

Effectively, yes. If you inherit a jail from a co-worker, where they have plucked random things out of it. You basically can't trust it.

I would rather a Jail to be a little larger in terms of disk space and know that it is consistent.

No-one wants to debug a bunch of PAM authorization issues, because your predecessor decided that they didn't need half of the PAM modules when they initially set up the jail. Even if there is no co-worker, even "yourself from ~3 years ago" has the same effect. You might not be able to remember what crap was deleted (I certainly can't).

Which is why the FreeBSD base is so damn important. And why I predict PkgBase is going to be very damaging as people inevitably start to misuse it.

 

[Jose]

I guess I'm a little confused. If I create a thin jail by cloning a ZFS snapshot of a full install, that thin jail has all of base plus whatever else you did before the snapshot was taken, no?

Which is why the FreeBSD base is so damn important. And why I predict PkgBase is going to be very damaging as people inevitably start to misuse it.

Oh yeah. I used Pkgbase when it was called "Gentoo Linux". It was real. It was fun. It wasn't real fun.

 

[kpedersen]

I guess I'm a little confused. If I create a thin jail by cloning a ZFS snapshot of a full install, that thin jail has all of base plus whatever else you did before the snapshot was taken, no?

Hmm, based on the definition here, my whole premise is different. A thin Jail (originally coined from "thin chroot") is one that is purposefully reduced to the minimal software required to run a service. Typically libc, /dev/null, /bin/sh and any required libs. Apparently this is now called a "minimal jail".

If it is really just a ZFS+nullmount, then these days, I thought that was just how people did jails

In that case, I apologies to the thread for my noise!

Oh yeah. I used Pkgbase when it was called "Gentoo Linux". It was real. It was fun. It wasn't real fun.

Heh. Revisiting screenshots (or even ancient hard disks containing a Gentoo install) always has me thinking to myself, what kind of random sodding Frankenstein('s monster) cruft is this?! I'm sure it made sense to the duct tape at the time.

 

[degauss4]

Thick jails just also seem to be more resilient towards "oopsie" mistakes.

What kind of oopsie mistakes are you having? I thought that thinjails were cloned from a template and it's own dataset. Any changes on the thinjail should only reflect on the dataself itself, right? That's how I understood it worked

 

[jbo@]

What kind of oopsie mistakes are you having? I thought that thinjails were cloned from a template and it's own dataset. Any changes on the thinjail should only reflect on the dataself itself, right? That's how I understood it worked

Hmm... if that is indeed how thin jails work under the hood then that is a fair point.
It was my assumption (!!) that the same dataset is shared across the jails meaning if you screw up the base of one jail all of them will be affected. If instead thin jails are creating a clone of the "source dataset" that would not be a problem.
I'll have to read up & investigate on how this actually works. I'm only recently moving from jail management tools to doing it manually.

 

[Jose]

Hmm, based on the definition here, my whole premise is different. A thin Jail (originally coined from "thin chroot") is one that is purposefully reduced to the minimal software required to run a service. Typically libc, /dev/null, /bin/sh and any required libs. Apparently this is now called a "minimal jail".

Hmm, that's an interesting script. Much the same debate happens over in Docker-land, BTW. The argument for "minimal" containers is a smaller attack surface, and hopefully therefore a less violent update treadmill.

 

[DaLynX]

I can't vote because I really use both:
_ thin jail for testing purpose, convenient because they are quick to deploy, take no space, generally I create a jail do my tests and then destroy the jail when I am done.
_ thick jail for something I want to keep around for a long period of time.

I am not an expert so I can be wrong but to me thick jails are more robust and trustful than thin jails when it comes to upgrade/update and security.
But in the end I use thick as well as thin jails because they both have their advantages and inconveniences depending on the situation.

How does a thick jail bring more in terms of security? From what I understand the only risk with thin jails would be cross-jail compromise of the base. Which cannot happen since it's mounted read-only.

 

[jbo@]

How does a thick jail bring more in terms of security?

I'd like to learn more about this too.
From the handbook:

Advantages of Thick Jails: [...] Security: Since the jail contains its own base system, vulnerabilities or issues affecting the jail environment won’t directly impact the host or other jails.

Disadvantages of Thin Jails: [...] Security Concerns: The reduced isolation in thin jails could pose security risks, as a compromise in one jail might have a greater potential to affect other jails or the host system.

 

[PMc]

The differentiation is new to me, but mine do certainly qualify as "thick" jails. For "thin" jails I would need to work out appropriate maintenance (which is certainly possible, but is work), while when considering a jail as a nearly full-featured system, most of the tooling, network layout, deploy scheme etc. is all the same.

So as long as there aren't real numbers of rather similar entities, I don't see a benefit in thinning.

But let's have a look into the handbook where this idea comes from: that idea goes back to when a chroot would contain only what is needed for the task (and then always run into something still missing).
I don't think thin jails are simpler to maintain, because such "unified maintenance" needs to be properly developed first. And you will probably need some automation for the genuine systems' maintenance anyway, and thick jails can then go along mostly identical with that automation. So better to invest into that.

 

[DaLynX]

I think that's a good point and is the main reason I'm running thick jails too.

Thin jails look like a more elegant solution, they would save me disk space (which is small in my personal setups) and time when upgrading, because updating 4 systems would be much faster than 20-something.

But it would require much more investment to deepen my understanding and maybe develop custom scripts. And of course having to relearn in two years what present day me had to learn, and decided to do.

I'm also hoping pkgbase will make upgrades easier and especially faster.