Base Jail

[Beeblebrox]

"Base Jail" is a term used by ezjail. The name may be different for other systems, but the basic concept is about mounting the base or read-only jail onto the thin/flavor jail.

My question: Most thin/flavor jails require same/similar basic ports (like lang/perl, devel/cmake, etc). It would make sense to place the "common ports" into the "base jail" and update base instead of each separate jail for those common ports.

The problem is, base/fat jail's /usr is only partially mounted while many sub/folders are sym-links, and thin-jail keeps its own /usr/local.

My "idea": define custom install path for ports in base jail so that those ports are not installed in <jaildir>/usr/local, but somewhere else like <base-jaildir>/usr/alljails.

On the other hand, It's very likely I have no idea what I'm talking about...

 

[UNIXgod]

You could create a "build jail". The idea is a jail that simply builds the binaries that the "thin" jails can pull from... Come to think about it that may be what exjail is doing by fancy symlinking to /usr/ports/distfiles inside the so called "Base Jail".

Also consider that one jails port may be compiled with different options than another.

Another thing to consider is that if you use zfs with compression and dedup you pretty much have everything covered.

Just my 0.02¢

 

[_martin]

As @UNIXgod mentioned, the idea behind basejail is to mount it read-only and use symlinks to build a 'classic' directory structure.

If you are using ZFS, you can create (very nice and easy with send/receive) new fileset, e.g. "yourbase", install desired ports from host to "yourbase" location (using PREFIX) and modify jail symlinks to point to "yourbase" and include /usr/local too.

But by doing that ezjail-admin becomes pretty much unusable for you.

You can maybe ask author for feature request ?

 

[fbsd1]

sysutils/qjail has instructions in its man page to do what you want.

 

[Beeblebrox]

@fbsd1: from qjail(8)()

In the situation where you want "all" the jails that you EVER create to have the same ports included as a standard, follow these steps. After running the "qjail install" sub-command and before you start creating all your production jails. Create a single directory tree type "standard" jail and populate it with your selection of standard ports. When your satisfied with the "standard" jail, delete the /usr/jails/newjail directory and copy the "standard" jail to create a new /usr/jails/newjail directory. cp -rf /usr/jails/standard /usr/jails/newjail. From that point on, all new jails created using the newjail template will contain your standard ports.

Yes, but as I understand, since "newjail" is a template it will merely copy all the ports in it, to all other jails. What happens during update of ports? simply updating newjail will not be enough and all jails which exist must be updated separately and individually. Am I wrong? If, on the other hand, newly created jails don't create new copies of ports from newjail but link to them (like basejail links into newjail) then an update of newjail alone will have updated the "base ports pool", and that is what I am looking for. As an example to the structure I am trying to describe, the depends list for www/lighttpd includes (among others) these:

archivers/unzip converters/libiconv devel/autoconf devel/automake devel/cmake
devel/gmake devel/libtool devel/pkg-config lang/perl5.12 textproc/libxml2

But these ports are in the depends list of almost every mid-to-top level port. If these ports were installed and upgraded through a single jail and distributed to child jails, then there would be about 50% less ports to update in the jail hosting www/lighttpd.

@unixgod: This is a great example of zfs' dedup feature, but my primary aim is to have "least ammount of effort during update". In the explanation at previous paragraph, dedup works great for duplicating the jail's ports but does not simplify the "updating ceremony". Correct me if I'm wrong, but an update of a port in one jail will just make dedup go (no.of.jails - 1=new.dedup.number)+(new.separate.dedup.entry) until the same port in a second jail gets updated.

Re buildjail: I'm in the process of re-shuffling all my jails and tinderbox is one of the jails (or chroot env) I plan. I have been running ports-mgmt/portmaster with the -g flag for some time so I already have a good collection of local-built packages to start with.

one jails port may be compiled with different options than another

Not an issue in my case, but thanks for the reminder.

@matoatlantis:

install desired ports from host to "yourbase" location (using PREFIX) and modify jail symlinks to point to "yourbase"

I doubt that a PREFIX modification in one jail will retain its consistency accross the nullfs mounted child jail - that was one of the points of my question. Also, one must consider the fact that the child jail's /var/db/pkg will have no record of ports provided through "yourbase", which will eventually cause problems.

 

[Quip]

try unionfs

You can consider mount_unionfs(8). This way, you can stack one directory on top of the other.. It means, you can have /usr/local in base jail with some ports and mount another directory over it in each jail. Then you will have mix of both contents.

I did not test it, so I am not sure that it will work for you. unionfs on FreeBSD is still somewhat experimental.

 

[_martin]

@matoatlantis:

I doubt that a PREFIX modification in one jail will retain its consistency accross the nullfs mounted child jail - that was one of the points of my question. Also, one must consider the fact that the child jail's /var/db/pkg will have no record of ports provided through "yourbase", which will eventually cause problems.

I think we don't understand each other. Even basejail is created the very same way - binaries are extracted (or installed in case of using sources) to some specific location (DESTDIR). Using host, you can set the PREFIX to install desired ports to basedir location: $BASEJAILPATH/usr/local.

This filesystem is mounted ro using nullfs and has nothing to do with symlinks. You just need to add symlink to basejail/usr/local in jail itself.

But you must pay attention during upgrade, etc. So it's not an ideal solution and needs some extra work outside of the ezjail-admin script.

 

[Beeblebrox]

@matoatlantis: We are talking about the same thing but from different angles. From my first post:

... the basic concept is about mounting the base or read-only jail onto the thin/flavor jail

What I mean here, is that the base jail is mount_nullfs'ed (ro as I mention) onto/under the newjail.

This filesystem is mounted ro using nullfs and has nothing to do with symlinks.

As I had said in my first post except, symlinks have all to do with it in the basejail->newjail structure, as all folders residing in basejail are symilnks under newjail/base, which is exactly what you are saying.

You just need to add symlink to basejail/usr/local in jail itself.

If you do this, you cannot add ANY NEW ports into the newjail (or ex:sqljail) and will have to add all ports to basejail.

I doubt that a PREFIX modification in one jail will retain its consistency accross the nullfs mounted child jail

Here's what I mean by above: You have 3 jail folders - basejail, newjail (the template) and sqljail. Per my description above, we added the "base ports" to newjail but specified PREFIX as newjail/usr/baseports. When sqljail starts, we also nullfs_mount newjail/usr/baseports into sqljail/usr/baseports. Is the sqljail's file structure now aware of usr/baseports even if we were to add the PATH into ENV? This was the point in my last post (plus you must also figure out how to merge /var/db/pkg from newjail and sqljail).

 

[Beeblebrox]

@Quip: A most useul and very interesting concept. Perfect for what I was describing. Thanks!

 

[_martin]

If you do this, you cannot add ANY NEW ports into the newjail (or ex:sqljail) and will have to add all ports to basejail.

I stand corrected. That's a good point - you would be left without option of installing custom packages/ports.

 

[Beeblebrox]

@Quip: I've played around and made some headway with a jail setup employing unionfs combined with zfs dedup (details to follow). Sample fstab entry as apetizer:

/data/jails/j64base/usr/local /data/jails/pxe/usr/local unionfs ro,below,noatime 0 0

However, I ran into a strange problem in this structure: Most services start fine, with the exception of dhcpd. The jail hangs and freezes when I start dhcpd and stopping the jail does not solve it - I have to reboot because of hanged processes. This does not happen when I do not employ unionfs and in that case dhcpd starts fine. Any ideas why?

EDIT: Although a great idea, unionfs does not seem to be fully working and is buggy. IBM has a similar concept with [ports]net/openafs[/port] but it really is a whole different concept and too "heavy" when using ufs or zfs. SOLVED - only because the solution set (for the present moment) is a null-set. If unionfs works well at some date, the solution-set will no longer be null!