Restoring jails

bsus · Feb 20, 2012

Hi,

I been a while abstinent from jails and I found many partly contradictory statements about managing jails. So I think it is a good idea to start the onethousandandone thread about jails

At the moment I am running about five jails, configured manually following the handbook. Unfortunately I left them uncultivated for a long time since now and they are fully outdated. Now I need a fast solution to get back fit jails.

I see two possible solutions:
1.) Remove all jails and create some new using ezjail
2.) Update the given jails with make world

What solution would you prefer? How does the solution run in detail?

Regards,
bsus

SirDice · Feb 20, 2012

Jails and ZFS are a really nice way to deal with it, if you can go that route.

Just build a jail in a ZFS filesystem, snapshot it and you can clone it as many times as you want.

Code:

root@molly:~#zfs list -r -t snapshot,filesystem fbsd0/jails
NAME                              USED  AVAIL  REFER  MOUNTPOINT
fbsd0/jails                      3.14G  1.32T  46.4K  /jails
fbsd0/jails/clean_jail            341M  1.32T   341M  /jails/clean_jail
fbsd0/jails/clean_jail@20120214   293K      -   341M  -
fbsd0/jails/j-build-amd64        2.81G  1.32T  3.14G  /jails/j-build-amd64

The jail is simply created with:
# zfs clone fbsd0/jails@20120214 fbsd0/jails/anotherone

Add a few parameters in rc.conf and you're good to go :e

If you separate the data from the jails, updating them should be relatively easy too. With some clever renaming to swap the 'old' snapshot with a 'new' one for each jail.

razrx · Feb 20, 2012

SirDice said:
Jails and ZFS are a really nice way to deal with it, if you can go that route.
The jail is simply created with:
# zfs clone fbsd0/jails@20120214 fbsd0/jails/anotherone

Add a few parameters in rc.conf and you're good to go :e

If you separate the data from the jails, updating them should be relatively easy too. With some clever renaming to swap the 'old' snapshot with a 'new' one for each jail.

I just started with my 1st jail. Thought it was about time I finally got the hang of jails. I cloned my first jail from a snapshot which I populated by using [cmd=]make installworld DESTDIR=/export/jails/jmail1/root[/cmd]
So now I got a vanilla jail on a ZFS dataset. Great.

I'm planning to only run a mailserver in jmail1 which I want to install from ports. On the host I use portsnap to maintain the ports tree. How would I best go about this? Should I do a [cmd=]portsnap fetch update[/cmd] from within each jail or is there a way to re-use the ports tree from my host?

SirDice · Feb 21, 2012

razrx said:
I'm planning to only run a mailserver in jmail1 which I want to install from ports. On the host I use portsnap to maintain the ports tree. How would I best go about this? Should I do a [cmd=]portsnap fetch update[/cmd] from within each jail or is there a way to re-use the ports tree from my host?

Use nullfs(5):
# mount -t nullfs /usr/ports /jails/myjail/usr/ports/

bsus · Feb 21, 2012

So to summarize:

1. Create a new ZFS dataset which keeps the main system.
2. Clone this dataset
3. mount ports over jailfs

What is unclear for me:
How does the software get updated?
How does the base system get updated?

Regards

razrx · Feb 21, 2012

The way I see it, you update the software within each jail using your method of choice using ports. That will probably be portmaster (ports-mgmt/portmaster).

I'm interested to hear how people are updating the base system within a jail as well. I guess one would update the actual host as so:

Code:

% make buildworld
% make kernel
% make installworld

Then run:
# make installworld DESTDIR=/path/to/jail to update said jail.

bsus · Feb 22, 2012

Hi,

As long you are not using the stable branch you also can use freebsd-update fetch/install.

What disturbs me is that it seems that I have to update each base-system extra.

As example I would have following zfs datasets:

jroot
jfamp (clone of jroot with apache-mysql-php ports)
jfamp1
jfamp2 (clones of jfamp)
jwww (clone of jroot with apache ports)
jmail (clone of jroot with mail ports)
jmail1 (clone of jmail)
...

When I now want to update the basesystem, I would have to rebuild or update each jail. My question is, if there isn't a way to symlink the basesystem so that I only have to update one base-system?

Regards

bsus · Feb 23, 2012

Code:

zfs create storage/home/jails
zfs create storage/home/jails/mroot
cd /usr/src
make installworld DESTDIR=/home/jails/mroot
zfs snapshot storage/home/jails/mroot storage/home/jails/mroot@20120223
zfs clone storage/home/jails/mroot@20120223 storage/home/jails/famp
zfs list
storage/home/jails         394M  5.56T   256K  /usr/home/jails
storage/home/jails/famp   12.8K  5.56T   394M  /usr/home/jails/famp
storage/home/jails/mroot   394M  5.56T   394M  /usr/home/jails/mroot
mergemaster -t /home/jails/mroot/var/tmp/temproot -D /home/jails/famp -i

How can I now update:
1.) the base system
2.) the ports software
without looping each jail with update commands?

Regards

_martin · Feb 23, 2012

Depends what you are running in those jails. Sometimes it doesn't hurt to do a fresh install (cleanup old mess, etc.), you can use application configuration files/data from old jails. ezjail + ZFS == dream come true

. It has this solaris-zones feel

.

Plus with ezjail it's really easy to keep them up-to-date. So you'd be set for next update when needed.

bsus · Feb 23, 2012

How does the configuration differ from default with using zfs (clone)?

_martin · Feb 23, 2012

bsus said:
How does the configuration differs from default with using zfs (clone)?

Jail is jail no matter what underlying FS you use. With ezjail, see the man page for ezjail and ezjail-admin command for more details. Basically, you do:

# ezjail-admin create [-xbi] [-f flavour] [-r jailroot] [-s size] [-c bde|eli|zfs] [-C args] [-a archive] jailname jailip

where you specify -c zfs to create new ZFS dataset for jail.

bsus · Feb 23, 2012

I now created some jails using ezjail on a zfs dataset following this tutorial

But I am still unsure where I can let zfs play its features like cloning...
I didn't mention at which point I could use them because I don't see what ezjail does.

_martin · Feb 23, 2012

That tutorial is good, but little bit older I think. It's not using ZFS at all.

For example, let's say you have a ZFS pool zpool and want to set jails in /local/jails with ezjail.

# zfs create -o mountpoint=/local/jails zpool/jails
# zfs list zpool/jails

Code:

NAME          USED  AVAIL  REFER  MOUNTPOINT
zpool/jails    31K  1.95G    31K  /local/jails

Customize the ezjail configuration in /usr/local/etc/ezjail.conf. My settings differ only in:

# grep -vE '^$|^#' /usr/local/etc/ezjail.conf

Code:

ezjail_jaildir=/local/jails
ezjail_ftphost=ftp.sk.freebsd.org
ezjail_use_zfs="YES"
ezjail_jailzfs="zpool/jails"

Choose the FTP server closest to you. You can either use your sources or fetch already built base. I'll fetch the base here:

# ezjail-admin install

Now the basejail is installed. See the man pages for flavors and how you can set the defaults in basejail ($jail/etc/rc.conf, DNS, etc.).

Now I'll create new jail (IP address is active on one of my loopback interfaces already):

# ezjail-admin create -c zfs mail 192.168.252.1

Code:

ZFS: create the jail filesystem

You can see the separate dataset was created on pool for this jail:

# zfs list

Code:

NAME                   USED  AVAIL  REFER  MOUNTPOINT
zpool                  333M  1.63G    31K  none
zpool/jails            333M  1.63G    50K  /local/jails
zpool/jails/basejail   329M  1.63G   329M  /local/jails/basejail
zpool/jails/mail      1.70M  1.63G  1.70M  /local/jails/mail
zpool/jails/newjail   1.70M  1.63G  1.70M  /local/jails/newjail

To start the jail:

# /usr/local/etc/rc.d/ezjail start mail

Code:

Configuring jails:.
Starting jails: mail.

# jls

Code:

   JID  IP Address      Hostname                      Path
     1  192.168.252.1   mail                          /local/jails/mail

To access the console:

# ezjail-admin console mail

Anything else is the same as you had already done on your jails.

bsus · Feb 24, 2012

Hi,

Thank you for your howto, you have saved me a lot work

One last question:

Code:

df
storage/jails          5964729829        389 5964729440     0%    /usr/jails
storage/jails/basejail 5966621403    1891963 5964729440     0%    /usr/jails/basejail
storage/jails/famp     5964734571       5131 5964729440     0%    /usr/jails/famp
storage/jails/newjail  5964734686       5246 5964729440     0%    /usr/jails/newjail
/usr/jails/basejail    5966621403    1891963 5964729440     0%    /usr/jails/famp/basejail
devfs                           1          1          0   100%    /usr/jails/famp/dev
fdescfs                         1          1          0   100%    /usr/jails/famp/dev/fd
procfs                          4          4          0   100%    /usr/jails/famp/proc

What is the function of

Code:

/usr/jails/basejail    5966621403    1891963 5964729440     0%    /usr/jails/famp/basejail

?

Best Regards,
bsus

_martin · Feb 24, 2012

bsus said:
What is the function of

Code:

/usr/jails/basejail 5966621403 1891963 5964729440 0% /usr/jails/famp/basejail

Check it this way from host:

# mount -v | grep basejail | grep mail

Code:

/local/jails/basejail on /local/jails/mail/basejail (nullfs, local, read-only)

Basically it's a local mount, read-only. See mount_nullfs(8) for details.

Both security and saving space on filesystem is achieved by this. This is how ezjail does its work. In my example above I've fetched binary base and created a basejail - a template for any new jail I'll create. This can be fine-tuned by flavors (add custom software, etc) and newjail configuration.

It is mounted read-only, so the the base system cannot be changed within jail, not even by its root (user root in jail).

As an example look here:

Code:

mail:(/)# ll | grep basejail
drwxr-xr-x   9 root  wheel    9 Feb  3 14:59 basejail
lrwxr-xr-x   1 root  wheel   13 Feb  3 14:59 bin -> /basejail/bin
lrwxr-xr-x   1 root  wheel   14 Feb  3 14:59 boot -> /basejail/boot
lrwxr-xr-x   1 root  wheel   13 Feb  3 14:59 lib -> /basejail/lib
lrwxr-xr-x   1 root  wheel   17 Feb  3 14:59 libexec -> /basejail/libexec
lrwxr-xr-x   1 root  wheel   16 Feb  3 14:59 rescue -> /basejail/rescue
lrwxr-xr-x   1 root  wheel   14 Feb  3 14:59 sbin -> /basejail/sbin
mail:(/)#

Inside of the jail, all mandatory directories (bin/lib/libexec/sbin, etc.) are linked to the one and only directory: read-only basejail.

This also comes in handy during update - you just need to update basejail and every jail will have its system userspace updated. Update is done via ezjail-admin command.

I forgot to mention: set the restrictive permissions on $JAILDIR, /local/jails in my example:

# ll -d /local/jails

Code:

drwx------  9 root  wheel  9 Feb 10 15:31 /local/jails

bsus · Feb 24, 2012

Both security and saving space on filesystem is achieved by this. This is how ezjail does its work. In my example above I've fetched binary base and created a basejail - a template for any new jail I'll create. This can be fine-tuned by flavors (add custom software, etc) and newjail configuration.

So ezjail creates another copy of the jail. Smart, so I "multiplicate" one FreeBSDApacheMySQLPHP jail. That is nice

This also comes in handy during update - you just need to update basejail (should be somewhat the same for flavors; not 100% sure as I'm not using any) and every jail will have its system userspace updated. Update is done via ezjail-admin command.

I was already wondering how they managed to update as little as possible.

Are you developer of ezjail tool?

I forgot to mention: set the restrictive permissions on $JAILDIR, /local/jails in my example:

Does a simple:
# chmod -Rfv 700 /usr/jails do?
Or do I have to use ACLs?

_martin · Feb 24, 2012

bsus said:
Does a simple:
# chmod -Rfv 700 /usr/jails do?
Or do I have to use ACLs?

That's one big NO-NO!. Jail itself has to be readable to others (within jails, etc.) - just set /usr/jails, do the simple:

# chmod 700 /usr/jails; chown root:wheel /usr/jails

Leave any other descendant permissions as is.
Indeed this tool is great to manage jails. It's a wrapper script, but very useful one.

Are you developer of ezjail tool?

Nope. To see who's the developer you can use ports (and check who's maintaining it in ports):

Code:

# cd /usr/ports/sysutils/ezjail
# grep ^MAINTAINER Makefile ; cat pkg-descr
MAINTAINER=     erdgeist@erdgeist.org
This port contains two scripts to easily create, manipulate and run
FreeBSD jails.

WWW: http://erdgeist.org/arts/software/ezjail/

bsus · Feb 24, 2012

Ok thank you for your help

Maybe someone could copy your posts together and reopen a howto thread?

Regards,
bsus

_martin · Feb 24, 2012

bsus said:
Ok thank you for youre help

Maybe someone could copy youre posts together and reopen a howto thread?

Regards,
bsus

No problem.

I'm not sure if my post is howto material but I can add it there along with quick PF setup and an example of port redirection.

UPDATE: I have added this thread to howto section: http://forums.freebsd.org/showthread.php?p=167879

bsus · Feb 25, 2012

Thank you, have bookmarked it

Restoring jails

bsus

SirDice

Administrator

razrx

SirDice

Administrator

bsus

razrx

bsus

bsus

_martin

bsus

_martin

bsus

_martin

bsus

_martin

bsus

_martin

bsus

_martin

bsus