jails share packages, binaries, perl modules across CBSD jails

[RPUgCZUqR4qZ]

We have 3 servers with ~40 jails on each under CBSD management.
Each jail running our perl application.

We would like to share installed packages, binaries, installed perl modules across jails.
Plain is
master jail as a base image
jails for application created via `cbsd jclone` , then via ZFS we share folders from master jail to app jails

main goal is immediate installation packages and perl modules on all jails

what is possible downsides of this approach?

thank you

 

[SirDice]

what is possible downsides of this approach?

Instantly breaking everything at the same time. You can prevent it by running test upgrades/changes on a different system. But this is always a possibility, if there's something wrong with the installed packages on that base application jail every other jail immediately breaks too.

 

[RPUgCZUqR4qZ]

Thank you.

I think better solution is use Ansible playbooks for creating master jail and for redeployment jails.

Unfortunately FreeBSD docker port is too old, and probably not a best solution for using with CBSD.

 

[Ole]

We have 3 servers with ~40 jails on each under CBSD management.
Each jail running our perl application.

We would like to share installed packages, binaries, installed perl modules across jails.
Plain is
master jail as a base image
jails for application created via `cbsd jclone` , then via ZFS we share folders from master jail to app jails

main goal is immediate installation packages and perl modules on all jails

what is possible downsides of this approach?

thank you

If we are talking about single server only, you can use nullfs to make some directory shared between jails. For example, use ~cbsd/jails-fstab/jailname/fstab.local file for any custom mount, e.g.:

/tmp/shared /usr/local/shared nullfs rw 0 0

For example:

mkdir /tmp/test
cd /tmp/test

create /tmp/test/CBSDfile file like this:

preup()
{
        [ ! -d /tmp/shared ] && mkdir /tmp/shared
}

# we may not write the same settings in each container
#globals()
#{
#    pkg_bootstrap="0"
#    sysrc="syslogd_enable=NO cron_enable=NO sendmail_enable=NO sendmail_submit_enable=NO sendmail_outbound_enable=NO sendmail_msp_queue_enable=NO"
#}

jail_test1()
{
        ip4_addr="DHCP"
        host_hostname="${jname}.example.com"
        pkg_bootstrap="0"
        sysrc="syslogd_enable=NO cron_enable=NO sendmail_enable=NO sendmail_submit_enable=NO sendmail_outbound_enable=NO sendmail_msp_queue_enable=NO"
}

jail_test2()
{
        ip4_addr="DHCP"
        host_hostname="${jname}.example.com"
        pkg_bootstrap="0"
        sysrc="syslogd_enable=NO cron_enable=NO sendmail_enable=NO sendmail_submit_enable=NO sendmail_outbound_enable=NO sendmail_msp_queue_enable=NO"
}

jail_test3()
{
        ip4_addr="DHCP"
        host_hostname="${jname}.example.com sendmail_enable=NO sendmail_submit_enable=NO sendmail_outbound_enable=NO sendmail_msp_queue_enable=NO"
        pkg_bootstrap="0"
        sysrc="syslogd_enable=NO cron_enable=NO"
}

create fstab.local template and put to 'jails-system' directory:

mkdir jails-system
cat > jails-system/fstab.local <<EOF
/tmp/shared /usr/local/shared nullfs rw 0 0
EOF

When you run `cbsd up`, you will create three containers that have ~cbsd/jails-fstab/test{1-3}/fstab.local:

cbsd up
# let's save some artifact to a common directory
date > /tmp/shared/date.txt

# read data.txt from jails:
cbsd jexec jname='test*' cat /usr/local/shared/date.txt
test1: success in 0 sec:

Thu Apr 25 10:57:46 MSK 2024

test2: success in 0 sec:

Thu Apr 25 10:57:46 MSK 2024

test3: success in 0 sec:

Thu Apr 25 10:57:46 MSK 2024

This way you can mount a single set of packages (e.g. `/usr/local/` directory) into each container ( in RO mode if you need write protection ).

If you use multiple nodes, then you can share a network file system (and still use fstab.local), or as you noticed earlier - create a 'master jail' image (or snapshot) and create a container from it, (use `cbsd jsnapshot` to create <SNAPSHOT> of 'gold' jail), e.g.:

cbsd jcreate jname=xxx1 zfs_snapsrc=<SNAPSHOT>

or create jail from HTTP server (use `cbsd jexport --help` to create `myapp.img`):

cbsd jcreate jname=myapp1 from=https://dl.example.com/img/amd64/14.0/myapp.img pkg_bootstrap=0 runasap=1 ip4_addr=DHCP

or via CBSDfile:

jail_myapp1()
{
        # Use remote image. You can comment this line to build the image locally.
        from="https://dl.example.com/img/`uname -m`/`sysctl -n kern.osrelease | cut -d - -f 1`/${jname}/${jname}.img"

        ip4_addr="DHCP"
        host_hostname="${jname}.my.domain"
        ver="native"
        interface="auto"
        runasap=1
        pkg_bootstrap=0
}