jails Installing world into an empty jail

[bcrashkopf]

I'm trying to build an ultra-thin template for a set of jailed applications and I'm running into some issues getting a base system installed from source. make installworld doesn't seem to install the default scripts or config files into /etc. The only instructions I can find are for updating these files using etcupdate, but it doesn't seem to perform the function I'm looking for. I've also tried the distributeworld make target, but it doesn't do the thing either.

So I guess my question is: What is the proper workflow for installation into an empty directory?

For completeness, I'm using 14.2-RELEASE and here's what I'm doing:

export DESTDIR=`realpath -q $world/dist`
export SRCCONF=`realpath -q $world/src.conf`
make -C /usr/src buildworld
make -C /usr/src installworld

# src.conf
WITHOUT_TOOLCHAIN=1

WITHOUT_BOOT=1
WITHOUT_TESTS=1

WITHOUT_EXAMPLES=1
WITHOUT_MAN=1
WITHOUT_HTML=1
WITHOUT_DOCCOMPRESS=1
WITHOUT_SHAREDOCS=1
WITHOUT_INCLUDES=1

WITHOUT_ACCT=1
WITHOUT_AUDIT=1
WITHOUT_BSNMP=1
WITHOUT_CALENDAR=1
WITHOUT_DTRACE=1
WITHOUT_EE=1
WITHOUT_FINGER=1
WITHOUT_KDUMP=1
WITHOUT_LOCATE=1
WITHOUT_LPR=1
WITHOUT_PMC=1
WITHOUT_QUOTAS=1
WITHOUT_RESCUE=1
WITHOUT_ROUTED=1
WITHOUT_STATS=1
WITHOUT_TCSH=1
WITHOUT_VI=1

WITHOUT_INETD=1
WITHOUT_NTP=1
WITHOUT_OPENSSH=1
WITHOUT_PPP=1
WITHOUT_RBOOTD=1
WITHOUT_TALK=1
WITHOUT_TELNET=1
WITHOUT_TFTP=1

WITHOUT_MAIL=1

WITHOUT_BHYVE=1

WITHOUT_ACPI=1
WITHOUT_APM=1
WITHOUT_USB=1
WITHOUT_WIRELESS=1
WITHOUT_BLUETOOTH=1
WITHOUT_FLOPPY=1
WITHOUT_SYSCONS=1
WITHOUT_VT=1

WITHOUT_ZFS=1

 

[T-Daemon]

I've also tried the distributeworld make target, but it doesn't do the thing either.

On a 14.2 test setup, your custom src.conf, "distributeworld" fails with a missing file/directory error. I was unable to determine what the cause could be. The target fails also on a minimal src.conf (WITHOUT_LIB32, WITHOUT_DEBUG_FILES) 14.2 buildworld I use to update my system.

Meanwhile, as a workaround, build a base distribution file. Assuming SRCCONF is exported:

# make  -C  /usr/src/release  base.txz

/usr/obj/usr/src/amd64.amd64/release/base.txz can be extracted into the empty jail (or used by jail management tools).

"distributeworld" seems to have no issues when executed from release Makefile

/usr/src/release/Makefile

132 base.txz:
133         mkdir -p ${DISTDIR}
134         ( cd ${WORLDDIR} && ${IMAKE} distributeworld DISTDIR=${.OBJDIR}/${DISTDIR} )

 

[T-Daemon]

"distributeworld" fails with a missing file/directory error.

"distributeworld" seems to have no issues when executed from release Makefile

This target is valid when building release(7):

build(7)

     distributeworld  Distribute everything compiled by a preceding buildworld
                      step.  Files are placed in the directory hierarchy
                      specified by make(1) variable DISTDIR.  This target is
                      used while building a release; see release(7).

 

[T-Daemon]

make installworld doesn't seem to install the default scripts or config files into /etc.

"distribution" seems to be the appropriate target.

From /usr/src/UPDATING (releng/14.2)

2006         To cross-install current onto a separate partition

2020         make distribution DESTDIR=${CURRENT_ROOT} # if newfs'd

This populates /etc.

Example (assuming: DESTDIR and SRCCONF are exported, installworld)

/usr/src # make distribution DISTDIR=/tmp

EDIT

Instead of digging in the source tree I should have simply read the jail(8) manual.

EXAMPLES
     ...
   Setting up a Jail Directory Tree
     To set up a jail directory tree containing an entire FreeBSD
     distribution, the following sh(1) command script can be used:

           D=/here/is/the/jail
           cd /usr/src
           mkdir -p $D
           make world DESTDIR=$D
           make distribution DESTDIR=$D

     In many cases this example would put far more in the jail than needed.
     In the other extreme case a jail might contain only one file: the
     executable to be run in the jail.

     We recommend experimentation, and caution that it is a lot easier to
     start with a “fat” jail and remove things until it stops working, than it
     is to start with a “thin” jail and add things until it works.

/usr/src/Makefile

# world               - buildworld + installworld, no kernel.

 

[bcrashkopf]

Thanks for taking a look!

"distributeworld" seems to have no issues when executed from release Makefile

I was able to get distributeworld to run completely from /usr/src/Makefile by only setting DISTDIR and not DESTDIR. The two paths get concatenated for some recipes, but their relationship to each other is not made clear by the documentation.

This target is valid when building release(7):

And yet it's not mentioned in the release(7) manual, only the build(7) manual. I supposed I'll just have to dig into the release build system if I want to understand.

Meanwhile, as a workaround, build a base distribution file. Assuming SRCCONF is exported:

# make  -C  /usr/src/release  base.txz

/usr/obj/usr/src/amd64.amd64/release/base.txz can be extracted into the empty jail (or used by jail management tools).

This works! I had tried make -C /usr/src/release release previously, but it choked on not having a kernel.

 

[bcrashkopf]

"distribution" seems to be the appropriate target.

This mostly seems to work, though it leaves some files under the DISTDIR path while installing to DESTDIR

root@shiva:~ # find /dist -type f
/dist/base/usr/share/misc/ascii
/dist/base/usr/share/misc/birthtoken
/dist/base/usr/share/misc/bsd-family-tree

I think I'll just use your workaround of making the base tarball, I've spent enough time exploring this rabbit-hole

Thanks again!

 

[bcrashkopf]

Instead of digging in the source tree I should have simply read the jail(8) manual.
...
D=/here/is/the/jail
cd /usr/src mkdir -p $D
make world DESTDIR=$D
make distribution DESTDIR=$D

I'd call that a more or less official answer. Any idea what can be done to update the build(7) man page or the jails section of the handbook to refer to this information? I'd gladly volunteer an hour or two of my time to save others going down the same rabbit hole that I did.

 

[T-Daemon]

Any idea what can be done to update the build(7) man page or the jails section of the handbook to refer to this information?

I can think of two options: Option 1 (I would favor) discuss the problem on freebsd-current@ mailing list first, then open a PR, or option 2, open a PR directly.

One PR should address the issue for build(7), to include "world" and "distribution" targets, the other, jail(8), to add a reference to build(7) in the "Setting up a Jail Directory Tree" paragraph, addressing those two targets used in the example.

For the "world" target (src/Makefile world - buildworld + installworld, no kernel) one can argue with the presence of "kernel" (kernel Equivalent to buildkernel followed by installkernel) in build(7).

The "distribution" target on the other hand lacks any description anywhere (as far I can tell grep'ing through the source code). Perhaps it can be added on this occasion.

The FreeBSD handbook might not be considered to include a paragraph for creating a jail from a compiled source. FreeBSD developers might argue this topic is covered already in the jail(8) manual.

This happened with (base.txz) dist file in Bug 200803 - Change instructions in Jail(8) to use prebuilt distfiles to build a jail (see "Comment 4"), but this was 10 years ago.

That's why I suggested to discuss the issue on freebsd-current@ first, then open a PR.

Apropos PR, there is a FreeBSD article "Writing FreeBSD Problem Reports" explaining how to write a good problem report.