Created jail with mismatched versions using bsdinstall.

Recently after an upgraded from freebsd 14.3 to 14.4 we got not working ssh sever.
When we started the ssh server we got.

Starting sshd.
/usr/libexec/sshd-auth does not exist or is not executable
/etc/rc.d/sshd: WARNING: failed to start sshd done.


The file /usr/libexec/sshd-auth is present in the 14.4 freebsd version, not in the previous versions.

It looks no one on the internet expect this one Chinese person got the same problem.


It looks like we have version mismatch inside the jail.

freebsd-version -ru
14.4-RELEASE-p5
14.3-RELEASE


How did it happen ?

Unlike most jail users, for creating jail we use
bsdinstall jail rootfs


And it downloads the newest freebsd version to

/usr/freebsd-dist/base.txz
/usr/freebsd-dist/MANIFES


But if the files are present, it does not check them and does not download them again, but it expects it is the newest version.
Which may not be true if you used the command while ago.
So you create a jail with version mismatch.

I have fixed my jail create script called jcreate, so now it is checking the version of /usr/freebsd-dist/base.txz .
And offers to clear the cache so the new version is downloaded.

Hope it helps someone.

Bash:
#! /bin/sh

jailName="$1"

if [ -z "${jailName}" ];
then
 echo " missing jail name "
 exit
fi

yesno() {
  while :; do
    printf "%s [y/n]: " "$1"
    read ans
    case "$ans" in y|Y) return 0 ;; n|N) return 1 ;; *) echo "y or n only" ;; esac
  done
}


mkdir -p /jails/$jailName

if zfs list zroot/jails/$jailName; then
  echo " Dataset $jailName already exists !"
  exit
else
 zfs create -o mountpoint=/jails/$jailName zroot/jails/$jailName
 # zfs set recordsize=8K zroot/jails/$jailName
 cd  /jails/$jailName

 echo "In file /usr/freebsd-dist/base.txz is already downloaded version"
 echo
 tar -xOf /usr/freebsd-dist/base.txz ./bin/freebsd-version | sed -n 's/.*USERLAND_VERSION="\([^"]*\)".*/\1/p'
 echo

 echo "The newest versions are:"
 fetch -qo - https://download.freebsd.org/releases/amd64 | grep -Eo '[0-9]+\.[0-9]+-RELEASE' | sort -Vu
 echo
 echo "I must always install the newest available version when creating a jail or you get in trouble."
 echo "You would get jail with version mismatch as bsdinstall is expecting the newest version"
 echo "in the directory /usr/freebsd-dist/ and it does not check it."
 echo
if yesno "Should I delete /usr/freebsd-dist/base.txz, so bsdinstall downloads newest version? [y/n]?"; then
  rm /usr/freebsd-dist/base.txz
  rm /usr/freebsd-dist/MANIFEST
fi


 #  it creates the directiory rootfs on on its own
 bsdinstall jail rootfs
 datum=`date "+%Y-%m-%d %H:%M:%S"`
 printf "$jailName{\n\$note = \"Created $datum\";\n\$ip = \"192.168.144.XXX/24\";\n}">/jails/$jailName/c.conf

 echo "Doing update patches"

 freebsd-update -b /jails/$jailName/rootfs fetch
 freebsd-update -b /jails/$jailName/rootfs install

 echo "Do not forget to change ip adress in /jails/$jailName/c.conf"
 echo "Remember to setup firewall."
 echo "Install apps for example by \"pkg install screen vim mc\" inside."
fi
 
A jail doesn't have a kernel, it runs on the host's kernel.
You are right that it is not proof of the versions mismatch.
However look at the following:

root@hokuspokus4:/ # freebsd-version
14.3-RELEASE
root@hokuspokus4:/ # freebsd-update fetch
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 3 mirrors found.
Fetching metadata signature for 14.4-RELEASE from update1.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.
The following files will be added as part of updating to
14.4-RELEASE-p7:


Why doe freebsd-update fetch want to upgrade to 14.4-RELEASE ? I believe it should only apply patches to the 14.3-RELEASE.
 
I see the freebsd-update fetch inside a jail does not work correctly if the version of jail and the physical computer differs.
And in the case physical machine has got a higher version applies the wrong set of patches as it reads the version of kernel instead of userland.
Much better would be if freebsd-update would detect the situation it is not able to handle and refuse to proceed.

As coming from Linux doing upgrade inside a container is normal way how to proceed.

So the best way is to do it from outside the jail using -j jailname parameter ?
 
So the takeaway from this is?
Make sure you update /usr/FreeBSD-dist/ manually when updating a JAIL HOST if you create jails with bsdinstall.
 
So the takeaway from this is?
Make sure you update /usr/FreeBSD-dist/ manually when updating a JAIL HOST if you create jails with bsdinstall.
No. I was wrong. The main take away is not to do upgrade from inside a jail, or you can end up with unconsistent system.

If you you use bsdinstall, it is good practice to delete the content /usr/freebsd-dist/ so you do not install older version. So you can skip the upgrade steps. But it was not the main cause of my troubles. The main trouble was, that I was doing upgrade from inside the jail which makes a mess. It was mixing older version with new patches.

The tricky thing was in my case, that the update from inside used to work when the jail and physical server had the same version. So I learned to do it. It stopped working after I upgraded the physical server and made a mess from a jail instead. Applying newer patches 14.4 on older 14.3 version which caused a missing file, but could also cause other, possibly security issues.
 
So doing it from outside did not help.
I have jail hokuspokus2 on version 14.3 release and try to upgrade to the latest patch level of 14.3 branch. I run.
freebsd-update -b /jails/hokuspokus2/rootfs fetch


But it tries to upgrade to version 14.4-RELEASE-p7 . So it is the same like when I do it from inside? Why ?
 
I found this statement.


Starting with a commit merged in February 2026, freebsd-update now requires you to specify the current version when using the -b flag for a non-root directory. You can do this either with the --currently-running command-line option or by setting UNAME_r in the environment.

So if you run the

freebsd-update -b /jails/hokuspokus2/rootfs fetch

You have to use the --currently-runnig or the patches will be applied from the wrong version, and the jail will report a wrong version.

So I if the host version is different then the jails version it is absolutely necessary to use --currently-runnig .
Unfortunately there are many wrong examples around without it. For example here: https://freebsdfoundation.org/freebsd-project/resources/introduction-to-freebsd-jails/

It is dangerous, because normally you will not notice a problem, but you may get very strange behavior later, or miss a security fix.

Now a warning message was added ?

 
Initially, you spoke of the -j switch, not the -b one.

In freebsd-update(8), you can read:
Code:
-j jail       Operate  on  the    given  jail  specified by jid or name.
              (The version of the installed userland is     detected  and
              the --currently-running option is    no more    required.)

That said, I never used the -j switch. For my thin jails, the script is something like:
Code:
ver=`$Jdir/base/bin/freebsd-version -u`
env UNAME_r=$ver freebsd-update -b $Jdir/base \
 -d $Jdir/base/var/db/freebsd-update -f $Jdir/base/etc/freebsd-update.conf \
 -r $release upgrade install
So, I set the jail version with UNAME_r var, but also I use a special config file with -f where I explicitly set that I don't want any upgrade of the kernel.

PS: your link to mail-archive leads to a 404 page.
 
Initially, you spoke of the -j switch, not the -b one.

So, I set the jail version with UNAME_r var, but also I use a special config file with -f where I explicitly set that I don't want any upgrade of the kernel.

PS: your link to mail-archive leads to a 404 page.
The -j switch or the UNAME_r is also possible solution how to avoid my troubles. Thanks for the -d and -f parameters. I will add them.

I corrected the link to the mail-archive. The take away really is that in order to freebsd-update to work correctly, you must provide the version or jail name with the -j parameter. Otherwise you get version mismatch and hidden trouble for later.

The surprising fact is that you need the give the version also if you do only patching. This is what is missing in the documentation. And it is missing in the examples you can find on the internet. Seems many people got it wrong.

--currently-running release
Do not detect the currently-running release; instead,
assume that the system is running the specified release.
This is most likely to be useful when upgrading jails.

--curently-running needs to be used not only when upgrading, but also when updating a jail. That was confusing for me, because I did not see an example of that use anywhere.
 
I corrected the link to the mail-archive. The take away really is that in order to freebsd-update to work correctly it must give it the version or your jail (or jail name with the -j parameter). Otherwise you get in hidden trouble later.
I see. It's just an additional security layer which avoid freebsd-update to run if a jail version isn't defined. But, you always needed to give the jail version to freebsd-update in case you use -b option.

The surprising fact is that you need the give it a freebsd version also if you do only patching too. This is what is missing in the documentation. And it is missing in the examples you can find on the internet.
There is no great difference between upgrade and update. I use the same code to patch the base jail (except the commands are fetch install, of course).
 
Back
Top