Cannot upgrade jail from 11.1-RELEASE to 11.2-RELEASE

K

kjpetrie

I'm unsure whether this is a Base System problem or a ports problem as it involves the first within the other.

I maintain a number of jails on a test and live server using sysutils/ezjail and am trying to upgrade my test server from 11.1 to 11.2. I have done the host system but attempting to upgrade the base jail produces:

Code: 
ezjail-admin update -U -s 11.1-RELEASE

src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 2 mirrors found.
Fetching metadata signature for 11.1-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files... done.
Inspecting system... done.

The following components of FreeBSD seem to be installed:
kernel/generic world/base

The following components of FreeBSD do not seem to be installed:
kernel/generic-dbg world/base-dbg world/doc

Does this look reasonable (y/n)? y

Fetching metadata signature for 11.2-RELEASE from update5.freebsd.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files... done.
Inspecting system...  done.
Fetching files from 11.1-RELEASE for merging... done.
Preparing to download files... done.
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
/usr/sbin/freebsd-update: cannot open files/.gz: No such file or directory
Attempting to automatically merge changes in files... done.

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /boot/device.hints
Does this look reasonable (y/n)?  y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/auto_master
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/autofs/include_ldap
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/autofs/include_nis
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/autofs/include_nis_nullfs
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/autofs/special_hosts
Does this look reasonable (y/n)?  y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/autofs/special_media
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/autofs/special_noauto
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/autofs/special_null
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/blacklistd.conf
Does this look reasonable (y/n)? y

The following file will be removed, as it no longer exists in
FreeBSD 11.2-RELEASE: /etc/dma/dma.conf
Does this look reasonable (y/n)?  y

The following files will be added as part of updating to 11.2-RELEASE-p0:
/etc/rc.d/ipfw_netflow
/var/db/etcupdate/current/etc/rc.d/ipfw_netflow
To install the downloaded upgrades, run "/usr/sbin/freebsd-update install".
src component not installed, skipped
Installing updates...install: /usr/jails/basejail//etc/rc.d/ipfw_netflow: No such file or directory
install: /usr/jails/basejail//var/db/etcupdate/current/etc/rc.d/ipfw_netflow: No such file or directory
 done.

src component not installed, skipped
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
src component not installed, skipped
....

Those last three lines repeat indefinitely in what appears to be an infinite loop.

What am I doing wrong here and how do I fix it?
 
Thanks. I've been studying ezjail-admin(8) most of the day but can't see anything about upgrading to a new minor version.

When you say a new base jail doesn't affect pre-built jails, do you mean it will be used (either automatically or by a simple changeover procedure) or that I'd have to build new jails as well?
 
I do concur that something odd seems to be going on. Do note that I don't use managers such as ezjail because I prefer to stay in full control myself.

Even so:

Code: 
root@psi:~ # uname -a
FreeBSD psi.intranet.lan 11.2-RELEASE FreeBSD 11.2-RELEASE #0 r335886: Sun Jul  8 03:46:16 CEST 2018     peter@zefiris.intranet.lan:/usr/obj/usr/src/sys/ZEFIRIS  amd64
root@psi:~ # freebsd-version
11.1-RELEASE-p8
root@psi:~ # freebsd-update upgrade -r 11.2
freebsd-update: Cannot upgrade from 11.2-RELEASE to itself
That seems odd to me because it would appear that freebsd-update only looks at the version version yet has no attention for the actual userland version. Especially if you consider that I'm only letting it manage the World component and nothing else.

As said: I don't have any ezjail hands on experience but this does make me wonder if the issues could be related.
 
Sorry, I got the version wrong. I think you do need the full name, not just the number. I've corrected my post.

Anyway, back to my problem. Having killed the looping process, I now find the jails claim to be upgraded. However, mergemaster fails (whether I use the "trusted" or "untrusted" method) with:
Code: 
root@zzzz:/usr/src # mergemaster -U

*** The directory specified for the temporary root environment,
    /var/tmp/temproot, exists.  This can be a security risk if untrusted
    users have access to the system.

  Use 'd' to delete the old /var/tmp/temproot and continue
  Use 't' to select a new temporary root directory
  Use 'e' to exit mergemaster

  Default is to use /var/tmp/temproot as is

How should I deal with this? [Use the existing /var/tmp/temproot] d

   *** Deleting the old /var/tmp/temproot

*** Creating the temporary root environment in /var/tmp/temproot
*** /var/tmp/temproot ready for use
*** Creating and populating directory structure in /var/tmp/temproot

make: don't know how to make distrib-dirs. Stop

  *** FATAL ERROR: Cannot 'cd' to /usr/src and install files to
      the temproot environment
 
OK I can't speak to 3rd party jail(8) management. Because they ultimately make jails, and management much less flexible, and are often prone to "mysterious" failures that are harder to diagnose. But having said that. It clearly appears that your jail environment tainted/corrupt. You will need to delete everything, and start afresh.
Honestly, tho.
You could far more easily create one the "traditional" way, and save a lot more time, be less subject to these kind of interruptions.
But that's just my opinion. :)
If you feel you must use a 3rd party (ports) management application. You will probably find ports-mgmt/synth just works, and works well.

--Chris
 
Ports might technically be classed as third-party applications, but this port is commended and its advantages explained in the FreeBSD Handbook, so I think it is really a port because for many users it is an optional extra rather than because it is foreign to FreeBSD. I am following instructions in the Handbook.

I chose FreeBSD because I want a secure, easily maintained OS on my rented server with limited resources, not because I want to spend my time playing with jails or ironing out quirks. My test server, on which this problem has emerged, reflects the constraints in which my live server operates. It has taken years so far to build the system I now have and I cannot start again from scratch as I need to get it finished so the live server can go fully live in pursuit of my true object.

I have come here for help. Scrapping my current jails would not be helpful.
 
Did you install the source tree under /usr/src ?
 
No. On my test server I have a /usr/src directory containing only contrib and usr.bin directories (dating from 2015), whereas there are far more directories (dated 27th July) on the live server. Would that be the result of the upgrading process being confused by the absence of the src component?

On both machines /usr/jails/<jailname>/usr/src are symbolic links to /basejail/usr/src.

mergemaster works well on the live server, but not on the test server. I'm trying to remember whether the live server also went into the loop at the end of ezjail-admin update -U -s 11.1-RELEASE. I think it did.
 
kjpetrie said:
Ports might technically be classed as third-party applications, but this port is commended and its advantages explained in the FreeBSD Handbook, so I think it is really a port because for many users it is an optional extra rather than because it is foreign to FreeBSD. I am following instructions in the Handbook.
Click to expand...
But then you're ignoring a golden rule for Unix(-like) environments: "What works for me doesn't have to work for you". Just because it's mentioned in the handbook doesn't imply that this is a given standard on managing jails. In fact, it's not.

And you're probably also unaware of another very important aspect of working with Unix: the so called Unix philosophy which is very much alive within FreeBSD as well. In summary:
  • Write programs that do one thing and do it well.
  • Write programs to work together.
  • Write programs to handle text streams, because that is a universal interface.
The reason this is important is because those high-end tools like ezjail basically do one thing: they make use of all the underlying small programs to accomplish their goals.

Although this can sometimes make it easier on an end user to perform certain tasks it also obscures them from the things which are really going on, often making them somewhat dependent on the tool at hand. If they tool can't fix their problem they're lost. Even though they wouldn't have to be if they'd simply understand the underlying mechanics so that they can apply those.

Small (offtopic) sidestep: the FreeBSD installer. There are several people who have tried to study it and see if they could script it into doing something else. Nothing wrong with that perse, but most likely unneeded because all the installer does is perform a few specific steps. Steps which you can easily perform yourself, as I've demonstrated in this guide. So instead of studying the installer on how to script it some people would be much better off if they'd simply replace the installer entirely with a script of their own.

This situation really is no different.

Despite my earlier issues I eventually upgraded my jail with ease, yet manually. By simply using the original archives (base.txz, kernel.txz and lib32.txz) to replace what had to be replaced.

kjpetrie said:
I chose FreeBSD because I want a secure, easily maintained OS on my rented server with limited resources, not because I want to spend my time playing with jails or ironing out quirks.
Click to expand...
An operating system is only as secure as its administrators make it. There is no such thing as a "secure operating system". Of course there's a bit more to it than that, sure, but my main point here: if you're relying on the OS to keep you secure then you'll honestly be in big problems really soon, especially if this involves a Unix environment.

I mean... Windows will refuse to remove most of c:\windows even if you try it. It could still damage your system, sure, but it has many failsaves. Unix on the other hand has no problems at all with wiping itself out from existence (a famous "rm -rf" command which I shall not copy here).

But now I do get confused... If you have limited resources then why on earth set up an environment with "a number of jails"? Sorry, but I really don't follow the logic there. Because all it will do is add time to your administrative tasks. All those jails need to be maintained as well.

Seriously: I'd rather use one well maintained FreeBSD environment to host my services than using several "vaguely maintained" virtual environments such as jails. The added security from process separation (note: process, it still uses the same kernel!) could be easily nullified from lack of administration. I am NOT trying to imply that this is the case here, but I do think it's an important aspect to keep in mind.

But maintaining one system is enough work as it is. Why add a multitude to that like this?

Just using a jail is by no means a guarantee that you'll be safe no matter what.

kjpetrie said:
It has taken years so far to build the system I now have and I cannot start again from scratch as I need to get it finished so the live server can go fully live in pursuit of my true object.
Click to expand...
Who said anything about having to start from scratch?

Know how I upgraded my jail? I somewhat trashed it. But that doesn't mean I have to start from scratch, as in: re-invent all the cool stuff I put into it over the years. That's nonsense.

You just have to know what you're doing.

I obviously secured /etc and /usr/local/etc because of the configuration. I then secured /var entirely because I'm lazy and I didn't want to waste time on that. Then one final visit to the jail to secure the output from pkg info -qo and I was well on my way. (several aspects within my jail are provided from the host itself).

Next stop: I cloned zroot/opt/jails/psi (=backup), trashed the original and recreated the filesystem (deleting on ZFS is resource intensive, this was therefor quicker). I extracted the new base system and then dumped my backups from /etc and /usr/local/etc in there. Of course just the relevant files. Then I checked the data I needed from /var and added that too (in my case this was only /var/cron/tabs and /var/log) and then I started my 'new' jail. Went in ( # jexec psi csh) and re-installed all the binary packages: # pkg install `cat /root/pkg.list`.

Done.

So even though I started from 'scratch' I never got any setbacks from it.
 
kjpetrie said:
No. On my test server I have a /usr/src directory containing only contrib and usr.bin directories, whereas there are far more directories on the live server.

On both machines /usr/jails/<jailname>/usr/src are symbolic links to /basejail/usr/src.

mergemaster works well on the live server, but not on the test server.
Click to expand...
If I remember, last time I updated my jails from 11.1-RELEASE to 11.2-RELEASE, I had to fetch the source tree to run mergemaster.
 
Thanks dlegrand. At least you're offering useful information and not making wild assumptions about what I do or don't know or might be trying to achieve.
 
kjpetrie said:
Ports might technically be classed as third-party applications, but this port is commended and its advantages explained in the FreeBSD Handbook, so I think it is really a port because for many users it is an optional extra rather than because it is foreign to FreeBSD. I am following instructions in the Handbook.

I chose FreeBSD because I want a secure, easily maintained OS on my rented server with limited resources, not because I want to spend my time playing with jails or ironing out quirks. My test server, on which this problem has emerged, reflects the constraints in which my live server operates. It has taken years so far to build the system I now have and I cannot start again from scratch as I need to get it finished so the live server can go fully live in pursuit of my true object.

I have come here for help. Scrapping my current jails would not be helpful.
Click to expand...
kjpetrie
My reply to you was not intended to be denigrating in any way. Quite the opposite. As I've been riding UNIX in it's various stages for ~30yrs I was only attempting to share my overall experience(s) in this area in an attempt to save you some (unnecessary) grief.
You should do whatever you want. It's your system, thereby your choice. I can accomplish a "custom" (traditional) jail(8) in ~16 line shell script. To me, this seems like an extremely lightweight, and simple management. It's a lighter weight option. The direction you chose, has additional requirements that a traditional jail, does not. My choice is without limitation, yours is.
But again; it's your system, and your choice. I was only trying to help, and help you see the difference.

--Chris
 
You must log in or register to reply here.
Back
Top