Upgrade failed 12.4 -> 13.2 (bash is broken)

Hello Community,

I have a problem with an upgrade of a server running currently 12.4-RELEASE-p9.

I am using a custom kernel and I tried upgrade the base system with following procedure:

1. sh BUILD132 & (to build world, kernel and packages)
1. bectl create 13.2-RELEASE
2. bectl mount 13.2-RELEASE /mnt
3. env ABI=FreeBSD:13:amd64 IGNORE_OSVERSION=yes pkg -r /mnt install -r LOCAL-base -g 'FreeBSD-*'
4. diff /etc/master.passwd /mnt/etc/master.passwd (no difference found)
5. diff /etc/group /mnt/etc/group (no difference found)
6. bectl activate 13.2-RELEASE
7. sync;sync;reboot
Here I can't proceed because I couldn't log in.
8. pkg bootstrap -f -r FreeBSD
9. pkg update -f
10. pkg upgrade -f
11. sync;sync;reboot

/usr/local/etc/pkg/repos/LOCAL-base.conf contains:
Code:
LOCAL-base: {
  url: "file://usr/obj/usr/src/13.2/repo/${ABI}/latest";
  mirror_type: "none",
  enabled: yes
}

/usr/local/etc/pkg/repos/FreeBSD.conf contains:
Code:
FreeBSD: {
  url: "pkg+http://pkg.FreeBSD.org/${ABI}/latest";
  mirror_type: "srv",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}
Build script which I run previously:
BUILD132
Code:
#!/bin/sh
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
if [ 'sysctl -n security.jail.jailed' -eq 0 ]; then
kldstat -q -m filemon || kldload filemon
cd /usr/src/13.2 || exit 100
nice make -j 8 TARGET=amd64 TARGET_ARCH=amd64 buildworld > /var/log/build_amd64:13.2_world.log 2>&1 || exit 100
nice make -j 8 TARGET=amd64 TARGET_ARCH=amd64 buildkernel > /var/log/build_amd64:13.2_kernel.log 2>&1 || exit 99
PKGLOG=/var/log/build_amd64:13.2_pkgs.log && echo >> $PKGLOG
make TARGET=amd64 TARGET_ARCH=amd64 packages > $PKGLOG 2>&1
ret=?
if [ $ret -eq 0 ]; then
  cd /usr/obj/usr/src/13.2/repo/FreeBSD:13:amd64 || exit 97
  ln -sfh 'ls -rtd 13.2* | tail -1' latest
else
  echo Packages not created. >>$PKGLOG
fi

I tested this procedure several times in my test VMs with no problem.
I noticed that my kernel log of the server looks a little bit different than in my VM
VM:
cat /var/log/build_amd64:13.2_kernel.log
Output:
--- kernel.full ---
linking kernel.full
ctfmerge -L VERSION -g -o kernel.full . . .
text data bss dec hex filename
23055782 1848005 4437760 29341547 0x1bfb76b kernel.full
--- kernel.debug ---
objcopy --only-keep-debug kernel.full kernel.debug
--- kernel ---
objcopy --strip-debug --add-gnu-debuglink=kernel.debug kernel.full kernel
-----------------------------------------------------------------------------
>>> Kernel build for GENERIC completed on ......
>>> Kernel(s) GERNERIC build .....

On the server I don't have the linking part.
Only
cat /var/log/build_amd64:13.2_kernel.log
Output:
Building /usr/obj/usr/src/13.2/amd64.amd64/sys/GENERIC/modules/usr/src/13.2/sys/modules/zfs/zfs.ko
--------------------------------------------------------------------------------------
>>> Kernel build for GENERIC completed on ......
>>> Kernel(s) MODULAR GERNERIC build .....


I still tried to upgrade it to 13.2-RELEASE-p9 and after I tried to login I get the message:

ld-elf.so.1: Shared object "libncurses.so.8" not found, required by "bash"
login:

So I assume that there is a problem with login with my user (uses /bin/bash as shell)
I recognized that during the upgrade I got a few errors.
So I rebooted the server and selected the old bootenvirement.

the log of /var/log/build_amd64:13.2_world.log and /var/log/build_amd64:13.2_packages.log looks fine.

Now I deleted the compilied files with rm -rf /usr/obj/usr/src/13.2* and run the BUILD132 script again and got the previous mentioned log /var/log/build_amd64:13.2_kernel.log.

Is there a good chance to fix this "Installation try" with the root acc (uses csh) or is there a better way?
I have a ssh connection to the server and I can get physical access to it but I would try to troubleshoot it via ssh for obvious reasons.
 
Upgrades should always be done as root. And a generic kernel is suggested as well. Read and use the handbook. Doing it another way than suggested is asking for uneeded trouble. You can start up into single user mode (s at the boot prompt) as recovery, or use "freebsd-update rollback". You need to log in as root. After that change the user shell. Bash is not part of the base system. So I suggest not using that for root anyways.
 
Thanks for the reply.

I upgraded the system with the root acc.
sudo su -

Just because this is my first major upgrade of the system I never logged in as a root. (I overtooked the system of another sysadmin)
Now I changed the password of the root account. (The other sysadmin forgot it. And I never changed it)

Is it reasonable to concider that I can fix the issue through:
via ssh:
sudo su -
bectl destroy 13.2-RELEASE
bect create 13.2-RELEASE
bectl mount 13.2-RELEASE /mnt
env ABI=FreeBSD:13:amd64 IGNORE_OSVERSION=yes pkg -r /mnt install -r LOCAL-base -g 'FreeBSD-*'
diff /etc/master.passwd /mnt/etc/master.passwd
diff /etc/group /mnt/etc/group
Then when I am site: (could be days in between)
login:root
password:....
bectl activate 13.2-RELEASE
sync;sync;reboot
login:root
password:....
pkg bootstrap -f -r FreeBSD
pkg update -f
pkg upgrade -f
sync;sync;reboot
 
I am not experienced in upgrading over network :). Single machine user here. But your problem with the first attempt seems to be bash. So if it is not a essential system you can try it out. But set the editor variable to something available in the base system.
You might need to run etcupdate as well between major version upgrades.
 
using bash (or any non-base shell) for root is a bad idea
If you're doing remote upgrades it's also a bad idea to use for your user account. Stick to a shell from the base, at least temporarily, during the upgrade. Once the upgrade is completed you can switch back.

The "problem" is that the base OS libraries get upgraded. Which causes all ports/packages to fail (well, almost all of them), that's why they all need to be reinstalled.
 
Back
Top