Solved very odd freebsd-update upgrade interaction with mounted slice

BSDfanNo2

BSDfanNo2

I am running freebsd-update upgrade to upgrade from 11.0-BETA4 to 11.0-RC1. It is taking forever (as compared with previous times that I upgraded using this method). It seems to be attempting to do something with files in another slice in another disk where I have Fedora (which is an ext4 slice mounted readonly as /mnt/fedora in my FreeBSD system).

I have used Linux for years but I am a FreeBSD beginner. What is going on here? Why is freebsd-update concerning itself with /mnt/fedora as that has nothing to do with the base system? I have probably a million or more files in Fedora that is why this is taking so long, there are thousands upon thousands of lines like the one below scrolling up on my screen and it takes forever!

Code: 
cp: ///boot/kernel.old/./mnt/fedora/home/xxxxx/devel/beaglebone/buildroot-2013.02-Buildroot_toolchain/output/build/host-binutils-2.21.1/ld/scripttempl/elfd30v.sc: Cross-device link
 
BSDfanNo2, if the error message is accurate, it appears to be looking at
/boot/kernel.old/mnt/…, which shouldn't exist in a normal system. Do you perhaps have a rogue symlink there which should not exist and should be deleted?

As a general bit of advice, I would suggest avoiding having "foreign" filesystems mounted while doing upgrades. It shouldn't cause an ugly failure normally, but it's usually just unnecessary risk and clutter for the upgrade process.
 
I used this command to mount the Fedora partition, no rogue sym link:

mount -r -t ext2fs /dev/ada0s3 /mnt/fedora

I thought that (per the Handbook) /mnt was the recommended folder for mounting? I will keept in mind your suggestion to unmount file systems before running freebsd-update!

In any case inadvertently I pressed ^C and stopped the update process, which appeared to be stuck somewhere else as scrolling had stopped for a long time (even though htop showed high CPU utilisation). The last line was this:

cp: ///boot/kernel.old/./proc/0/cmdline: Cross-device link

Freebsd-update recreated in /boot/kernel.old/mnt/fedora the whole directory structure of the Fedora partition (/boot, /dev, /etc. /proc, /sys, etc) and all the folders are empty!
 
Yeah, mounting stuff at /mnt/thing should normally be ok.

BSDfanNo2 said:
Freebsd-update recreated in /boot/kernel.old/mnt/fedora the whole directory structure of the Fedora partition (/boot, /dev, /etc. /proc, /sys, etc) and all the folders are empty!
Click to expand...

If that's really what happened, it seems really quite bizarre. My best guess (and this is really a shot in the dark) is that there's an inappropriate symlink somewhere inside /boot. Running find /boot -type l -ls is worth a try. On a standard 10.3 install, there are no symlinks anywhere inside that directory. Is there anything unusual or non-standard about your /boot (perhaps related to multi-booting the system)?
 
Looking through the scripts I think I know what's happening, just don't exactly know why.

This is the bit of code that creates a directory structure:
Code: 
2720 	        mkdir -p $BASEDIR/$BACKUPKERNELDIR
2721 	        mtree -cdn -p "${BASEDIR}/${KERNELDIR}" | \
2722 	            mtree -Ue -p "${BASEDIR}/${BACKUPKERNELDIR}" > /dev/null

The copying happens a bit further on:
Code: 
2742 	        # Backup all the kernel files using hardlinks.
2743 	        (cd ${BASEDIR}/${KERNELDIR} && find . -type f $FINDFILTER -exec \
2744 	            cp -pl '{}' ${BASEDIR}/${BACKUPKERNELDIR}/'{}' \;)
2745

If ${KERNELDIR} is empty (for whatever reason), you'd see something similar to what you have. Which begs the question, why is ${KERNELDIR} empty?

What does sysctl kern.bootfile output?
 
Murph said:
Yeah, mounting stuff at /mnt/thing should normally be ok.

If that's really what happened, it seems really quite bizarre. My best guess (and this is really a shot in the dark) is that there's an inappropriate symlink somewhere inside /boot. Running find /boot -type l -ls is worth a try. On a standard 10.3 install, there are no symlinks anywhere inside that directory. Is there anything unusual or non-standard about your /boot (perhaps related to multi-booting the system)?
Click to expand...

I ran find /boot -type l -ls and it found nothing (though it took a few minutes to find nothing on an old PC!).

I have multi-boot but I use grub within Ubuntu (I also have installed Fedora, FreeBSD and NetBSD). The only change I made with respect to boot in the FreeBSD slice was to reduce the boot wait timeout from 10 to 1 seconds.
 
SirDice said:
Looking through the scripts I think I know what's happening, just don't exactly know why.

This is the bit of code that creates a directory structure:
Code: 
2720             mkdir -p $BASEDIR/$BACKUPKERNELDIR
2721             mtree -cdn -p "${BASEDIR}/${KERNELDIR}" | \
2722                 mtree -Ue -p "${BASEDIR}/${BACKUPKERNELDIR}" > /dev/null

The copying happens a bit further on:
Code: 
2742             # Backup all the kernel files using hardlinks.
2743             (cd ${BASEDIR}/${KERNELDIR} && find . -type f $FINDFILTER -exec \
2744                 cp -pl '{}' ${BASEDIR}/${BACKUPKERNELDIR}/'{}' \;)
2745

If ${KERNELDIR} is empty (for whatever reason), you'd see something similar to what you have. Which begs the question, why is ${KERNELDIR} empty?

What does sysctl kern.bootfile output?
Click to expand...
Code: 
# sysctl kern.bootfile
kern.bootfile: /kernel
Funny thing: I have /boot/kernel but no /kernel. In this slice I had previously installed 10.3 and last week I upgraded with freebsd-update to 11.0-BETA4 (or 3), and the update went well.
 
BSDfanNo2 said:
Code: 
# sysctl kern.bootfile
kern.bootfile: /kernel
Funny thing: I have /boot/kernel but no /kernel. In this slice I had previously installed 10.3 and last week I upgraded with freebsd-update to 11.0-BETA4 (or 3), and the update went well.
Click to expand...

That seems likely to be a step closer to the root cause of this problem. A normal standard install should give this:
Code: 
# sysctl kern.bootfile
kern.bootfile: /boot/kernel/kernel
So, now the question is why is it not giving the full path to the kernel? I don't have any immediate suggestions to explain that.
 
BSDfanNo2 said:
Code: 
# sysctl kern.bootfile
kern.bootfile: /kernel
Click to expand...
Ok, at least we identified why. Your kern.bootfile causes this:
Code: 
        BOOTFILE=`sysctl -n kern.bootfile`
        KERNELDIR=${BOOTFILE%/kernel}
        if ! [ -d ${KERNELDIR} ]; then
                echo "Cannot identify running kernel"
                exit 1
        fi
So ${KERNELDIR} ends up with / when it should be /boot/kernel.

Funny thing: I have /boot/kernel but no /kernel. In this slice I had previously installed 10.3 and last week I upgraded with freebsd-update to 11.0-BETA4 (or 3), and the update went well.
Click to expand...
That makes it even stranger. If you look at the code it does a check to see if ${KERNELDIR} is a directory, if it isn't it exits the script. So in your case it should simply exit with "Cannot identify running kernel".
 
I figured out why kern.bootfile is /kernel instead of /boot/kernel/kernel. I am using Grub installed in a Lubuntu partition to boot. Grub with the help of os-probe is able to find Fedora and Lubuntu, but for NetBSD and FreeBSD. I manually created a file 40_custom (see below) in order to boot the latter two. I can boot FreeBSD in three different ways, and for the first two kern.bootfile is /boot/kernel/kernel, and for the last one (boot directly to kernel bypassing the FreeBSD boot loaders) kern.bootfile is /kernel. So that is the explanation.

I did not invent the commands to boot, I got them from from some sites on the net. I have booted FreeBSD directly to kernel many times without any problem - until yesterday when freebsd-update choked on it. So the question arises: is this is a freebsd-update bug or just my Grub configuration that is wrong. Any ideas?

Code: 
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.

# /etc/grub.d/40_custom

# NetBSD
menuentry "NetBSD 7.0.1" {
insmod ufs2
set root=(hd0,4)
knetbsd /netbsd --root=wd0a
}

# FreeBSD - Chainload the embedded boot record
# Putting (hd1) in the chainloader command instead of root did not work.
menuentry "FreeBSD 11.0 - via chainloader to boot record"{
insmod part_gpt
set root=(hd1)
chainloader +1
}

# FreeBSD - Use the BSD 1st stage loader
# This does not work for some unknown reason
#menuentry "FreeBSD 11.0 - via 1st stage boot loader"{
#insmod part_gpt
#set root=(hd1,1)
#chainloader +1
#}

#  FreeBSD - Use the BSD 2nd stage loader
menuentry "FreeBSD 11.0 - via 2nd stage boot loader"{
insmod part_gpt
insmod ufs2
set root=(hd1,3)
kfreebsd /boot/loader
}

#  FreeBSD - Load the FreeBSD kernel directly
menuentry "FreeBSD 11.0 - directly to kernel" {
insmod part_gpt
insmod ufs2
set root=(hd1,3)
kfreebsd /boot/kernel/kernel
kfreebsd_loadenv /boot/device.hints
set kFreeBSD.vfs.root.mountfrom=ufs:/dev/ada1p3
set kFreeBSD.vfs.root.mountfrom.options=rw
}
 
Although I still do not understand the precise reason why booting directly to kernel causes kern.bootfile to have the wrong value, I do not need to know it, so I did the following:
  • commented out in my Grub configuration the option to boot directly to kernel;
  • deleted /boot/kernel.old as it was full of junk;
  • renamed /usr/src as /usr/srcsvn since freebsd-update wanted to update it;
  • created an empty /usr/src;
  • ran freebsd-update upgrade -r 11.0-RC1 ...
  • ... and all went well.

Thanks SirDice and Murph for your help, and I will mark this issue as solved.
 
Ah, I just realised that the following was not a good idea, so I reverted that change and instead will modify /etc/freebsd-update.conf to disable source update.

BSDfanNo2 said:
  • renamed /usr/src as /usr/srcsvn since freebsd-update wanted to update it;
  • created an empty /usr/src;
Click to expand...
 
You must log in or register to reply here.
Back
Top