Solved Infinite segfault preventing boot after 10.0->10.3 upgrade

L

listentoreason

I'm in a bit of a tight spot here. A month ago I posted an issue where pkg upgrade left multiple packages non-functional. It was pointed out that 10.0 was end-of-life, so I waited until I had a quiet weekend to do a full upgrade. I had an opportunity today, so I attempted to simply:

Code: 
freebsd-update -r 11.0-RELEASE upgrade
... but that complained with:

Code: 
The update metadata is correctly signed, but failed an integrity check.
Cowardly refusing to proceed any further.
Finding similar problems on the forum, I found instructions that I should first upgrade 10 fully, then proceed to 11. So I ran:

Code: 
freebsd-update fetch install
freebsd-update upgrade -r 10.3-RELEASE
freebsd-update install
shutdown -r now
freebsd-update install
Everything seemed (?) to go without incident, until I rebooted and attempted the post-reboot install; The shell announced that it was "Installing updates" and then immediately went into an apparently infinite loop of segmentation faults. I couldn't Ctrl-C out, so after watching a page or two of segfaults go by I did a hard reset of the server. I am now in a state where I can't even get to a shell; I get:

Code: 
pid 19 (sh), uid 0: exited on signal 11
init: single user shell terminated
Enter full pathname of shell or Return for /bin/sh
Attempts to proceed at this point repeat the above koan over and over, simply incrementing the pid counter by one each time.
Very eager for any advice on how I can recover the system! I'm happy to provide any additional information, but to be honest I'm not sure I can if I can't even get a boot shell.
Thanks,
CAT
 
I would suggest to boot from a live media (CD / DVD / USB) and:

a) fsck your partitions (or scrub if ZFS)

b) backup your data

c) evaluate the situation:

you could possibly reinstall the kernel and world ... or else reinstall the system completely, if it was me I would choose the fastest path... but depend on you and your system.
 
ASX said:
I would suggest to boot from a live media (CD / DVD / USB) and:

a) fsck your partitions (or scrub if ZFS)

b) backup your data

c) evaluate the situation:

you could possibly reinstall the kernel and world ... or else reinstall the system completely, if it was me I would choose the fastest path... but depend on you and your system.
Click to expand...

Thank you, I was presuming that a live CD would be involved. Do you have any favored resources (links) that would detail this step-by-step? I'm fairly nervous at this point and from past experience I know that web searches on FreeBSD approaches can lead to outdated/deprecated instructions.

A lot of my panic has to do with my zpool (the reason for the server existing) being geli encrypted, and concerns that I won't be able to restore the pool if I mess up the remount. In particular, I never finished labeling my drives, so I'm terrified that an attempt to reconstitue the pool will fail to properly associate /var/backups/gpt_GELI-da[1-11].eli with the appropriate devices.

EDIT (for posterity): This was a silly fear. The geli metadata are stored with each drive, see post here. Also, even if it wasn't, and I had to associate each drive with a backed-up metadata file, it's not an 11-factorial problem like I thought, since I don't have to mount all 11 drives simultaneously; I could have tried them each individually against the 11 files, which is at worst a sum(1:11) problem (66 trial-and-error associations). I got this into my head a couple years ago and didn't think about it carefully. Nutshell is that geli drives are self-contained and should be recoverable unless something horrible happens to the last block *and* you lose the backup .eli file.
 
Last edited:
Also: I'm presuming that I am going to ultimately need to reinstall the system from CD/DVD. If/when that happens, how much of the file system can I "keep in place" to minimize disruption? In particular, folders like /etc/. It's been a very long time since I've done an install from media, I can't recall if there are options to keep existing configuration files in place.

Thanks,
CAT
 
The official FreeBSD ISOs/IMGs all allow to start a shell after booting ...

Considering your worries, you could use dd to backup your raw disk(s) before doing anything else; that is you will be able to restore the whole disk(s), if necessary.... but that is going to take lot of storage and time.
 
listentoreason said:
If/when that happens, how much of the file system can I "keep in place" to minimize disruption?
Click to expand...

Again, that would be me:
write down a list of all installed/used software and the corresponding setting locations, and backup them all, in any case backup the whole /etc and may be /usr/local/etc, or even backup the whole root filesystem ...
 
Ok, I'm back up and running, thanks for the help. I managed to forget to backup /usr/local so I may be reinventing the wheel at some point in the future, but my system is now happily running 11.0-RELEASE, and the zpool is up and went through a scrub with no issues. Here's a cross-link to my thread in the storage subforum; As mentioned there, I'll try to make a retrospective post about what I did in case it's of use to others. Thanks again for your help!
 
You must log in or register to reply here.
Back
Top