/etc/rc: Error Usage load load_rc_config name. Can't boot after upgrade to 11-RELEASE

[Hanky-panky]

Hi Tobik and thank you for your competent help.

Yep, I did try in the first the system debug option you suggest, then like you said it wasn't usefull becouse the system die before processing rc.conf.

The boot order of the files, no one here seems to know about this very important aspect is: rc, rc.subr, rc.conf, obviously each one of this file present and processed also in the defaults subfolder, exactly before in defaults and then in /etc folder.

Related with the /etc copy I did try everything, so both the useless and like you said dangerous blind copy and some cherry picked one, this one for now limited by the difficult to access the system when it hangs with still anything loaded.

I should phisically exyract the disk, connect it to a working machine considering usb subsystem wont work for me in this limited single user mode. Then, like we all here posting in this thread before you, we are not experts, so no one was able to highlight this boot order.

When I do have some little time, considering I still have the backup of the failing machines, I will cherry pick rc and rc.subr from a working 11 machine and copy it to a broken machine and I pretty much I'm sure it will work. I will also diff broken and working files to properly understand code changes.

I want to know whats went wrong and you offered me a GREAT hint: that snippet from rc.subr was extracted from a 10.3 machine upgraded I think from 8 to 10.3 in rolling mode during the years then you said it is a 9.3 one so probably here where is the problem: been not correctly upgraded by vintage software like freebsd-update.

So it is now clear that rc.subr or maybe rc too or rc and rc.subr went upgraded to 11 in a not compatible way with older OS versions.

In my case, considering I'm at 19 not correctly upgraded machines, that sort of mergemaster implemented in freebsd-update is unable to correctly upgrade those files.

Last night also my FreeBSD based router/NAS crashed badly at rc.subr like all the other machines.

Lucky I'm the backup type of guy, so it was just a matter of time to have it back and running.

For all the three production machines I'm up and running at 11 via source upgrade then I'm still really interested in see which file fail in buggy freebsd-update.


Ps: sorry for no formatting. I type on my beloved iphone.

 

[Hanky-panky]

When I do have some little time, considering I still have the backup of the failing machines, I will cherry pick rc and rc.subr from a working 11 machine and copy it to a broken machine and I pretty much I'm sure it will work. I will also diff broken and working files to properly understand code changes.

Finally FIXED by me, obviously NOT solved becouse the bug is still there. I will fill a PR when I do have some little time.

As I was wondering it was dumb freebsd-update messin with configuration files on NINETEEN in my case failed upgrades.

What I did? Simply booted the 11-0-RELEASE minimal install cd, mounted local file systems of the failed machines and simply copy the /etc/rc.subtr to local mounted file system /mnt/etc.

Rebooting the system was enough to... discover how many others mess dumbly perverted freebsd-update makes!

The system this time passed the fixed rc.subr then it doesn't still boot and this time complained iabout wrong errors in /etc/defaults/rc.conf.

Rebooted once again in single user mode, remounted file systems RW and edited with my fav ee editor(I always deeply hated stupid vi and found how very many mess of <<<<<<<< >>>>>>> that stupid "mergemaster like" did in that file. I personally and manually correctly edited the file, saved it and just typing exit was enough to boot the system.

Yes, the system booted fine then the debug option I do activated showed me how many (like over 10) messed config files I still had to edit to have a perfectly working system. Dumb "mergemaster like" tool in dumb freebsd-update makes so many mistakes even in another vital file like /etc/sysctl.

I hope with this definitive post to really help people facing this problem.

In my experience freebsd-update it seems only partially usefull updating between patches and minor upgrades, not surely to upgrade between two majour OS upgrades.

That's all and thankx to no one but myself and tobik..[/FILE]

 

[Terri_Kennedy]

As I was wondering it was dumb freebsd-update messin with configuration files on NINETEEN in my case failed upgrades.

I don't know if this post will be of any help to you because it seems your mind is already made up, but perhaps it will help someone else...

There's a famous quote about "doing the same thing over and over and expecting different results". I can understand why you might try upgrading a second or possibly even a third system to see if it has the problem, but actually rendering 19 systems non-bootable?

I understand the frustration of not having a part of the system perform the way you expect, and I've run into places where incorrect assumptions on the part of a committer led to unexpected results. The most recent one I can remember is that building sysutils/lsof on (at least) early FreeBSD 10.x releases would fail with an inexplicable error if the make.conf(5) file didn't exist. It could be empty, but it had to be there or you'd get the error. There's no requirement that a system have an /etc/make.conf file, a new install didn't provide one, and almost everything worked fine without it. But the developer(s) either made an assumption that the file always exists, or more likely didn't realize they'd introduced an unwanted dependency. If I run into a "it works for everybody else" scenario like that, my first thought is "what did I do wrong", not "gee, those developers screwed up". I asked the developers what I was doing wrong, and after they and I couldn't find anything, we eventually discovered the unexpected dependency.

I believe that one of the sticky posts in the guidelines area mentions that these forums are mostly for user-to-user interactions. While there are some developers on the forums, not all parts of the system are represented and it isn't possible to force developers to participate here [insert witty "herding cats" comment here]. If you're trying to get the attention of a developer, one of the mailing lists or a correctly-targeted PR (to let auto-assignment work) is probably your best bet.

Rebooting the system was enough to... discover how many others mess dumbly perverted freebsd-update makes!

I don't use freebsd-update(8), and there are other major parts of FreeBSD I don't use (for example, I don't use the installer). However, I did follow its development with interest and I hope I can speak cogently about it. Here goes:

freebsd-update's update "blobs" (for lack of a better term) are done by comparing a vanilla install of version A with a vanilla install of version B. If a file didn't change, freebsd-update doesn't provide a new copy. Likewise, it doesn't checksum a user's file to look for missing / incorrect files. It just provides the files needed to get from A to B. In some cases, that involves updates to a file that the user has likely edited, in which case it assists with merging the new version's file with the customized user file. It can't always determine what the user intended, so sometimes manual editing is necessary.

If you change part of the system that freebsd-update doesn't expect, then all bets are off. Most people who make those types of changes also build from source instead of using freebsd-update. If someone decides that /bin/sh should actually be a symlink to /bin/tcsh because all "proper" shell scripts should have a shebang so they always use the right shell, freebsd-update won't save you.

In my experience freebsd-update it seems only partially usefull updating between patches and minor upgrades, not surely to upgrade between two majour OS upgrades.

You probably know this, but the "-STABLE" in a FreeBSD version indicates that interfaces shouldn't be changed in an incompatible way during the life of the major version. That isn't always honored, but it is a definite goal. Creating an update utility that lets you cross major versions is quite an accomplishment (even if it didn't work for you). There are often compatibility shims left in place to ease migration from release A.something to B.something. They generally go away by the time release C.something comes along. You may be running into something like that if your systems were originally FreeBSD 9.x or older.

A case could be made that a desirable tool would be one that checked all the configuration files for the correct SVN revision and SHA1 hash for the "before" version, in order to prevent problems like the one you encountered. The fact that such a tool doesn't exist might be due to lack of interest in creating it, but it is more likely due to a lack of perceived need for it.

 

[kpa]

A case could be made that a desirable tool would be one that checked all the configuration files for the correct SVN revision and SHA1 hash for the "before" version, in order to prevent problems like the one you encountered. The fact that such a tool doesn't exist might be due to lack of interest in creating it, but it is more likely due to a lack of perceived need for it.

This is not realistic as long as the system allows you to skip many versions in one go, the tool would would potentially have to know all the preceding versions to the beginning of FreeBSD's history and also be able to decide if the files are modified which older version the modified version resembles most closely.

The real concern I have about freebsd-update(8) is that it and its documentation do a very poor job of explaining what the configuration merge process actually is and how it works. You do get it after doing it for a while but neither the manual page nor the handbook have an example run to show what it actully involves and why you should choose this way or that way when asked which option to choose.

On top of that we have two different merge tools that don't work the exact same way as already noted. OpenBSD manages the same configuration merge process with just one tool, why can't we?

 

[Terri_Kennedy]

This is not realistic as long as the system allows you to skip many versions in one go, the tool would would potentially have to know all the preceding versions to the beginning of FreeBSD's history and also be able to decide if the files are modified which older version the modified version resembles most closely.

The process that creates the freebsd-update "blobs" has information on every system file for each version that freebsd-update allows you to update from. Otherwise it couldn't generate the deltas for what changed. Also, freebsd-update doesn't support all releases back to the beginning of time - it is a relatively new utility.

I just looked at the freebsd-update(8) manpage, and found this:

IDS - Compare the system against a "known good" index of the installed release.

So it seems that something like my proposed tool has already been implemented.

The real concern I have about freebsd-update(8) is that it and its documentation do a very poor job of explaining what the configuration merge process actually is and how it works. You do get it after doing it for a while but neither the manual page nor the handbook have an example run to show what it actully involves and why you should choose this way or that way when asked which option to choose.

From a quick look at the documentation, I agree with you. But perhaps freebsd-update prints some useful information during the process - as I said, I've never used it.

On top of that we have two different merge tools that don't work the exact same way as already noted. OpenBSD manages the same configuration merge process with just one tool, why can't we?

Aside from the glib "there are always more than one way to do <thing> on a Unix-like system" response, FreeBSD provides one set of tools for users who prefer to use pre-built binaries (freebsd-update, pkg, etc.) and another for users who want to "roll their own" (SVN, make kernel, portupgrade, etc.). Both of those are still command-line-based and there are a number of projects based on FreeBSD (FreeNAS, PC-BSD, etc.) which add a graphical interface as a third method. While simply duplicating the functionality of one tool and calling it something else is a waste of everyone's time, I see the benefit of having multiple tools aimed at users with varying skillsets who have different goals. Aside from people who switched over to using freebsd-update once it became generally available because they didn't want to build everything from source for each update, it seems the two groups of users (freebsd-update vs. build-from-source) are pretty happy with the system they have, and there isn't a lot of switching back and forth. A single tool that tried to be a solution for both group would probably be seen as a poorer tool than the two tools FreeBSD has now. Again, this is just supposition on my part, as I'm in the build-from-source group, haven't used freebsd-update, and don't know anyone who has.

 

[gkontos]

freebsd-update is a very powerful tool but it assumes that your system files have not been altered by you. If it notices a difference it will create a:

<<<<<<<<<<<<<<<<
{some code}
----
{new code}
>>>>>>>>>>>>>>>

You have the ability to edit the file in respect. I have used it many times and it is a life saver if you have to upgrade 20 machines. I am using FreeBSD since 6.0-RELEASE and it has only been a year that I am using binary upgrades.

Generally speaking, even if you upgrade from source, you SHOULD run a tool like IPMI or VNC, depending on the service that you have. IMHO mergemaster is more difficult to run for a newbie.