Local rc.d scripts in /usr/local/etc/rc.d not callable from "service"-command

[nimo]

Hi,

I just updated a jail from 13.1 to 14.0. My problem now is that services installed from packages that puts rc.d-scripts in /usr/local/etc/rc.d is not callable from the service-command. However, they work if being called directly like this:
/usr/local/etc/rc.d/tomcat9 start


# service tomcat9 status
tomcat9 does not exist in /etc/rc.d or the local startup
directories (YES), or is not executable

It is enabled in /etc/rc.conf

And config looks good:

# grep -ri "/usr/local/etc/rc.d" /etc/*
/etc/defaults/rc.conf:local_startup="/usr/local/etc/rc.d" # startup script dirs.


What could be wrong?

 

[cy@]

grep local_startup /etc/rc.conf

 

[SirDice]

# service tomcat9 status
tomcat9 does not exist in /etc/rc.d or the local startup
directories (YES), or is not executable

The error message is odd, it prints "YES" when it should be printing /usr/local/etc/rc.d/ there.

# If the script was not found
echo "$script does not exist in /etc/rc.d or the local startup"
echo "directories (${local_startup}), or is not executable"
exit 1

 

[nimo]

The error message is odd, it prints "YES" when it should be printing /usr/local/etc/rc.d/ there.

I had a typo in rc.conf (version of tomcat had changed from 85 to 9 tomcat85_enable="YES"). This is now fixed. But the problem still remains but the error message is now:

# service tomcat9 start
tomcat9 does not exist in /etc/rc.d or the local startup
directories (/etc/rc.d), or is not executable

grep local_startup /etc/rc.conf

No hits:

# grep local_startup /etc/rc.conf
#

 

[SirDice]

# service tomcat9 start 
tomcat9 does not exist in /etc/rc.d or the local startup 
directories (/etc/rc.d), or is not executable

Some progress I guess. Looks like $_localbase is empty:

local_startup="${_localbase}/etc/rc.d" # startup script dirs.

That gets set at the top:

# Set default value of _localbase if not previously set
: ${_localbase:="/usr/local"}

So, you have defined _localbase somewhere, with an empty string in it.

Can you post the whole /etc/rc.conf?

 

[nimo]

Can you post the whole /etc/rc.conf?


hostname="subsonic"

cron_flags=" -J 15"

subsonic_enable="YES"
subsonic_http_port="4040"
subsonic_https_port="4043"
subsonic_port="4040"
subsonic_context_path="/subsonic"
subsonic_ssl="NO"
subsonic_music_folder=/zstorage/media
subsonic_max_memory=512
#jetty_flags="-Dfile.encoding=UTF-8 -Duser.language=en -Duser.region=US -Xms96m -Xmx512m -Djava.awt.headless=true"



# Disable Sendmail by default
sendmail_enable="NONE"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Run secure syslog
syslogd_flags="-c -ss"

# Enable IPv6
ipv6_activate_all_interfaces="NO"

sshd_enable="YES"
tomcat9_enable="YES"
tomcat9_env="LANG=en_US.UTF-8 TZ=Europe/Stockholm"


So, you have defined _localbase somewhere, with an empty string in it.

Thanks a ton! This pointed me in the right direction! Turned out that I must have screwed up /etc/defaults/rc.conf when freebsd-update ran merge on /etc. I fixed it by copying /etc/default/* from another system of the same FreeBSD-version.

 

[cy@]

Never edit /etc/defaults/rc.conf. If you think freebsd-update managed to hose it you might want to open a PR.

 

[nimo]

Never edit /etc/defaults/rc.conf. If you think freebsd-update managed to hose it you might want to open a PR.

So I upgraded an iocage-jail from 13.1->14.0 -- and for some reason freebsd-upgrade had me to manually merge all (or at least most) files in /etc. Mostly it was that it couldn't merge the header of the files or updated commented lines. But for some files there were actually changes but as it was so many files there is a high likelihood that I somewhere lost my sharpness and made mistakes when it came to which files I kept and which I removed. For /etc/defaults/rc.conf there were none of my changes in it from the beginning -- it was just more default lines added that freebsd-update couldn't handle automatically it seems -- and then when I had ot maually merge it I obviously screwed up.

But I am not sure how I can reproduce this potential bug when it comes to freebsd-update not being able to automatically handle uncommented lines? Would it be worth to open a PR also then?

Then I have a general question about updating -- if I understand the handbook correctly then etcupdate is only used when updating from source -- is there any way to use it also when not updating from source? The reason I am askin is that 10 years ago when I were running Gentoo Linux I remember that over there it as possible to configure it in a way that it would automatically update all non-edited /etc-files. What is the best practice for doing this in jails in FreeBSD?

 

[cy@]

So I upgraded an iocage-jail from 13.1->14.0 -- and for some reason freebsd-upgrade had me to manually merge all (or at least most) files in /etc. Mostly it was that it couldn't merge the header of the files or updated commented lines. But for some files there were actually changes but as it was so many files there is a high likelihood that I somewhere lost my sharpness and made mistakes when it came to which files I kept and which I removed. For /etc/defaults/rc.conf there were none of my changes in it from the beginning -- it was just more default lines added that freebsd-update couldn't handle automatically it seems -- and then when I had ot maually merge it I obviously screwed up.

I recall having to do a three-way merge (that's what this process is called in CS terms) once or twice during freebsd-update at $JOB.

But I am not sure how I can reproduce this potential bug when it comes to freebsd-update not being able to automatically handle uncommented lines? Would it be worth to open a PR also then?

Then I have a general question about updating -- if I understand the handbook correctly then etcupdate is only used when updating from source -- is there any way to use it also when not updating from source? The reason I am askin is that 10 years ago when I were running Gentoo Linux I remember that over there it as possible to configure it in a way that it would automatically update all non-edited /etc-files. What is the best practice for doing this in jails in FreeBSD?

Non-edited /etc files are automatically updated by freebsd-update and etcupdate. See my comment about Red Hat below.

Yes, etcupdate is only to be used when updating from source. Back in the day there was mergemaster. It too used the three-way merge algorithm.

The other approach is the one used by Red Hat. They will attempt to replace the file, saving the original as a .RPMSAVE file. Then it's up to you to make sense of the differences. I don't know if they use a three-way merge.

You can play around with three-way diffs using the diff3(1) command. Three-way merge is an industry standard. Subversion, GIT, hg, etcupdate, back in the old days mergemaster, and other tools use three-way merge. The results are "<<<<", "====", and ">>>>" lines. If there are few you're fine. But if there are a ton of them in a file, going through all the "<<<<", "====", and ">>>>" can become confusing after a while. When I become discombobulated I quit and start over again. It's too easy with complex differences to make a mistake.

Before we had Subversion, when we used CVS, we had to do merges by hand. Before mergemaster(1), the precessor to etcupdate(1), sysadmins had to keep track of altered files in /etc and merge them by hand. This was true of the varioius Linuxen at the time too. We've come a long way.

 

[nimo]

The results are "<<<<", "====", and ">>>>" lines. If there are few you're fine. But if there are a ton of them in a file, going through all the "<<<<", "====", and ">>>>" can become confusing after a while.

Yes! This is what new takes 2 hours to do per jail that I am updating! And 95% of this is non-edited files. Like stuff in /etc/rc.d or similar that I have never touched!

Non-edited /etc files are automatically updated by freebsd-update and etcupdate

So I wonder why that is not the case for me? Might it be that I have waited too many release before doing upgrade?

 

[T-Aoki]

And 95% of this is non-edited files.

It could happen when only version strings/copyright lines are changed.
Or previous tree is not properly created.

Might it be that I have waited too many release before doing upgrade?

Possibly.

Basically, updates from the same major version (i.e., 13.2 to 13.3) or upgrades from the latest point releases [*.0, *.1,...] of previous major version to next major version at the moment the target (to be upgraded to) version is released (i.e., 13.4 to 14.2) is supported.

So, for example, upgrading from 12.0 to 14.2 is not supported, thus, there are possibilities to fail.

On the other hand, version strings and copyrights are usually almost at the top of files, and this easily causes mis-merges. This is because (usually) 3 or more matching lines are wanted to be existing before modified lines to be sure to automated merges. (What I recall the same reason recently is changes of root shell in /etc/master.passwd.)

 

[cy@]

Yes! This is what new takes 2 hours to do per jail that I am updating! And 95% of this is non-edited files. Like stuff in /etc/rc.d or similar that I have never touched!

Here is what I do. I have a baseline chroot which is updated regularly (using src buildworld/installworld). The baseline chroot, in /jails/template/root is nullfs mounted to my various jails (/jails/*/root). Any directory tree, i.e. /etc, /var,..., are unionfs mounted on top of the nullfs mount. This allows me to update /jails/template/root, virtually updating all nullfs mounted /jails/*/root jails without touching them because the underlying base filesystem is updated. One update updates all of them. Problem solved.

So I wonder why that is not the case for me? Might it be that I have waited too many release before doing upgrade?

Most likely.