/etc/rc.d/cron does not start at boot

[scrappywan]

I recently installed FreeBSD 11.1 on my desktop and wanted to use zfs-tools to create periodic snapshots. Everything related to zfs-tools including my crontab file is setup exactly as another FreeBSD server I run. The only issue on my desktop is that cron does not start, or fails to start during the boot process. If I run # /etc/rc.d/cron start then zfs-tools starts taking snapshots as per specified cron times. I am not seeing anything in mail, and AFAIK, /var/log/cron doesn't seem to show anything out of the ordinary.

 

[ShelLuser]

Have you by any chance made changes in /etc/defaults/rc.conf? Because that could explain something I guess. You might want to take a look at /var/log/messages and/or /var/log/cron to see if those mention anything specific.

Also; it's usually better to use the service command: # service cron start. If that somehow complaints instead of (re)starting then you'll know that the problem lies with your configuration.

 

[scrappywan]

Have you by any chance made changes in /etc/defaults/rc.conf? Because that could explain something I guess.

No, I haven't changed anything there. Here is the output of /etc/defaults/rc.conf where cron is concerned:

cron_enable="YES"       # Run the periodic job daemon.
cron_program="/usr/sbin/cron"   # Which cron executable to run (if enabled).
cron_dst="YES"          # Handle DST transitions intelligently (YES/NO)
cron_flags=""           # Which options to pass to the cron daemon.

After a reboot a few minutes ago, I started cron via # service cron start and it started without any error messages.

You might want to take a look at /var/log/messages and/or /var/log/cron to see if those mention anything specific.

Ok, found a few issues which showed up in /var/log/messages:

Mar  1 07:43:13 bloaty pflogd[60313]: [priv]: failed to open /var/log/pflog: Is a directory
Mar  1 07:44:13 bloaty pflogd[60313]: [priv]: failed to open /var/log/pflog: Is a directory
Mar  1 07:46:14 bloaty last message repeated 2 times
Mar  1 07:52:17 bloaty last message repeated 6 times
Mar  1 07:52:24 bloaty mark: /etc/rc.d/cron: WARNING: failed to start cron
Mar  1 07:53:18 bloaty pflogd[60313]: [priv]: failed to open /var/log/pflog: Is a directory
Mar  1 07:54:18 bloaty pflogd[60313]: [priv]: failed to open /var/log/pflog: Is a directory

Well, my bad creating that pflog file as a directory! Fixed that. After reboot I still don't have a working cron service, however, the cron warning has not shown itself since in /var/log/messages after rebooting. I still had to start cron service manually.

Here are the most recent lines from /var/log/cron:

Mar  1 07:40:00 bloaty /usr/sbin/cron[63337]: (root) CMD (/usr/libexec/atrun)
Mar  1 07:55:00 bloaty /usr/sbin/cron[90076]: (operator) CMD (/usr/libexec/save-entropy)
Mar  1 07:55:00 bloaty /usr/sbin/cron[94291]: (root) CMD (/usr/libexec/atrun)
Mar  1 08:05:00 bloaty /usr/sbin/cron[60129]: (root) CMD (/usr/libexec/atrun)
Mar  1 08:10:00 bloaty /usr/sbin/cron[48776]: (root) CMD (/usr/libexec/atrun)
Mar  1 08:11:00 bloaty /usr/sbin/cron[25184]: (operator) CMD (/usr/libexec/save-entropy)
Mar  1 08:15:00 bloaty /usr/sbin/cron[98106]: (root) CMD (/usr/libexec/atrun)
Mar  1 08:15:00 bloaty /usr/sbin/cron[98112]: (root) CMD (/usr/local/sbin/zfs-auto-snapshot frequent 4 )

 

[ShelLuser]

Interesting issue.

First the most obvious: how did you install your local crontab changes? If you started messing around with /etc/crontab then this could be related (though unlikely). The best practice for local additions is to use a users crontab. Use root for system processes, and other ID's for their related tasks.

So, for example, my DNS server grabs named.root from InterNIC every once in a while (once per month) and the process is run as the UID of my DNS server (pdns). So if I have to change this I'd use # crontab -u pdns -e. The other advantage is that all your changes are located in /var/cron/tabs so it's easy to keep an overview.

Next option..

Cron depends on a few other things to be present. From an RC perspective this is LOGIN and FILESYSTEMS. In other words: your filesystems need to be mounted (/etc/rc.d/FILESYSTEMS) and user services need to be started (/etc/rc.d/LOGIN). So I wonder if it's possible that something like this prevents cron from starting during the boot phase.

I trust that your base system is installed onto a regular (single) ZFS pool and doesn't use any external filesystems?

On that subject: what does sysrc zfs_enable tell you?

Finally, this is a useless hunch (just trying to rule out options): ldd `which cron`, what does that show you?

 

[Eric A. Borisch]

Try rc_debug="YES" in rc.conf and reboot to see if anything useful pops up?

 

[scrappywan]

After disabling services in /etc/rc.conf I found that pcdm display manager was the cause of cron not starting.

Try rc_debug="YES" in rc.conf and reboot to see if anything useful pops up?

Good idea. Hopefully it can help explain why pcdm is affecting cron? I'll give it a try with pcdm enabled to see the output.

@ShelLuser: Thanks for giving me some good options and possibilities to look at. I appreciate the time you spent trying to help me troubleshoot this issue. To answer your previous questions, I was editing cron with # crontab -e and zfs_enable=YES

 

[scrappywan]

Try rc_debug="YES" in rc.conf and reboot to see if anything useful pops up?

As requested, this is the output for command # service cron status after enabling rc_debug="YES" then re-enabling pcdm_enable="YES" after a reboot:

root@bloaty: # service cron status
/etc/rc.d/cron: DEBUG: checkyesno: cron_dst is set to YES.
/etc/rc.d/cron: DEBUG: pid file (/var/run/cron.pid): not readable.
/etc/rc.d/cron: DEBUG: checkyesno: cron_enable is set to YES.
cron is not running.

While I have no problem manually starting X after boot, I am sort of curious why pcdm is causing cron to not start.

 

[ShelLuser]

The plot thickens, it's a bit weird for /var/run/cron.pid not to be available after booting. I also wonder if pcdm isn't merely a symptom here and not so much the cause. Because one service shouldn't directly affect the other.

Do you have /var or /var/run stored on a separate location by any chance? What does mount tell you?

 

[Eric A. Borisch]

As requested, this is the output for command # service cron status after enabling rc_debug="YES" then re-enabling pcdm_enable="YES" after a reboot:

root@bloaty: # service cron status
/etc/rc.d/cron: DEBUG: checkyesno: cron_dst is set to YES.
/etc/rc.d/cron: DEBUG: pid file (/var/run/cron.pid): not readable.
/etc/rc.d/cron: DEBUG: checkyesno: cron_enable is set to YES.
cron is not running.

While I have no problem manually starting X after boot, I am sort of curious why pcdm is causing cron to not start.

Is there anything in the boot up messages -- rather than running service cron status after boot? grep rc.d/cron /var/log/messages

 

[scrappywan]

The plot thickens, it's a bit weird for /var/run/cron.pid not to be available after booting. I also wonder if pcdm isn't merely a symptom here and not so much the cause. Because one service shouldn't directly affect the other.

Do you have /var or /var/run stored on a separate location by any chance? What does mount tell you?

No. I never set up alternate mounts. The install was just your run-of-the-mill FreeBSD on zroot installation. During install, I did checkmark hide system processes from other users/groups plus most of the other security options, but I wouldn't think it would matter here since cron runs as root.

 

[scrappywan]

Is there anything in the boot up messages -- rather than running service cron status after boot? grep rc.d/cron /var/log/messages

Here is the output from messages:

root@bloaty:/var/run # grep rc.d/cron /var/log/messages
Mar  1 20:17:54 bloaty root: /etc/rc.shutdown: DEBUG: run_rc_script /etc/rc.d/cron faststop
Mar  1 20:19:16 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_dst is set to YES.
Mar  1 20:19:16 bloaty mark: /etc/rc.d/cron: DEBUG: pid file (/var/run/cron.pid): not readable.
Mar  1 20:19:16 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_enable is set to YES.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_dst is set to YES.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: pid file (/var/run/cron.pid): not readable.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_enable is set to YES.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: run_rc_command: doit:  limits -C daemon /usr/sbin/cron  -s

 

[Eric A. Borisch]

Output of service -e with and without pcdm enabled?

 

[scrappywan]

Here is the output from messages:

root@bloaty:/var/run # grep rc.d/cron /var/log/messages
Mar  1 20:17:54 bloaty root: /etc/rc.shutdown: DEBUG: run_rc_script /etc/rc.d/cron faststop
Mar  1 20:19:16 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_dst is set to YES.
Mar  1 20:19:16 bloaty mark: /etc/rc.d/cron: DEBUG: pid file (/var/run/cron.pid): not readable.
Mar  1 20:19:16 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_enable is set to YES.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_dst is set to YES.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: pid file (/var/run/cron.pid): not readable.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_enable is set to YES.
Mar  1 20:26:40 bloaty mark: /etc/rc.d/cron: DEBUG: run_rc_command: doit:  limits -C daemon /usr/sbin/cron  -s

After another reboot I no longer receive the "pid file (/var/run/cron.pid): not readable." message. I did perform a forced package re-installation before the last reboot. It wasn't related to the cron issue, but maybe something changed?

Below is the only thing that showed up after the most recent boot:

Mar  1 22:27:25 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_dst is set to YES.
Mar  1 22:27:25 bloaty mark: /etc/rc.d/cron: DEBUG: checkyesno: cron_enable is set to YES.

Edit: Re-enabling pcdm brought back "pid file (/var/run/cron.pid): not readable." again

 

[scrappywan]

Output of service -e with and without pcdm enabled?

pcdm disabled:

root@bloaty:~ # service -e
/etc/rc.d/hostid
/etc/rc.d/zvol
/etc/rc.d/hostid_save
/etc/rc.d/zfsbe
/etc/rc.d/zfs
/etc/rc.d/cleanvar
/etc/rc.d/ip6addrctl
/etc/rc.d/netif
/etc/rc.d/devd
/etc/rc.d/pflog
/etc/rc.d/pf
/etc/rc.d/newsyslog
/etc/rc.d/syslogd
/etc/rc.d/savecore
/etc/rc.d/dmesg
/etc/rc.d/virecover
/etc/rc.d/ntpd
/usr/local/etc/rc.d/dbus
/etc/rc.d/motd
/etc/rc.d/powerd
/etc/rc.d/rctl
/etc/rc.d/sshd
/usr/local/etc/rc.d/iocage
/etc/rc.d/moused
/etc/rc.d/sendmail
/etc/rc.d/cron
/etc/rc.d/mixer
/etc/rc.d/gptboot
/etc/rc.d/bgfsck
/etc/rc.d/autounmountd
/etc/rc.d/automountd
/etc/rc.d/automount

After pcdm enabled:

root@bloaty:~ # service -e
/etc/rc.d/hostid
/etc/rc.d/zvol
/etc/rc.d/hostid_save
/etc/rc.d/zfsbe
/etc/rc.d/zfs
/etc/rc.d/cleanvar
/etc/rc.d/ip6addrctl
/etc/rc.d/netif
/etc/rc.d/devd
/etc/rc.d/pflog
/etc/rc.d/pf
/etc/rc.d/newsyslog
/etc/rc.d/syslogd
/etc/rc.d/savecore
/etc/rc.d/dmesg
/etc/rc.d/virecover
/etc/rc.d/ntpd
/usr/local/etc/rc.d/dbus
/etc/rc.d/motd
/etc/rc.d/powerd
/etc/rc.d/rctl
/usr/local/etc/rc.d/pcdm
/etc/rc.d/sshd
/usr/local/etc/rc.d/iocage
/etc/rc.d/moused
/etc/rc.d/sendmail
/etc/rc.d/cron
/etc/rc.d/mixer
/etc/rc.d/gptboot
/etc/rc.d/bgfsck
/etc/rc.d/autounmountd
/etc/rc.d/automountd
/etc/rc.d/automount

 

[ShelLuser]

Almost sounds like a bug in x11/pcdm. I'll see if I can set up a quick test using a virtual machine later on.

 

[Eric A. Borisch]

I think you may want to take this over to a TrueOS forum or pcdm's GitHub issues.