Boot order and dependencies - getting PPP to wait

Oh, wait. I don't see any "shell" command.
The ppp(8) manual saysAll external commands (executed via the "shell" or "!bg" commands) are executed.... So "shell" is for foreground and "!bg" is for background commands. You may want to wait for some things, and not for others (so different scripts run by "shell" and, subsequently, "!bg"). Choose wisely.
So... Just move pfctl, ntpdate, and named out of the rc hierarchy and into ppp.linkup.cmd and in ppp.linkup
Code:
CenturyLink:
    !bg /etc/ppp.linkup.cmd
Yes, and no. You still want rc scripts to work by themselves when you choose to restart an individual service. And one of the places where you restart those services (with service xxx restart) is in the script that gets invoked by the "link-up" asynchronous event. You manage this on a case by case basis. Make sure that the rc script does not compromise the system if the Internet is not up, and does what you want when the Internet is up.
 
  • On substantial networks, I use three internal NTP time servers on physical Unix hosts (with clock chips) running as peers at strata 10, 12, and 14 -- so I don't have to have the Internet time servers (typically at superior strata 0, 1, 2, ...) to keep time, but will always use them if available.
Agree. My setup is similar but much simpler: I have ONE server machine on my home network (which runs FreeBSD). If that machine is down, all networking is down: no routing, no DNS, no DHCP addresses, the WiFi is unreachable, and so on. That machine has a half-decent hardware clock (the typical PC motherboard one), good enough to get through a few hours or days without outside NTP. Once the network to the outside world comes up, it synchronizes itself to real NTP servers.

BUT: One thing I found is that it is crazy to run a Unix machine without a hardware clock. That was the situation on my Raspberry Pi machines for a while. You end with files have modifications times of 1970, or log entries in places like /var/log/messages that jump back and forth by decades, and break any form of log analysis. So what I did is: (a) Configure SystemD (the much maligned software) so if nothing else is available, it uses a half-decent estimate of current time from date stamps is finds on disks, and (b) add inexpensive 5-pin real-time clock chips (attach right to the GPIO) wherever possible. With this, the problems of insane times getting persisted have gone away.
 
This is where I am stuck. If I set "named_enable=YES" in rc.conf, then it's going to get run before the PPP link comes up, and throw all those errors and so forth and sometimes get wedged. Similarly, ntpdate will try to run - and fail because the PPP link isn't up yet. but if I DON'T have _enable=YES then I can't use service restart, etc. I seem to be in trouble either way.
 
in rc.conf
named_setup="/root/xdate.sh"

sh:
#!/bin/sh
D=$(date +%Y%m%d)
if [ $D -lt 20260101 ]
then
 R=$(stat -f "%Sm" -t "%Y%m%d%H%M.%S" /root/xdate.sh)
 date $R
fi
then upon starting named the date will be set to the file modif time of /root/xdate.sh if the date is before 2026-01-01
you can then run another script from cron which checks if /root/xdate.sh if older then current time and touches it
then when you boot you will have a okish date not something like 1970
 
Possibly not okish enough. Named failed when the difference wasn't even 30 hours.

I have to believe that there's a solution to this, or am I really trying to do something so outlandish? I just want the sequence of ntpdate, named, and pfctl to run once the link comes up, but still be able to manage them through "service". Maybe it doesn't exist, but it doesn't feel like it should be this hard. We have all these powerful mechanisms, there's got to be a clever use of them that can do this.
 
well you can get the reference date from some log file or if you run the secondary script should be enough as long as you are not down more than 30 hours
find /root/ -name xdate.sh -mmin +60|xargs touch (run this every hour from cron)
 
Actually, I think we've gotten a bit off the path here. Even if we get the time right, named isn't going to like it if it can't load the zones because the PPP link is not up yet. And I don't know what the actual tolerance window is, I only know from experience that 30 hours was too much.
 
Ah hah! I think we may have found the key here. As I said, I'm still learning the full scope of rc functionality.

So, in rc.conf we would have
ntpdate_enable="YES"
ntpdate_hosts="north-america.pool.ntp.org"
ntpdate_setup="ping -qnc 1 8.8.8.8"
pf_enable="YES"
pf_setup="ping -qnc 1 8.8.8.8"
named_enable="YES"
named_setup="ping -qnc 1 8.8.8.8"
ntpd_enable="YES"
ntpd_setup="ping -qnc 1 8.8.8.8"

(note I should be able to use a DNS name in ntpdate_hosts because I have three outside nameservers in my resolv.conf file)
and in ppp.linkup
Code:
CenturyLink:
        !bg /etc/ppp.linkup.cmd
and in ppp.linkup.cmd
#!/bin/sh
service ntpdate restart &&\
service pf restart &&\
service named restart &&\
service ntpd restart


Or something much like that?
 
I just tested ping and the timeout for a failed ping was 11 seconds:
Code:
[gunsynd.1036] $ time ping -qnc 1 8.8.8.254; echo $?                
PING 8.8.8.254 (8.8.8.254): 56 data bytes

--- 8.8.8.254 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
   11.09s real     0.00s user     0.01s system
2
You might want to add "-W waittime" to the ping options to avoid a 44 second delay when booting.
 
Good suggestion. Also, since ntpd will be running after the first time, ntpdate will fail on subsequent runs, which is actually fine, because the system will have been keeping time. Therefore I don't want to check for failure on that. In fact, I feel like those tests on exit status are maybe just a complication with implications I haven't thought of yet, so I'm going to take those out.
# service ntpdate start ; echo $status
Setting date via ntp.
29 Jan 11:07:50 ntpdate[13823]: the NTP socket is in use, exiting
1


So - I think what I want in rc.conf is
ntpdate_enable="YES"
ntpdate_hosts="north-america.pool.ntp.org"
ntpdate_setup="ping -W 5000 -qnc 1 8.8.8.8"
pf_enable="YES"
pf_setup="ping -W 3000 qnc 1 8.8.8.8"
named_enable="YES"
named_setup="ping -W 3000 -qnc 1 8.8.8.8"
ntpd_enable="YES"
ntpd_setup="ping --W 3000 qnc 1 8.8.8.8"

and in ppp.linkup.cmd
#!/bin/sh
service ntpdate start
service pf restart
service named restart
service ntpd restart


Another option which would be much more complicated would be to have the ntpdate_setup put something in /tmp that the others would test for. That way we'd only have to wait for the failed ping once. Let me think about that. I'd just need to write two more scripts - one before ntpdate runs, and one that the others would use. This is a lot of little separate pieces I need to remember to take along when I migrate systems.....
 
Wait. It appears that "service" does not use the _setup commands. Is that true? If so - is it fair to say that the PPP link will NEVER be up at this point in the boot process? Could I just replace the pings with "/usr/bin/false" and basically not slow down the boot at all? I don't want to leave any boobytraps for myself, but this seems like a simple, reliable way to make sure that those services are not started until the link comes up.
 
So - it does vary by service. ntpdate and pf don't pay any attention to it, but named and ntpd both run it, but then start anyway!!!!!

ntpdate_setup="ping -W 3000 -qnc 1 8.8.8.254"
ntpd_setup="ping -W 3000 -qnc 1 8.8.8.254"
named_setup="ping -W 5000 -qnc 1 8.8.8.254"
pf_setup="ping -W 3000 -qnc 1 8.8.8.254"
/etc/rc.conf: 107 lines, 3125 characters
root@casper:/etc/rc.d # service ntpdate start
Setting date via ntp.
29 Jan 12:39:52 ntpdate[14543]: step time server 23.94.221.138 offset -0.001960 sec
root@casper:/etc/rc.d # service named start
Starting named.
PING 8.8.8.254 (8.8.8.254): 56 data bytes

--- 8.8.8.254 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
/usr/local/etc/rc.d/named: WARNING: failed to setup named
root@casper:/etc/rc.d # service named status
named is running as pid 14567.
root@casper:/etc/rc.d # service ntpd start
Starting ntpd.
PING 8.8.8.254 (8.8.8.254): 56 data bytes

--- 8.8.8.254 ping statistics ---
1 packets transmitted, 0 packets received, 100.0% packet loss
/etc/rc.d/ntpd: WARNING: failed to setup ntpd
root@casper:/etc/rc.d # service ntpd status
ntpd is running as pid 14609.
root@casper:/etc/rc.d #


So - even though it says "failed to setup" it starts it anyway - in fact, it looks like it starts it BEFORE it runs the _setup command. So - back to square 1. ALL the way back. I think that brings us all the way back to setting "service_enable" to NO and starting the services by direct invocation from ppp.linkup without using the "service" command at all. Dang!!!
 
you are are right about name_setup (it just warns)
but you can to this

ntpd_env_file="/root/ntpd.env"
# cat /root/ntpd.env
start_precmd="/root/xdate.sh"
a failed precmd will not start it
 
OK. This works as advertised for ntpd, and it also appears to work for ntpdate, but it does something strange when I try it with named:
root@casper:~ # service named start
PING 8.8.8.8 (8.8.8.8): 56 data bytes

--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 13.383/13.505/13.628/0.123 ms
Starting named.
ln: ./.: Operation not permitted
root@casper:~ #

and it doesn't get started, even though it should. I wondered if there was another named_env_file declaration around somewhere for named that I'm interfering with, but I haven't found it. I searched /etc/rc.conf /etc/defaults/rc.conf and /usr/local/etc/rc.d/named I have no idea what the "ln:./:Operation not permitted" is about!

Any clues?
 
named seems to have its own start_precmd which you override and probably thats not a good thing
what you can do is this
in the named_env_file
old_start_precmd="$start_precmd"
also you have to shove the ping code in the same env file
named_env_file
Code:
old_start_precmd="$start_precmd"
start_precmd="ping_test"
ping_test() {
if [ -n "$old_start_precmd" ]
then
$old_start_precmd || return 1
else
ping -c ....
fi
}
 
Hah! I think we got it! Thank you so much!

But first. I think I know what you were getting at, but I don't think that code does what you intended. Your code only executes the ping command if the original precmd does not exist. I want to execute it regardless, and might as well do it first, since we can stop as soon as we know it if we have to stop.

So - what I changed it to is this:
Code:
old_start_precmd="$start_precmd"
start_precmd="ping_test"
ping_test() {
ping -W 3000 -qnc 1 8.8.8.8 || return 1
if [ -n "$old_start_precmd" ]
then
$old_start_precmd || return 1
fi
}

That works, but then I thought I'd optimize it a bit. I only need to do the ping test for the first of the four dependent services. Also, if the link should happen to come up during this time, I don't want it to try to start the later services because they depend on the earlier ones. So, I ran rcorder to verify the sequence, which is as follows:
pf 78
ntpdate 115
named 118
ntpd 141


{As a side note, I also get a warning/error from rcorder: rcorder: file `/usr/local/etc/rc.d/saslauthd' is before unknown provision `imap' - is this something that should be reported?}

Then I created two environment files - one for doing a ping test, and the others doing a file test based on the ping test.
linkprecmdping
Code:
old_start_precmd="$start_precmd"
start_precmd="linkping_test"
linkping_test() {
if ping -W 3000 -qnc 1 8.8.8.254 > /tmp/linkpingfile ; then
    rm /tmp/linkpingfile
else
    return 1;
fi
if [ -n "$old_start_precmd" ]
then
$old_start_precmd || return 1
fi
}
and
linkprecmdfile
Code:
old_start_precmd="$start_precmd"
start_precmd="linkfile_test"
linkfile_test() {
if [ -e /tmp/linkpingfile ] ; then
    return 1;
fi
if [ -n "$old_start_precmd" ]
then
$old_start_precmd || return 1
fi
}
The end of my /etc/rc.conf file now looks like this:
rc.conf
Code:
pf_env_file="/root/linkprecmdping"
ntpdate_env_file="/root/linkprecmdfile"
named_env_file="/root/linkprecmdfile"
ntpd_env_file="/root/linkprecmdfile"

With the "bad" IP address in the ping test, we get
root@casper:~ # service pf start
root@casper:~ # service ntpdate start
root@casper:~ # service named start
/usr/local/etc/rc.d/named: WARNING: failed precmd routine for named
root@casper:~ # service ntpd start
/etc/rc.d/ntpd: WARNING: failed precmd routine for ntpd
root@casper:~ #


With the "good" IP address in the ping test, we get root@casper:/home/john # service pf start
Enabling pf.
root@casper:~ # service ntpdate start
Setting date via ntp.
30 Jan 15:26:01 ntpdate[22262]: step time server 193.29.63.226 offset +0.000223 sec
root@casper:~ # service named start
Starting named.
root@casper:~ # service ntpd start
Starting ntpd.


In support of this, I have
ppp.linkup
Code:
CenturyLink:
   !bg /root/linkupservices
and /root/linkupservices
Code:
service pf start
service ntpdate start
service named start
service ntpd start

Running out of time and ambition today, but I'll try this with the actual PPP link tomorrow.

Again - thanks!!!
 
Interestingly enough, in the boot execution (rcorder) analysis I did above, ppp(8) does actually get started first at #66, but I'm sure it doesn't come close to establishing the link before it reaches these other services. I just wish there was a way to keep ppp(8) from becoming a daemon UNTIL the link is up and running - then none of this would have been necessary. Also, I just realized, I'm going to have to go through my dhcpd configuration and change all the DNS references to IP addresses. It has been a nice feature and a nice documentary touch to be able to dedicate IP addresses by DNS name, but since my local DNS won't be up and running and I might need dhcpd services before then, I'll have to do the lookups manually.

So - for the six services I've been talking about, the order is:
ppp 66
pf 78
ntpdate 115
named 118
ntpd 141
dhcpd 159


Anyway, as you can imagine, up until now, booting has been a rather messy process of me having to stop and start various services manually because of these interdependencies. It sorta worked because, even though named was complaining about the time skew and inability to reach authoritative servers, it was satisfying local network requests, which meant that dhcpd was happy. But now, knowing that named probably WON'T be running when it kicks off dhcpd, it'll just be better to remove that dependency by doing the lookups myself.
 
Interestingly enough, in the boot execution (rcorder) analysis I did above, ppp(8) does actually get started first at #66, but I'm sure it doesn't come close to establishing the link before it reaches these other services. I just wish there was a way to keep ppp(8) from becoming a daemon UNTIL the link is up and running - then none of this would have been necessary. Also, I just realized, I'm going to have to go through my dhcpd configuration and change all the DNS references to IP addresses. It has been a nice feature and a nice documentary touch to be able to dedicate IP addresses by DNS name, but since my local DNS won't be up and running and I might need dhcpd services before then, I'll have to do the lookups manually.

So - for the six services I've been talking about, the order is:
ppp 66
pf 78
ntpdate 115
named 118
ntpd 141
dhcpd 159


Anyway, as you can imagine, up until now, booting has been a rather messy process of me having to stop and start various services manually because of these interdependencies. It sorta worked because, even though named was complaining about the time skew and inability to reach authoritative servers, it was satisfying local network requests, which meant that dhcpd was happy. But now, knowing that named probably WON'T be running when it kicks off dhcpd, it'll just be better to remove that dependency by doing the lookups myself.
I presume rc.d/ppp is for dial-up; rc.d/pppoed runs later because it needs ethernet.
 
Just wondered why people use ppp. I did many years ago, but have forgotten what for.

Isn't is simpler to connect a smartphone to a computer and establish an Internet connection via a USB Tether?

I started doing that a couple of months ago when my router broke and haven't looked back since. In fact I'm really glad my router broke.
 
a lot of providers have pppoe over pots or fiber. mobile data is usually slower, more expensive. also here you won't even get a public ip for mobile data, you are always behind isps nat
 
Back
Top