Solved freebsd-update hangs.

skunk

skunk

When I spawn freebsd-update via system(), it just gets stuck, does not return.

I tried various options, to no avail.
Last try was with
env ASSUME_ALWAYS_YES=YES PAGER=cat NOTTYOK=1 VERBOSELEVEL=nostats freebsd-update --no-stats --not-running-from-cr
on -b /jails/templates_ro/13.2-RELEASE/ fetch
which also just hangs.

When I do this in a terminal window, I get this:
Code: 
# env ASSUME_ALWAYS_YES=YES PAGER=cat NOTTYOK=1 VERBOSELEVEL=nostats freebsd-update --no-stats --not-running-from-cr
on -b /jails/templates_ro/13.2-RELEASE/ fetch
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 3 mirrors found.
Fetching public key from update2.freebsd.org... /usr/sbin/freebsd-update: cannot create : No such file or directory
failed.
Fetching public key from update1.freebsd.org... /usr/sbin/freebsd-update: cannot create : No such file or directory
failed.
Fetching public key from dualstack.aws.update.freebsd.org... /usr/sbin/freebsd-update: cannot create : No such file or directory
failed.
No mirrors remaining, giving up.

This may be because upgrading from this platform (amd64)
or release (13.2-RELEASE) is unsupported by freebsd-update. Only
platforms with Tier 1 support can be upgraded by freebsd-update.
See https://www.freebsd.org/platforms/ for more info.

If unsupported, FreeBSD must be upgraded by source.
#

This message just confuses me.
I believed 13.2 and amd64 are currently in full support.
Any idea what could be wrong?
 
The sha256 program's output is lost somehow.
This makes the key's hash check fail.
Thus the fetch_key() subroutine gets repetitively called, indefinitely, effectively hanging freebsd-update.

I made some investigations, and my feeling is that for some reason the sha256 program does not output nothing.

This is the current modified fetch_key() subroutine:
Added/modified lines are shown in red/blue color.
Code: 
fetch_key () {
        if [ -r pub.ssl ] && [ `${SHA256} -q pub.ssl` = ${KEYPRINT} ]; then
return 0
fi

echo -n "Fetching public key from ${SERVERNAME}... "
rm -f pub.ssl
echo
pwd
echo  DOING: fetch -q http://${SERVERNAME}/${FETCHDIR}/pub.ssl
echo keyprint:${KEYPRINT}:
[COLOR=rgb(41, 105, 176)]#       fetch ${QUIETFLAG} http://${SERVERNAME}/${FETCHDIR}/pub.ssl \[/COLOR]
[COLOR=rgb(184, 49, 47)]fetch -q http://${SERVERNAME}/${FETCHDIR}/pub.ssl \[/COLOR]
2>${QUIETREDIR} || true
if ! [ -r pub.ssl ]; then
echo "failed."
return 1
fi
echo -n pubssl:
cat pub.ssl
echo :
echo sha prog: ${SHA256}
echo -n sha:
${SHA256} pub.ssl
echo :
echo -n sha -q:
${SHA256} -q pub.ssl
echo :
echo -n catsha:
cat pub.ssl | ${SHA256}
echo :
echo Running sha256 -s hello
echo start
sha256 -s hello
echo done
echo Running stdouterrtest.c
echo start
/stdouterrtest/a.out
echo done

if ! [ `${SHA256} -q pub.ssl` = ${KEYPRINT} ]; then
echo "key has incorrect hash."
[COLOR=rgb(184, 49, 47)]exit 77[/COLOR]
rm -f pub.ssl
return 1
fi
echo "done."
}

To make sure that it is not something that makes sh not see the stdout of programs, I made a stdouterrtest.c, whose a.out is called above:

Code: 
#include <stdio.h>

main()
{
fprintf(stdout,"This is output on stdout! \n");
fprintf(stderr,"This is output on stderr! \n");
return 0;
}

I call the whole stuff with system() this way:
env ASSUME_ALWAYS_YES=YES PAGER=cat freebsd-update --not-running-from-cron -b /jails/templates_ro/13.2-RELEASE/ fetch > /tmp/testlog


The output in /tmp/testlog is:


Code: 
src component not installed, skipped
Looking up update.FreeBSD.org mirrors... 3 mirrors found.
Fetching public key from update.FreeBSD.org...
/var/db/freebsd-update
DOING: fetch -q http://update.FreeBSD.org/13.2-RELEASE/amd64/pub.ssl
keyprint:800651ef4b4c71c27e60786d7b487188970f4b4169cc055784e21eb71d410cc5:
pubssl:-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA2kXIMh58KXUCk1siiXwG
tKk55IG2khASQbx7nL7Z9Mh0Ie4VnPV1REvchfYFz6gZV9cX7Yz9R7uyEac8NRXN
gKXGxnzF887CsjcTxWvsEoFqTLG57O4LSSm+MO3Sg4HuxnanifeWec9PNz150vU/
+q6s6XTtNHSysuo7XcBjczJigSBAMjdTEMCexiHKGZm0c9tyR4grA1TqSk2Agr9+
7F7SA4Ck8wjM7srFSlYoyxHZtTE+9vz0tx0/5v2VW11NN0RBNoY1F90CCL41Ln73
yqABPWTfAaJ6Z6oQMwNBc+Q2SoCzK76XgZosCWylj3rmzSPhdTcrvFVa0IsUc2Gr
wZImlJ7UtJYpVrKXDvOb1UG/2yQx8Eci+XyPhcgUahSN4Vidz9YyiD7qpf11VYhL
BFeHnkm9LsrBJil4wGuOpXUUzMiKxpR6g3gfMA7PcBK6erxzFzcJHI3KMN0m3PIW
szig8XxdLTc/e9XihPWqDbpGqNMjl9Tfz/nN2LXJA+acf41LenFIY7x4IkHrPM6D
UNU+PsR6F+FMg3VgiJ3lrJmssVqLZRD9MMu51JtrN2NK1uhmFJDK7ZkLefHjdfTh
1HOcIyd8rRt/m8LsbQcdFlg/BTeqoG+wHburoDn9z7HknB4DBxcyRGxHdJuBzytj
YKZHuFglSqt+29fNS/WXWTcCAwEAAQ==
-----END PUBLIC KEY-----
:
sha prog: /sbin/sha256
sha::
sha -q::
catsha::
Running sha256 -s hello
start
done
Running stdouterrtest.c
start
This is output on stdout!
done
key has incorrect hash.

Any idea why there is no output of sha256, making the hash check fail and rendering freebsd-update hang in an endless loop?
 
cracauer@ thank you for your hint to use truss -f !

Is this a Capsicum problem?

Code: 
36852 111318: cap_rights_limit(1,{ CAP_PWRITE,CAP_FCNTL,CAP_FSTAT,CAP_EVENT,CAP_IOCTL }) = 0 (0x0)
36852 111318: cap_ioctls_limit(0x1,0x2188309ee5c0,0x3) = 0 (0x0)
36852 111318: cap_fcntls_limit(1,CAP_FCNTL_GETFL) = 0 (0x0)
36852 111318: cap_rights_limit(2,{ CAP_PWRITE,CAP_FCNTL,CAP_FSTAT,CAP_EVENT,CAP_IOCTL }) ERR#9 'Bad file descriptor'
36852 111318: write(2,"sha256: ",8)              ERR#9 'Bad file descriptor'
36852 111318: write(2,"unable to limit rights for stdio",32) ERR#9 'Bad file descriptor'
36852 111318: write(2,": ",2)                    ERR#9 'Bad file descriptor'
[...]
36852 111318: write(2,"Bad file descriptor\n",20) ERR#9 'Bad file descriptor'
36852 111318: exit(0x1)                         
36852 111318: process exit, rval = 1

Every call to sha256 ends with it dying with exit code 1, instead of just printing what it should print.
Any easy way to fix this?
 
If I read the truss output correctly, the sha256 program is trying to perform a write operation on file descriptor 2, which is normally stderr, and should be open. It gets error 9 = EBADF. Which means that whoever called sha256 has probably closed that open file (or maybe sha256 did that itself, but that seems unlikely). Now, why they did that is a fabulously good question.

Look at the truss output some more, and check what is happening to fd #2.
 
richardtoohey2 said:
If this is all going through the perl system() call, maybe you have to do something with file descriptors in the system call?
Click to expand...
Well, it turned out that the problem was not in the system call itself...

SirDice said:
You probably need to use IPC::Open3.
Click to expand...
There is not much information for dummies how to start background processes.
I actually considered to use some pre-made module... but they are all sooo complicated and big.

I wanted a simple solution good for CGI programs, just a few lines, no megamodules.
So I ended up searching for good articles and adapted the double-forking technique which is well described in these articles:
https://stackoverflow.com/questions/17599096/how-to-spawn-child-processes-that-dont-die-with-parent
https://digitalbunker.dev/understanding-daemons-unix/

The mistake I made was not understanding that the background process must not only throw away the standard filehandles, but create new ones, to avoid problems when calling other processes via system().
See the open()s I added to the fragment below...

Code: 
if ($pid == 0) {
            # GRANDCHILD
            # Close all open file descriptors
            close STDIN or die;
            close STDOUT or die;
            close STDERR or die;
            open(STDIN, '</dev/null') or die;
            open(STDOUT, ">$stdoutfile") or die;
            open(STDERR, ">$stderrfile") or die;
#             $cmd = 'truss -s 128 -aefH -o /tmp/trusslog.txt ' . $cmd;
            my @argv = split(/ /, $cmd);
            exec( @argv);
            exit;
        }

Also, keeping the output is nice side effect...
 
You must log in or register to reply here.
Back
Top