Solved Bash/Shell scripts called from Monit are failing if monit was started by rc script, but are working if Monit was started manually after upgrade to 12.

C

click

Hi all,

I've been battling with this for a few days now. After a upgrade to 12.1 my jail monitoring stopped working and started spewing strange errors.
I am using Monit in combination with a few scripts. My scripts work fine when I use them and Monit works as expected when I start it manually, but if Monit was started by its rc script iocage claims there is no list command.
I am in a bind here, even if I try calling iocage directly from Monit it still complaints that there is no such option 'list'?

Bash: 
root@zeus:~ # monit status some_jail
Monit 5.26.0 uptime: 18h 39m

Program 'some_jail'
  status                       Status failed
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  last exit value              1
  last output                  + JAIL_NAME=some_jail
                               + IOCAGE=/usr/local/bin/iocage
                               + '[' -z some_jail ']'
                               + '[' '!' -e /usr/local/bin/iocage ']'
                               + /usr/local/bin/iocage list -H
                               + grep some_jail
                               + awk '{print $3}'
                               + grep -wq up
                               Usage: iocage [OPTIONS] COMMAND [ARGS]...
                               Try "iocage --help" for help.

                               Error: No such command "list".
                               + [[ 1 -eq 0 ]]
                               + echo 'some_jail : down'
                               + exit 1
                               MONIT_SERVICE=some_jail
                               PWD=/
                               HOME=/
                               LANG=en_US.UTF-8
                               RC_PID=74075
                               SHLVL=1
                               LC_ALL=en_US.UTF-8
                               PATH=/sbin:/bin:/usr/sbin:/usr/bin
                               _=/usr/bin/env
                               some_jail : down
  data collected               Wed, 11 Dec 2019 09:50:08

Bash: 
root@zeus:~ # /usr/local/sbin/check_jail_state.sh some_jail
SHELL=/bin/csh
GROUP=wheel
MACHTYPE=x86_64
EDITOR=vi
PWD=/root
LOGNAME=root
HOME=/root
LANG=en_US.UTF-8
OSTYPE=FreeBSD
SSH_CONNECTION=10.168.3.3 50340 10.168.10.10 22
TERM=xterm-256color
HOST=<CUT>
USER=root
LE_WORKING_DIR=/root/.acme.sh
SHLVL=2
PAGER=more
SSH_CLIENT=10.168.3.3 50340 22
LC_ALL=en_US.UTF-8
VENDOR=amd
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
BLOCKSIZE=K
MAIL=/var/mail/root
SSH_TTY=/dev/pts/1
HOSTTYPE=FreeBSD
REMOTEHOST=<CUT>
_=/usr/bin/env
+ JAIL_NAME=some_jail
+ IOCAGE=/usr/local/bin/iocage
+ '[' -z some_jail ']'
+ '[' '!' -e /usr/local/bin/iocage ']'
+ /usr/local/bin/iocage list -H
+ grep some_jail
+ awk '{print $3}'
+ grep -wq up
+ [[ 0 -eq 0 ]]
+ echo 'some_jail : up'
some_jail : up
+ exit 0some_jail

Bash: 
root@zeus:~ # service monit stop
Stopping monit.
Waiting for PIDS: 67233.
root@zeus:~ # monit
Starting Monit 5.26.0 daemon with http interface at [127.0.0.1]:2095
root@zeus:~ # monit monitor some_jail
root@zeus:~ #
root@zeus:~ # monit status some_jail
Monit 5.26.0 uptime: 2m

Program 'some_jail'
  status                       OK
  monitoring status            Monitored
  monitoring mode              active
  on reboot                    start
  last exit value              0
  last output                  + JAIL_NAME=some_jail
                               + IOCAGE=/usr/local/bin/iocage
                               + '[' -z some_jail ']'
                               + '[' '!' -e /usr/local/bin/iocage ']'
                               + /usr/local/bin/iocage list -H
                               + grep some_jail
                               + awk '{print $3}'
                               + grep -wq up
                               + [[ 0 -eq 0 ]]
                               + echo 'some_jail : up'
                               + exit 0
                               SHELL=/bin/csh
                               GROUP=wheel
                               MACHTYPE=x86_64
                               MONIT_SERVICE=some_jail
                               EDITOR=vi
                               PWD=/
                               LOGNAME=root
                               HOME=/root
                               LANG=en_US.UTF-8
                               OSTYPE=FreeBSD
                               SSH_CONNECTION=10.168.3.3 50340 10.168.10.10 22
                               TERM=xterm-256color
                               HOST=<CUT>
                               USER=root
                               LE_WORKING_DIR=/root/.acme.sh
                               SHLVL=2
                               PAGER=more
                               SSH_CLIENT=10.168.3.3 50340 22
                               LC_ALL=en_US.UTF-8
                               VENDOR=amd
                               PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
                               BLOCKSIZE=K
                               MAIL=/var/mail/root
                               SSH_TTY=/dev/pts/1
                               HOSTTYPE=FreeBSD
                               REMOTEHOST=<CUT>
  data collected               Wed, 11 Dec 2019 10:12:21

Monit conf:
Bash: 
### Jail process
check program some_jail with path "/usr/local/sbin/check_jail_state.sh some_jail"
  group jails
  if status != 0 then restart
  start program "/usr/local/bin/iocage start some_jail"
  stop program "/usr/local/bin/iocage stop some_jail"
  if 5 restarts within 5 cycles then timeout
  depends on iocage_bin
  depends on check_jail_state.sh
  depends on some_jail-rootfs

Script:
Bash: 
#!/usr/local/bin/bash
export LANG=en_US.UTF-8
export LC_ALL=en_US.UTF-8
env
set -x
###################
JAIL_NAME=$1
IOCAGE="/usr/local/bin/iocage"

if [ -z "$JAIL_NAME" ]
then
 echo "Usage: $(basename "$0") jail_name"
 exit 1
fi

if [ ! -e $IOCAGE ]
then
 echo "Please install iocage first"
 exit 1
fi


$IOCAGE list -H | grep $JAIL_NAME | awk '{print $3}'| grep -wq "up"
 if [[ $? -eq 0 ]]
  then
   echo "$JAIL_NAME : up"
   exit 0
  else
   echo "$JAIL_NAME : down"
   exit 1
 fi
 
Found the bugger, it would seem that iocage is unable to find some of its libs and instead of staying that it says that there is no such command (useful right?) so the issue was the PATH all along.
If you do call iocage from a rc script make sure to include the full path in /usr/local/etc/rc.conf.d/monit:
Code: 
export PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
 
You must log in or register to reply here.
Back
Top