Solved Shell and Crontab

Hi. I get an error when I try to call this shell from crontab. I can run this script manually without problem, but it just doesn't work with cron.

backup.sh
Code:
#!/bin/sh
# Get mode from user input
#   Allowed modes:
#       all:        backups account, common, log, player, webserver
#       game:       backups account, common, player
#       gameLog:    backups log
#       web:        backups webserver
#       exceptLog:  backups account, common, player, webserver
MODE=$1
if [ "$MODE" == "" ]; then
    MODE="exceptLog"
fi
# Bins
MYSQLDUMP=`which mysqldump`
GZIP=`which gzip`
NCFTP=`which ncftp`
# Date for folders and filenames
DAY=$(date +"%Y-%m-%d")
FILETIME=$(date +"%Y-%m-%d.%T")
# Local backup folder (no trailing slash)
LOCAL_FOLDER="/tmp/backup"
# FTP Configuration
REMOTE_HOST="####"
REMOTE_USER="####"
REMOTE_PASS="####"
REMOTE_FOLDER="/backup/####/" # With trailing slash
# MySQL Configuration
MYSQL_USER="####"
MYSQL_PASS="####"
# Which databases shall we backup?
# Databases should be separated with a space
DATABASES=""
if [ "$MODE" == "all" ]; then
    DATABASES="account common log player webserver"
elif [ "$MODE" == "game" ]; then
    DATABASES="account common player"
elif [ "$MODE" == "gameLog" ]; then
    DATABASES="log"
elif [ "$MODE" == "web" ]; then
    DATABASES="webserver"
elif [ "$MODE" == "exceptLog" ]; then
    DATABASES="account common player webserver"
fi
# Check if DATABASES var is set...
if [ "$DATABASES" == "" ]; then
    echo -e "\033[31mThe specified mode doesn't exist...\033[0m"
    exit 1
fi
# Dump and compress
for db in $DATABASES
do
    FILE=$db.$FILETIME.gz
    echo -e "\033[32mDumping $db!\033[0m"
    $MYSQLDUMP -u $MYSQL_USER -p$MYSQL_PASS $db | $GZIP -9 > $LOCAL_FOLDER/$FILE
done
# Transfer all backup files to remote host
echo -e "\033[32m\nTransfering files!\033[0m"
$NCFTP -u$REMOTE_USER -p$REMOTE_PASS $REMOTE_HOST<<EOF
mkdir $REMOTE_FOLDER$DAY
cd $REMOTE_FOLDER$DAY
lcd $LOCAL_FOLDER
mput *
quit
EOF
# Delete local dump files
rm -f $LOCAL_FOLDER/*

I'm using FreeBSD 9.3. My crontab -l output is
Code:
* * * * * /usr/home/backup.sh
and I tried adding
Code:
* * * * * sh /usr/home/backup.sh
 
I have limited knowledge about FreeBSD. Could you tell me if the PATH /usr/home/SCRIPT is the wrong path to use? Or maybe take a look at my shell if have something wrong with this.
 
Have a look at /var/log/cron to see if the script is actually being run. If the script produces output, the output will get mailed to the user who's crontab it runs on. So check their mail too. Please post any errors you find.
 
SirDice, I haven't any email configured on my server. Then to see the output I try to make it write to the /usr/home/output.txt, but it just creates the file and didn't record any information.

/var/log/cron
Code:
Dec  4 13:20:00 ns203462 /usr/sbin/cron[10115]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:21:00 ns203462 /usr/sbin/cron[10135]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:22:00 ns203462 /usr/sbin/cron[10156]: (operator) CMD (/usr/libexec/save-entropy)
Dec  4 13:22:00 ns203462 /usr/sbin/cron[10157]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:23:00 ns203462 /usr/sbin/cron[10186]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:24:00 ns203462 /usr/sbin/cron[10206]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:25:00 ns203462 /usr/sbin/cron[10227]: (root) CMD (/usr/libexec/atrun)
Dec  4 13:25:00 ns203462 /usr/sbin/cron[10228]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:26:00 ns203462 /usr/sbin/cron[10248]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:27:00 ns203462 /usr/sbin/cron[10268]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)
Dec  4 13:28:00 ns203462 /usr/sbin/cron[10288]: (root) CMD (sh /usr/home/backup.sh game >> /usr/home/output.txt)

DutchDaemon, in my script do I have any place where I am not using the full PATH? Maybe
Code:
MYSQLDUMP=`which mysqldump`
 
Hi,

From man page you can read the environment variables that are set up by default:

Code:
  Several environment variables are set up automatically by the cron(8)
  daemon.  SHELL is set to /bin/sh, PATH is set to /usr/bin:/bin, and
  LOGNAME and HOME are set from the /etc/passwd line of the crontab's
  owner.  HOME, PATH and SHELL may be overridden by settings in the
  crontab; LOGNAME may not.

If you want to test your script in crontab environment you can use this command:

env -i SHELL=/bin/sh PATH=/usr/bin:/bin HOME=[I]"Your home"[/I] LOGNAME=[I]"your logname" [/I] [I]your_script.sh[/I]

Maybe you can check the error with this command.

That command has been extracted from https://www.freebsd.org/doc/handbook/configtuning-cron.html

Best regards
 
/usr/home/backup.sh game >> /usr/home/output.txt
This redirect only stdout, but not stderr. Use:
/usr/home/backup.sh game >> /usr/home/output.txt 2>&1
which should catch both, stdout and stderr and then show logs.
 
I try like this env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh
And this was the /etc/log/con
Code:
Dec  4 15:30:00 ns203462 /usr/sbin/cron[12813]: (root) CMD (env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh)
Dec  4 15:31:00 ns203462 /usr/sbin/cron[12835]: (root) CMD (env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh)
Dec  4 15:32:00 ns203462 /usr/sbin/cron[12857]: (root) CMD (env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh)
Dec  4 15:33:00 ns203462 /usr/sbin/cron[12882]: (operator) CMD (/usr/libexec/save-entropy)
Dec  4 15:33:00 ns203462 /usr/sbin/cron[12883]: (root) CMD (env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh)

And using /usr/home/backup.sh game >> /usr/home/output.txt 2>&1 it records info in the output.txt and this was the result.

Code:
[32mDumping account![0m
/usr/home/backup.sh: -u: not found
[32mDumping common![0m
/usr/home/backup.sh: -u: not found
[32mDumping player![0m
/usr/home/backup.sh: -u: not found
[32m
Transfering files![0m
/usr/home/backup.sh: -umetiness: not found
 
The date command has no full path, and you use ==, which is a bash-ism. use =.

And judging by the -u error, some of your variables (looks like $MYSQLDUMP in this case) are empty.

which is in your search PATH (it's in /usr/bin), but maybe it fails.
 
This is the which output.
Code:
root@ns203462:/usr/home # which mysqldump
/usr/local/bin/mysqldump
root@ns203462:/usr/home # which gzip
/usr/bin/gzip
root@ns203462:/usr/home # which ncftp
/usr/local/bin/ncftp
root@ns203462:/usr/home #
And about the date i didn't understand what you mean sorry.

If i run the script like this root@ns203462:/usr/home # sh backup.sh it work perfect and upload the files to the FTP. And i get this output.

Code:
root@ns203462:/usr/home # sh backup.sh
Dumping account!
Dumping common!
Dumping player!

Transfering files!
NcFTP 3.2.5 (Feb 02, 2011) by Mike Gleason (http://www.NcFTP.com/contact/).
Connecting to ##########...
Bienvenue,

    On Vous Héberge ?

Vous êtes connecté sur webm293.
This is a private system - No anonymous login
Logging in...
User metiness has group access to:  users
OK. Current restricted directory is /
Logged in to ###########.
OK. Current directory is /backup/##########/2014-12-04
account.2014-12-04.15:42:13.gz:                          2.73 kB   26.15 kB/s
common.2014-12-04.15:42:13.gz:                           1.53 kB   14.77 kB/s
player.2014-12-04.15:42:13.gz:                         310.72 kB    2.93 MB/s
 
I try like this env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh

With that command, did the script run without error? As far as I know, if you met any errors, then they should appear in stdout because you are not running the script in cron so the output of /var/log/cron is useless.

Anyway, you put a PATH different from cron environment. Could you change PATH variable to PATH=/usr/bin:/bin and test again?

If the script ran without any errors with env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh, then you can set up PATH variable inside crontab file and it should work.
 
With that command, did the script run without error? As far as I know, if you met any errors, then they should appear in stdout because you are not running the script in cron so the output of /var/log/cron is useless.

Anyway, you put a PATH different from cron environment. Could you change PATH variable to PATH=/usr/bin:/bin and test again?

If the script ran without any errors with env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh, then you can set up PATH variable inside crontab file and it should work.


Using
env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh

Code:
root@ns203462:/usr/home # env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/us
r/sbin HOME=/usr/home LOGNAME=root /usr/home/backup.sh
Dumping account!
/usr/home/backup.sh: -u: not found
Dumping common!
/usr/home/backup.sh: -u: not found
Dumping player!
/usr/home/backup.sh: -u: not found
Dumping webserver!
/usr/home/backup.sh: -u: not found

Transfering files!
/usr/home/backup.sh: -umetiness: not found
root@ns203462:/usr/home #
 
Keep in mind that which(1) only searches the user's PATH. In case of crontab(1), only /bin and /usr/bin are searched. As mysqldump and the other applications reside in /usr/local/bin/ the which(1) command fails to find them.

Yes, you are right, but then I don't understand this behaviour.

Acording to env manpage:

Code:
  -i  Execute the utility with only those environment variables speci‐
  fied by name=value options.  The environment inherited by env is
  ignored completely.

So, -i should create its own environment. But I perform the next test:

Code:
➜  ~  sh
$ env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin a=`which psql`; echo $a
a=/usr/local/bin/psql
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin
SHELL=/bin/sh

// which find the program but it's not in PATH

$ echo $PATH
/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/home/carlos/bin

// But it's on shell PATH

$ which psql
/usr/local/bin/psql

// And on shell works fine, as expected because is in shell path

$ export PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin
$ echo $PATH
/etc:/bin:/sbin:/usr/bin:/usr/sbin

// We put the same path from -i flag in shell environment

$ which psql

// This is as expected, psql isn't found because now is not in path

$ env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin a=`which psql`; echo $a
a=
PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin
SHELL=/bin/sh

// Now, env -i works as expected. psql isn't found.

$

It seems that -i flag doesn't work as expected because env inherited shell path or there is something I misunderstood?
 
Simplest solution, add this to your script, just below the #!/bin/sh line:
Code:
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin
 
Ok,

I think that -i flag applies after command is executed so in:

env -i SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin a=`which psql`

which is executed on sh environment.

Best regards
 
yagokurt, just wondering who wrote this script? Is it from a metin site (I think metin uses MySQL, right?) or did you write it yourself?
 
Simplest solution, add this to your script, just below the #!/bin/sh line:
Code:
PATH=/bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin:/usr/local/sbin

It works SirDice. Amazing. Thank you.

yagokurt, just wondering who wrote this script? Is it from a metin site (I think metin uses MySQL, right?) or did you write it yourself?

DutchDaemon, a friend who I have no contact with any more. He wrote it, I don't know, almost one year ago and I was using it manually and then I just wanted to put it automatic and daily. I use MariaDB SirDice and you helped me install it.

Thank you all who helped me.
 
Back
Top