shell scripts with multiple rsync lines and &&

P

ProServ

Hi,
We have some scripts that have multiple lines to run rsync as shown below having && at the end of the line causing
Code: 
Invalid null command.

/sbin/fsck -y /dev/da0p1 &&
/bin/sleep 5 &&
/sbin/mount /bkup &&
/bin/sleep 4 &&

Code: 
/usr/local/bin/rsync -azP --ignore-existing /root/ /bkup/root &&

/usr/local/bin/rsync -azP --ignore-existing /etc/ /bkup/etc &&
/usr/local/bin/rsync -azP --ignore-existing /usr/local/etc/ /bkup/usr-local-etc &&
/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/myname/ /bkup/myname &&

/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/1-bkup/ /bkup/dir1 &&

/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/2-bkup/ /bkup/dir2 &&

/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/3-bkup/ /bkup/dir3 &&

Two things here. How would we set the script to only run one rsync at a time until each one is finished and, how can we tell the script if /bkup is mounted to skip the line
Code: 
/sbin/mount /bkup &&

Thanks for your help.
 
It just means if one passes (returns 0) the next one is executed and so on.

If i'm reading your output correctly, the very ending && is nonsensical.

Note, you can also use || for, yes, or-ing.

Edit: I missed your question on mount.

Just use mount and grep for it or use df, ie, df /bkup and test the result code.
 
Can't you just have each rsync on its own line in a shell script so they run sequentially? So you have backup.sh and you run that?

Why do you need to use the && to join the commands together?

Apologies if I've missed something obvious!

And you can test for the mount point before you run the mount command - I'd have to dig out the shell code for that. It depends on which shell you are using?

I've got something like this:

Code: 
set NAS_MOUNTED="/mnt/nas/mounted.txt"
set NAS_DIR="/mnt/nas/`date +%A`"
...
echo "Checking for $NAS_MOUNTED"
if (-f $NAS_MOUNTED) then
    echo "Checking for $NAS_DIR"
    if (-d $NAS_DIR) then
        # Looks like NAS is mounted, and we've got a directory for today
        echo "Copying to NAS ..."
...

    else
        echo "NAS mounted, but directory $NAS_DIR does NOT exist"
    endif
else
    echo "$NAS_MOUNTED does NOT exist - is the NAS mounted?"
...
endif
So mounted.txt is just a text file - if I can find it then the /mnt/nas is good to go.
 
richardtoohey2 said:
Can't you just have each rsync on its own line in a shell script so they run sequentially? So you have backup.sh and you run that?

Why do you need to use the && to join the commands together?

Apologies if I've missed something obvious!

And you can test for the mount point before you run the mount command - I'd have to dig out the shell code for that. It depends on which shell you are using?
Click to expand...
On face value it looks like, over time, people have added directories to rsync. Yes, a bit clumsy but workable.
 
Hi Perhaps I am not explaining myself correctly.
1) if /bkup is already mounted, ignore the mount command or I guess I could add to the script /sbin/umount /bkup but in either case, if /bkup is mounted, the script doesn't work right

2) I thought && means; complete this task and then move to next command.
 
Does the command end with && by itself? Or is there another command after it? If your command is just ending with an && then it's telling you there's an empty command.

I'd just keep it simple and make a multi-line script (without any && or || or whatever) and run the script. You are making it more complicated than it needs to be and having issues.
 
ProServ said:
Hi Perhaps I am not explaining myself correctly.
1) if /bkup is already mounted, ignore the mount command or I guess I could add to the script /sbin/umount /bkup but in either case, if /bkup is mounted, the script doesn't work right

2) I thought && means; complete this task and then move to next command.
Click to expand...

1) Test for mount, see my answer above: https://forums.freebsd.org/threads/shell-scripts-with-multiple-rsync-lines-and.76673/post-475017

2) It does, but you've got it at the very end of the command. That's nonsensical. Perhaps it was a single & for backgrounding it?


What shell are you using?
 
richardtoohey2 said:
Can't you just have each rsync on its own line in a shell script so they run sequentially? So you have backup.sh and you run that?
Click to expand...

If you did this you would have to test for each command's failure. The && does this for you.

Personally I would write a function and pass it the from and to directories because all the commands are the same. Then you can test the function's result and the script will look nice and clean .

In sh, something like:

Code: 
cmd="/usr/local/bin/rsync -azP --ignore-existing"
opt=""

sync()
{
    from=$1
    to=$2
    $cmd $opt $from $to
    if [ $? -ne 0 ]; then
        echo "Failed rsync on copy ${from} to ${to}"
        exit 1
    fi
}

# Then:
opt=""
sync /usr/local/etc/ /bkup/usr-local-etc
opt="--remove-source-files"
sync /usr/local/ftp/myname/ /bkup/myname
sync /usr/local/ftp/1-bkup/ /bkup/dir1

# etc.
# Untested and rough!
 
The entire script is below. The shell is sh. Running ./script just terminates quickly, rsync doesn't run.
Code: 
#!/bin/sh
# /sbin/fsck -y /dev/da0p1 ||
# /bin/sleep 5 ||
# /sbin/mount /bkup ||
# /bin/sleep 4 ||
/usr/local/bin/rsync -azP --ignore-existing /root/ /bkup/root ||
/usr/local/bin/rsync -azP --ignore-existing /etc/ /bkup/etc ||
/usr/local/bin/rsync -azP --ignore-existing /usr/local/etc/ /bkup/usr-local-etc ||
/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/site1/ /bkup/site1 ||
/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/dir1-bkup/ /bkup/dir1 ||
/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/dir2-bkup/ /bkup/dir2 ||
/usr/local/bin/rsync -azP --ignore-existing --remove-source-files /usr/local/ftp/dir3-bkup/ /bkup/dir3 ||
/bin/sleep 4 ||
/sbin/umount -f /bkup ||
/bin/sleep 4 ||
/sbin/fsck -y /dev/da0p1
/bin/echo "/bkup has been unmounted"
by using #!/bin/sh when the script is run, do not get invalid command but as mentioned above it terminates without running all the rsync lines.....

Code: 
# ./rsnap-ftp.sh
sending incremental file list
./
scripts/
/bkup has been unmounted
 
Last edited by a moderator:
Not || but &&. Keep in mind, any one failure will cause the rest to not be run.

If it still doesn't run, add -x to the end of the shebang: #!/bin/sh -x

Overall, that script is SO messy. As you've found, when debugging, it's impossible to know where it fails etc.
Honestly, stop trying to be cute with the shell and write something you can understand in a year's time.
 
Few things:

1. A shell script shall not have lines ending with && or ||; one shall use line-continualtion character \; incorrect:

Bash: 
cmd1 &&
cmd2

correct:

Bash: 
cmd1 && cmd2

or

Bash: 
cmd1 && \
cmd2

2. To test if file-system location is a mount point or not you can use this shell function:

Bash: 
ismountpt() {
    local loc base d1 d2
    loc="${1%/}" # remove trailing slash, if such
    [ ! -d "${loc}" ] && return 1
    base="${loc%/*}" # get base dir
    : ${base:=/}
    d1="$(stat -f %d "${loc}")" # retrieve locations' devices
    d2="$(stat -f %d "${base}")"
    [ "${d1}" = "${d2}" ] && return 1
    return 0
}

and use it like this:

Bash: 
um=0
ismountpt /bkup || { mount /bkup; um=1; }
# ...
[ ${um} -eq 0 ] || umount /bkup

cmd1 && cmd2 means run cmd1 and only if it succeeds run cmd2. cmd1 || cmd2 means run cmd1 and only if it fails run cmd2.

Don't take it as offence, but it seems like you miss the basic rules of shell scripting; read a tutorial or two.
 
You must log in or register to reply here.
Back
Top