script to run rsync backups syntax help

P

ProServ

Hi,
Have a script that is supposed to umount /bkup if mounted, fsck /bkup (/dev/da0p1), mount /bkup (/dev/da0p1) and then run multiple rsync commands one at a time. Due to incorrect syntax errors this script is not working as expected.

The idea is to run one command line at a time until it completes. It was my understanding that '&&' means wait until this command is finished before starting next command. But when the script is run, it produces the error "Invalid null command."

The script is run by crontab with /bin/sh /root/scripts/script1.sh and in part is:
#!/bin/sh
/sbin/umount /bkup &&
/sbin/fsck -y /dev/da0p1 &&
/bin/sleep 2 &&
/sbin/mount /bkup &&
/bin/sleep 2 &&
/usr/local/bin/rsync -azP --ignore-existing -e 'ssh -p 19225' root@host1.my-domain.tld:/root /bkup/dir1/ &&
/usr/local/bin/rsync -azP --ignore-existing -e 'ssh -p 19225' root@host2.my-domain.tld:/etc /bkup/dir2/ &&
.
.
.
/bin/sleep 5 &&
/sbin/umount -f /bkup &&
/bin/sleep 2 &&
/sbin/fsck -y /dev/da0p1

There are 26 rsync commands in this script backing up different servers.
Have tried substituting '||' for '&&' on each line but the error is the same.
So the question is, what is the correct syntax to put into the script so that each line must complete BEFORE going to the next command?

Thanks.
 
You need the && and || things only if you want to have commands nested in a single line. Please try without && and || as
Code: 
echo "Hi"
sleep 2
echo "Again"
sleep 2
echo "Finito"
 
&& runs next command if the previous command completed successfully.

|| runs next command if the previous command failed.

Also there is no need for sleep and fsck.
 
So many questions:

Why do you fsck? Do you suspect that the file system is damaged? And why do you fsck -y? If the file system is damaged, don't you want to know exactly what the damage is, instead of blazing along?

Why all the sleep calls? I think they're a waste of time.

How about logging from this script? It does a lot of useful things. It will occasionally fail (everything on the planet does). When it fails, you need to debug why.

We absolutely need error handling. And you need to check preconditions.

Do you really want to allow root ssh login? That's a security hole big enough to drive a tractor through.

You seem to have very little experience with shell scripts. In your hands, writing scripts that run as root and copy lots of data around is unsafe. An old colleague used to call this kind of thing a "foot-shaped gun": A gun that is most suited for shooting oneself in the foot.

Design hints: Take common things (like the name of the file system, the name of the host, the ssh port, and so on) and put them into variables.

And write some shell functions to encapsulate commonly used operations, like running the rsync/ssh.

And think about using parallelism. Depending on where your bottlenecks are (disk or network? how many disks?), running multiple rsync's in parallel might help (or it might hurt). Or you might not care about performance.
 
Hi, is there code we could add to sh scripts that would check if external drive is already mounted? Trying to get script which does backups to mount /backup if not already mounted and if not mounted, mount the backup disk.

As for the # fsck -y , have had in the past encountered useless backups due to file getting messed up somehow. Since our backup rsync lines will check dest against current file (--ignore-existing), the damaged file will be replaced. I don't see how fsck -y will hurt in this case.

As for root login, pf does a great job for login failures.
 
Dear ProServ,
basically you can use the same methods as if you are doing the task on the console. You have to be more pedantic in error handling when writing scripts. In interactive use the brain can catch simple errors without thinking. A script just runs as programmed. About the question you can run mount and check its output. May be by pipe the output to grep which filters the output and tells you if something is included or not. That can be mount|grep backupserver or so. The result can then be part of an if statement which then run something or not.

What I usually do is to check the commands one by one interactively. One task is to check the expected output. The more difficult thing is to get an idea of the output in case something goes wrong like disk not attached, cable broken, ... You have to think about what to do if something goes wrong. This is more difficult than specifying the good path. Then it is good to test things in steps as small as possible. For that you can already use short scripts using the standard script shell /bin/sh. When you are happy with the small building blocks you can add them to improve your existing scripts.

My advise is not to just google, copy&paste but find out what is going on. If something goes wrong you will have to find out why. By copy&paste it is difficult to learn how things work.
 
You must log in or register to reply here.
Back
Top