Shell Bash Script Anomoly

fullauto2012

fullauto2012

Code: 
root@Washington:/var/SNAPSHOTS # uname -a
FreeBSD Washington.XXXXXXXXX.XXX 10.1-RELEASE FreeBSD 10.1-RELEASE #0 r274401: Tue Nov 11 21:02:49 UTC 2014     root@releng1.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC  amd64


Code: 
root@Washington:/var/SNAPSHOTS # bash -version
GNU bash, version 4.3.42(0)-release (amd64-portbld-freebsd10.1)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.


I have a nightly backup routine that creates a tarball and makes a copy to a backup server, deleting the oldest copy to maintain no more than 7 local, and 10 on the backup server. The backup server, about 2 hours later searches for and uploads the newest tarball to a remote server for offsite.

Everything was running flawlessly for some time until after the new year.

I ran the script in debug mode (set +x) and got the line which is:
Code: 
++ su liberty.roofing -c 'ssh 192.168.1.20 ls -U /usr/home/liberty.roofing/SNAPSHOTS | grep -v log | head -n1'
+ oldfile=snap_Washington_01-03-16.tar.gz

That variable is supposed to be the OLDFILE, yet it is reporting the latest file...

When I run that line from shell on the machine I get a different answer, which is what it should be reporting, and HAD been reporting till the new year.
Code: 
root@Washington:/var/SNAPSHOTS # su liberty.roofing -c 'ssh 192.168.1.20 ls -U /usr/home/liberty.roofing/SNAPSHOTS | grep -v log | head -n1'
snap_Washington_12-22-15.tar.gz

Not sure what is going on here. Anyone have any suggestions?
 
It's usually the environment :) Or an old file has been touched by something and now looks newer. Or a careless ls produces unexpected output from a subdirectory. More info needed.

You might benefit, if you changed the timestamps into yy.mm.dd, more convenient to sort, and order would not change by careless copying/moving.

Suggestion:

Code: 
purge_extra_files() {
  local nkeep=$1
  shift
  if [ $# -gt $nkeep ]
  then
    shift $nkeep
    echo purging "$@"
    echo rm -- "$@"
  fi
}

purge_extra_files 23 `ls -t /tmp/xxx*` # keep 23 newest, by modification time
purge_extra_files 50 `ls -r /tmp/xxx??.??.??.dat` # keep 50 newest, by alphabetical order

Juha
 
It'll be easier to sort by date if you use ISO 8601 dates. So use 20160104 for example. The reason is that a plain old ASCII sort will automatically sort by date.
 
Juha. I'm in and out of these files all the time which is exactly why I opted for using the creation date rather than the last edited date. I like your idea of changing the time stamp. I can use cut -d. to figure on a list to purge...
Also, as I am very green with regard to scripting what does 'local' and 'shift' do in that function you posted?

SirDice: Are you saying that by using your example in the file name ls will automatically sort on it? I was always under the impression ls only sorted alphabetically and not numerically.
 
I missed the usage of creation date, sorry. It's still easy to foul it with cp etc.

local makes the variable in question not to leak out of the function. Not very important in this case, but ... just obsessive/compulsive. shift throws away initial arguments, 1 or more. Script arguments or function arguments, depending where it is used.

So you create the list of files, sorted by ls as needed, where you call the function and the function then ignores the initial part and removes the rest if there were more files than wanted. The `` is the same as $(), if that was not clear.

date +%FT%T gives a nicely formatted timestamp 2016-01-05T03:58:23 so things wont break in 3000 :)

Juha
 
Last edited by a moderator:
fullauto2012 said:
SirDice: Are you saying that by using your example in the file name ls will automatically sort on it? I was always under the impression ls only sorted alphabetically and not numerically.
Click to expand...
It does sort alphabetically. The 20160105 date format sorts alphabetically into the right numerical order, that's why it's so useful ;)
 
You must log in or register to reply here.
Back
Top