Opinions please: ZFS auto-snapshot design

F

ffarn

Now that I have ZFS up and running I would like to implement an automated snapshot mechanism. I have looked at zfs-snapshot-mgmt, but for my purposes I believe there is a simpler solution based on this sun.com blog. Being a ZFS neophyte, I thought I'd get the opinions of some ZFS experts before proceeding.

My solution would make use of ZFS's ability to set user-defined parameters. Users would add a parameter "autosnap:<label>"=<count> to filesystems for which they want auto-snapshots. <label> is a user defined name and <count> is the number of snapshots with that label to rotate through. This would create automatic snapshots named <label>.0, <label>.1, etc. Every time a new snapshot is created its label is incremented and any snapshots with labels numbered <count> or higher are destroyed. The newest snapshot is always <label>.0, and the oldest snapshot would be <label>.<count-1>.

I would write a single script "zfs-autosnap.sh <label> <filesystem>|all" which would rotate the <label> snapshots for each <filesystem>. Filesystems that do not have an autosnap:<label> are ignored. The special filesystem "all" would represent all filesystems.

Users could then add some crontab entries that would regularly run zfs-autosnap <label> on a regular basis.

EXAMPLE:

The following example would cause 24 hourly snapshots to be kept for filesystem "zroot/home":
Code: 
zfs set autosnap:hourly=24 zroot/home

The following crontab entries would rotate the "hourly" snapshots every hour, "daily" every day, etc.
Code: 
0 * * * * root /root/bin/zfs-autosnap.sh hourly all
0 0 * * * root /root/bin/zfs-autosnap.sh daily all
0 0 * * 0 root /root/bin/zfs-autosnap.sh weekly all
0 0 1 * * root /root/bin/zfs-autosnap.sh monthly all

Note that by default, any child filesystems would inherit the same autosnap configuration as their parents. Snapshot filesystems, which would also inherit the parameters from their parents, would be ignored.

Seems simple to implement, simple to administer. Seems to be in keeping with the intent of the zfs features used. What do the ZFS experts out there think?
 
I'm not an expert per se, but I have been considering doing something like this myself in perl. Fundamentally it's not really that tough to do, but making it function in a convenient and reliable fashion takes a bit of thought.

It's probably easier to do this sort of thing via Perl or a more proper programming language as you can push and pop snapshots rather than having to iterate through them each time you want to remove a snapshot.

And rather than using parameters, I'd probably use a standalone file to control it. The main reason being that IMHO it's better to keep as much of the configuration stuff in the same place as humanly possible.
 
Here's the code

Well, since nobody has really tried to talk me out of it, here's the code. Feel free to implement in the language of your choosing, but /bin/sh is always installed. ;)

This uses zfs properties, but a separate config file is a reasonable suggestion. You would have to deal with property inheritance yourself though since the below script relies on ZFS to do that.

zfs_autosnap.sh:
Code: 
#!/bin/sh

usage() {
  cat <<EOF
usage: ${0##*/} <label> <filesystem>|all
  Rotates filesystem snapshots.  If filesystem has the property
  org.freebsd:snap:<label>=<count>, increments all existing snapshots labeled
  <label>.<num> and creates a new snapshot called <label>.0.  Destroys snapshot
  numbered <label>.<count>.
EOF
  exit
}

# Default values
: ${ZFS=/sbin/zfs} ${PROP=org.freebsd:snap}

# Command line switch: -n for testing
case "$1" in
-n) ZFSd="echo ${ZFS}"; shift ;;
*) ZFSd="${ZFS}"
esac

# Parse command line
LABEL=$1
shift || usage

notsnap() {
  [ "$($ZFS get -Ho value type "$1" 2>/dev/null)" != "snapshot" ]
}

# For each filesystem
while [ $# -ne 0 ]; do
  # Parse filesystem for special "all" name
  case "$1" in
  all) fs="" ;;
  *) fs=$1
  esac

  # Query ZFS for filesystems with the necessary property
  $ZFS get -H ${PROP}:${LABEL} $fs | while read N P V S; do
    [ "$V" -gt 0 ] 2>/dev/null && notsnap "$N" || continue

    # Destroy the oldest
    V=$(($V - 1))
    SNAP="${N}@${LABEL}"
    notsnap ${SNAP}.${V} || $ZFSd destroy ${SNAP}.${V}

    # Increment existing snapshots
    while [ $V -gt 0 ]; do
      next=$(($V - 1))
      notsnap ${SNAP}.${next} || $ZFSd rename "${SNAP}.${next}" "${SNAP}.${V}"
      V=$next
    done

    # Make New Snapshot
    $ZFSd snapshot ${SNAP}.0
  done
  shift
done

To use, either put something in /etc/crontab or you can use the periodic files daily.local, weekly.local, and monthly.local. I'm not sure which the FBSD advocates would prefer but since there is no hourly.local script that gets run, so if you want hourly snapshots you'll have to at least put that directly into crontab.

I use the following script for all of them:
*.local:
Code: 
#!/bin/sh

/root/bin/zfs_autosnap.sh $(/usr/bin/basename "$0" .local) all
This figures out which label to use by looking at the name the script was invoked with. Remember to make the script executable.

To turn on regular snapshots, I just set the parameter(s) for the filesystem:
Code: 
% zfs set org.freebsd:snap:daily=7 zroot/home
% zfs set org.freebsd:snap:weekly=5 zroot/home
 
This looks interesting. It's clean and simple and doesn't require any additional languages. I also like the use of properties to mark which filesystems to snapshot.

Presumably you've been using this for a while? How's it been going? Have you made any changes?

Have you thought about making a port for it? I know it's small, but it makes it more accessible.
 
tdb,

Thanks for asking. I've been using this exactly as written since the day I posted it. It works well.

I'd be happy to make a port of it... if I knew how. It shouldn't be hard. I'd probably have to generalize the weekly/daily/monthly.local setup, but that is pretty trivial. Right now I just link them all to the same place but that wouldn't work if someone was using those scripts for other tasks.
 
Looks very neat, Would it be possible to not rotate out the first snapshot taken? ie @daily.0 stays as is, a base snapshot if you like rather than being incremented to @daily.1 or is it just simpler to take a manual snaphot @14072010 for example and let your script continue doing its thing?
 
The script could be modified. I'm not saying functionality like that couldn't be added, but I think it would be cleaner to do that separately.

This script manages rotating snapshots. Static snapshots should probably be managed elsewhere. As you point out, static snapshots are naturally easy to handle manually, so no new system need be developed. If you come up with something, I'd be interested to see it, though.
 
I agree, static snapshots should be separate.

Would it be worth changing your property name to org.freebsd:autosnap ?

Regarding making a port, it might be easier to just provide a section of text to paste in to the crontab. Some people (myself included) might want hourly snapshots and the periodic scripts don't do that. The freebsd-snapshot port does it this way.

The script itself could just be included within the port like the portmaster port does.

I'm not completely sure if it's worth making a port for, but it would make it easy for people to find and use. If you fancy giving it a stab the porters handbook has lots of useful information, and I'd be happy to help.
 
Reading through the porter's handbook one gets discouraged putting this in as a port. Building a port may make this more accessible, but considering how this is just one small script, all the overhead of producing a port looks like it would be 5-10x the amount of work that writing the script was. Ports seem intended for much larger packages than just a single script w/ some cron edits.

That said, two quick questions:
1) This is probably a simple one, but how would I make a port that doesn't need to download a source tarball? autosnap.sh is a single script and it is too short to bother to set up an ftp site to carry it.
2) I'm not sure how to get a port to update crontab. It seems there are different flavors of cron out there that use different formats for crontab, i.e., cron.d vs. crontab. For those using just a single crontab file, editing that file automatically in a robust way seems very daunting.
 
Yeah, portmaster is an example of a port that just includes it in the files directory, so no need to set up a site for it.

Also, I wouldn't bother trying to automatically set up the crontab. Just print out something like "Put the following lines in /etc/crontab: ...", like the freebsd-snapshot port does.

All the port would do is install /usr/local/sbin/zfs-autosnap and give some appropriate lines for /etc/crontab.
 
You must log in or register to reply here.
Back
Top