ZFS retrieval of incremental backups

A couple of questions. I'm still in the testing phase of things, but want to solidify my understanding. I want to perform incremental backups to an off-site drive.

There are three datasets:
  • zfs create tank/user/norton
  • zfs create tank/user/wife
  • zfs create tank/user/kids

Sunday:
zfs snapshot -R tank/user@fullSunday
zfs send tank/user@fullSunday > /offsite/fullSunday

Monday:
zfs snapshot -R tank/user@incremMonday
zfs send -i tank/user@fullSunday tank/user@incremMonday > /offsite/incremMonday

Tuesday:
zfs snapshot -R tank/user@incremTuesday
zfs send -i tank/user@fullSunday tank/user@incremMonday > /offsite/incremTuesday

And so on until next Sunday when I do a full backup again.

If I want to retrieve a file that was created in tank/kids on Tuesday, but deleted on Wednesday I would do something like:
zfs create tank/restore
zfs recv -F tank/restore < /offsite/fullSunday
zfs recv tank/restore < /offsite/fullMonday
zfs recv -F tank/restore < /offsite/fullMonday
Then I can go to tank/restore and grab the file.

Sorry, if this had been long but I wanted to be thorough.

My questions:
  1. Is there a better way to get to the incremental on Tuesday? Currently, I am having to do three recv's. Worse if it was on Friday.
  2. Is there a way to just restore a file from a dataset like tank/wife, rather than restoring the entire tank pool? I know I could do the send/recv on just a dataset (instead of -R tank) but if I have 50 datasets, I don't want that many files.
  3. Is there a better way to do this overall?
Thank you for taking the time to read this. It is appreciated.

Bill
 
bnorton916 said:
Is there a better way to get to the incremental on Tuesday? Currently, I am having to do three recv's. Worse if it was on Friday.
An important aspect to keep in mind is that whenever you create a snapshot it remains in place until you destroyed it. So assuming that tank/user/norton is mounted as /home/norton you could easily retrieve your stuff from Tuesday by doing something like this:

Code:
# cd /home/norton/.zfs/snapshot/incremTuesday
# cp $file /home/user/norton/.
There's no direct need to receive and re-import or re-apply the external backup. And because ZFS uses a pool and shares it's data I'd suggest to try this approach first, it makes quickly retrieving a backup a lot easier. Personally I only keep off-site backups as my last means of defense, only to be used when there's no other option available.

bnorton916 said:
Is there a way to just restore a file from a dataset like tank/wife, rather than restoring the entire tank pool?
See above. Whenever you're importing a filesystem back into your pool you can opt to keep it under a different mount point so that you can then access it.

But this is also the caveat of using 'external' snapshots (in my opinion); you can only access them after you imported the whole file system back into your pool. Which can become an issue if you don't have much free space left.

bnorton916 said:
Is there a better way to do this overall?
Better is of course a very personal aspect.

I tend to make snapshots and keep them for a week, then the snapshot gets destroyed again. The good part here is that only changes get stored, ergo it may cost you a lot less space than you might assume right now.

For example, this is what my backup of /var/log looks like:

Code:
$ zfs list -rt all zroot/var/log
NAME                   USED  AVAIL  REFER  MOUNTPOINT
zroot/var/log          558M   111G   550M  /var/log
zroot/var/log@071013   910K      -   302M  -
zroot/var/log@081013   838K      -   307M  -
zroot/var/log@091013   853K      -   310M  -
zroot/var/log@101013   874K      -   313M  -
zroot/var/log@111013   896K      -   317M  -
zroot/var/log@121013   848K      -   321M  -
zroot/var/log@131013  1.01M      -   324M  -
For the record; this is on a rather busy server, most of the space mentioned here is consumed by logfiles from Apache. But this approach also allows me to quickly access anything on a specific day.

Maybe you'll find this useful; I wrote a script to handle the whole snapshot rotation. Although zfs(8) provides an example where a snapshot is created, the other destroyed and the rest renamed I didn't like that approach at all because when it comes to backups (and filesystems in general) I want to keep the required access to a minimum. So I came up with this:

Code:
#!/bin/sh

## Snapshot.ZFS v1.0
##
## A script which will manage ZFS snapshots on the
## filesystem(s) of your choosing.

### Configuration section.

# ZFS pool to use.
POOL="zroot";

# Filesystem(s) to use.
FS="/home /usr/local /var"

# Retention; how many snapshots should be kept?
RETENTION=7

# Recursive; process a filesystem and all it's children?
RECURSE=yes;

### Script definitions <-> ** Don't change anything below this line! **

CURDAT=$(date "+%d%m%y");
PRVDAT=$(date -v-${RETENTION}d "+%d%m%y");
PROG=$(basename $0);

if [ ${RECURSE} == "yes" ]; then
        OPTS="-r";
fi

### Script starts here ###

if [ "$1/" == "/" ]; then
        echo "Error: no command specified."             > /dev/stderr;
        echo                                            > /dev/stderr;
        echo "Usage:"                                   > /dev/stderr;
        echo "${PROG} y : Manage snapshots."            > /dev/stderr;
        echo                                            > /dev/stderr;
        exit 1;
fi

if [ "$1" == "y" ]; then

        # Make & clean snapshot(s)
        for a in $FS; do
                ZFS=$(zfs list -r ${POOL} | grep -e "${a}$" | cut -d ' ' -f1);
                if [ "$ZFS/" == "/" ]; then
                        echo "${PROG}: Can't process ${a}: not a ZFS filesystem." >/dev/stderr;
                else
                        $(zfs snapshot ${OPTS} ${ZFS}@${CURDAT} > /dev/null 2>&1) || echo "${PROG}: Error creating snapshot ${ZFS}@${CURDAT}" > /dev/stderr
                        $(zfs destroy ${OPTS} ${ZFS}@${PRVDAT} > /dev/null 2>&1) || echo "${PROG}: Error destroying snapshot ${ZFS}@${PRVDAT}" > /dev/stderr
                fi
        done;
else
        echo "Error: wrong parameter used."             > /dev/stderr;
        echo                                            > /dev/stderr;
        echo "Usage:"                                   > /dev/stderr;
        echo "${PROG} y : Manage snapshots."            > /dev/stderr;
        echo                                            > /dev/stderr;
        exit 1;
fi

Edit:

If you plan on trying out the script but don't want to take any risks then my advice is to comment out the line with "$(zfs destroy ${OPTS}". You can do so by placing a # in front of it. This will make sure that snapshots only get created but not destroyed. I did add some failsaves into the script but obviously it best to be save than sore :)

Also, when looking at this script again I also noticed that in my environment the user who executes this (root in my case) needs to have /bin and /sbin in its PATH. Normally those should already be present, but I figured I'd mention it anyway.
 
ShelLuser said:
[An important aspect to keep in mind is that whenever you create a snapshot it remains in place until you destroyed it. So assuming that tank/user/norton is mounted as /home/norton you could easily retrieve your stuff from Tuesday by doing something like this:
Code:
# cd /home/norton/.zfs/snapshot/incremTuesday
# cp $file /home/user/norton/.

Ah, I knew the snapshot information was stored in .zfs but I didn't realize that I could actually go in and grab the files there. This makes things much easier. I will still need to store the snapshots off-machine, but I don't mind going through a bunch of steps if it is something I will probably never have to do. This also makes having to mount the whole filesystem a non-issue for me. I will probably never have to do it, so being an arduous task is not a big deal.

ShelLuser said:
[
Maybe you'll find this useful; I wrote a script to handle the whole snapshot rotation. Although zfs(8) provides an example where a snapshot is created, the other destroyed and the rest renamed I didn't like that approach at all because when it comes to backups (and filesystems in general) I want to keep the required access to a minimum.

Thanks for this. I will certainly look at it and may use it.

I appreciate you taking the time to answer my questions.

Bill
 
Back
Top