ZFS Automate ZFS snapshot backups to another drive

Buck

Buck

So there are several utilities in ports that provide convenient snapshot management, but I haven't been able to find one that would similarly automate cloning and managing those snapshots to a different drive/zfs pool installed in the same machine.
What I'm looking to have is not a clone of the original ZFS datasets selected for backup but rather a directory tree that looks like

/backuppool/mainpoolbackups/usr/home/.zfs/snapshot-01.01.2020/
/backuppool/mainpoolbackups/usr/home/.zfs/snapshot-10.01.2020/
...

I know it's possible to send specific snapshots from one pool to another manually, but is there a way to automate the process using some kind of utility that would look for current snapshots on the main pool, compare them to snapshot backups on the backup pool and sync those so that those pools contain identical snapshots? Maybe there's another approach I should try to achieve what I'm looking for?
 
I want to do the same thing on my FreeNAS Server but I'm not sure if it will work...

I setup a "periodic snapshot task" in FreeNAS which will create 1 snapshot each day "HDDpool/HDD-Share@auto_xyz" and delete snapshots older than 1 week so I am protected against ransomware.

But I also want to run a backup script manually to replicate the datasets to a USB-HDD once every 1 oder 2 months like in the Script T-Daemon posted.
How is that working together with the periodic snapshots which will be created and also deleted between two backups?

Is it unimportent If other snapshots are created or deleted as long as the manually created backup snapshots are untouched?

Like here?:

Initial backup...
zfs snapshot "HDDpool/HDD-Share@backup_Initial"
zfs send "HDDpool/HDD-Share@backup_initial" | zfs recv "backuppool/HDD-Share_backup"
... one month later while 30 "HDDpool/HDD-Share@auto_xyz" snapshots were created and 23 "@auto" Snapshots already deleted...
zfs snapshot "HDDpool/HDD-Share@backup_2"
zfs send -i "HDDpool/HDD-Share@backup_initial" "HDDpool/HDD-Share@backup_2" | zfs recv "backuppool/HDD-Share_backup"
zfs destroy "HDDpool/HDD-Share@backup_Initial"
...some weeks and dozens of created and deleted "@auto" snapshots later...
zfs snapshot "HDDpool/HDD-Share@backup_3"
zfs send -i "HDDpool/HDD-Share@backup_2" "HDDpool/HDD-Share@backup_3" | zfs recv "backuppool/HDD-Share_backup"
zfs destroy "HDDpool/HDD-Share@backup_2"
 
Don't know why exactly but I wasn't able to get Znapzend to work properly. Snapshots are replicated but their contents isn't matched with the source snapshot contents. I see a .zfs/snapname-date/... folders with correct names but inside the contents is all the same (seemingly unchanged from some previous snapshot), even though original .zfs folders on the main drive all have correct contents.
 
Buck said:
Don't know why exactly but I wasn't able to get Znapzend to work properly. Snapshots are replicated but their contents isn't matched with the source snapshot contents. I see a .zfs/snapname-date/... folders with correct names but inside the contents is all the same (seemingly unchanged from some previous snapshot), even though original .zfs folders on the main drive all have correct contents.
Click to expand...
Hmm that's strange. I get what you're saying. Settings and can you post some examples?
 
A rather old request, but better late then never

Buck said:
So there are several utilities in ports that provide convenient snapshot management, but I haven't been able to find one that would similarly automate cloning and managing those snapshots to a different drive/zfs pool installed in the same machine.
What I'm looking to have is not a clone of the original ZFS datasets selected for backup but rather a directory tree that looks like

/backuppool/mainpoolbackups/usr/home/.zfs/snapshot-01.01.2020/
/backuppool/mainpoolbackups/usr/home/.zfs/snapshot-10.01.2020/
...

I know it's possible to send specific snapshots from one pool to another manually, but is there a way to automate the process using some kind of utility that would look for current snapshots on the main pool, compare them to snapshot backups on the backup pool and sync those so that those pools contain identical snapshots? Maybe there's another approach I should try to achieve what I'm looking for?
Click to expand...
If someone want to "freeze" some snapshots forever, with optional restore of files one by one, it is possible to try a somewhat radical approach, storing snapshots inside a single archive, by the mighty (I am joking of course) zpaqfranz
github.com

GitHub - fcorbelli/zpaqfranz: Deduplicating archiver with encryption and paranoid-level tests. Swiss army knife for the serious backup and disaster recovery manager. Ransomware neutralizer. Win/Linux/Unix

Deduplicating archiver with encryption and paranoid-level tests. Swiss army knife for the serious backup and disaster recovery manager. Ransomware neutralizer. Win/Linux/Unix - fcorbelli/zpaqfranz
github.com github.com
with the zfsadd command.

The archive can be stored everywhere, just like a .tar or .7z or whatever, with optional encryption (something not easy to do with replicas).

The purpose is to maintain the data without time limit, just forever (aka: a backup policy for very valuable files), NOT a somewhat "hacked" zfs replica-superscript, that will require, sooner or later, purging of older snaps.
 
I spent several days trying to implement this and just now ended up with this automatic script. So far everything works. Script just placed in /usr/local/etc/periodic/daily
on my server all jails placed in one dataset jails/containers
jails/containers/jail_one
jails/containers/jail_two

after script run one snapshot steel present in source dataset. It needed and always be replaced to latest by script.

sh: 
jails                                       90.5G  1.67T   128K  /jails
jails/containers                            83.8G  1.67T  83.8G  /jails/containers
jails/containers@2025-09-09_09-22-16        11.6M      -  83.8G  -
jails/media                                  197M  1.67T   197M  /jails/media
jails/services                              5.93G  1.67T  5.93G  /jails/services
jails/templates                              450M  1.67T   450M  /jails/templates
opt                                         86.7G  7.06T   112K  /opt
opt/backups                                 86.6G  7.06T   132K  /opt/backups
opt/backups/containers                      84.8G  7.06T  83.8G  /opt/backups/containers
opt/backups/containers@2025-09-09_08-10-49  13.4M      -  84.2G  -
opt/backups/containers@2025-09-09_08-11-00  12.4M      -  84.2G  -
opt/backups/containers@2025-09-09_08-13-59  60.7M      -  83.8G  -
opt/backups/containers@2025-09-09_08-19-05  56.9M      -  83.8G  -
opt/backups/containers@2025-09-09_09-00-00   107M      -  83.8G  -
opt/backups/containers@2025-09-09_09-22-16     0B      -  83.8G  -

Script containers-backup (modified, optimized)
sh: 
#!/bin/sh

# === Configuration ===
JAILS_DATASET="jails/containers"
BACKUP_DATASET_FILES="opt/backups/containers"
TIMESTAMP=`date +%Y-%m-%d_%H-%M-%S`
RETENTION_DAYS=120

# === Step 1: Prepare the ZFS structure ===
echo "Checking and creating datasets for backups..."
zfs create -p "${BACKUP_DATASET_FILES}"

# === Step 2: Backup ===
# Determine the new snapshot name once
NEW_SNAPSHOT_SOURCE="${JAILS_DATASET}@${TIMESTAMP}"
echo "Create a ZFS snapshot for the dataset '${JAILS_DATASET}'..."
zfs snapshot "${NEW_SNAPSHOT_SOURCE}"

# Get the latest snapshot on the target pool.
LAST_SNAPSHOT_ON_TARGET=`zfs list -t snapshot -o name "${BACKUP_DATASET_FILES}" | tail -n 1`

if [ -n "${LAST_SNAPSHOT_ON_TARGET}" ]; then
    # If there is at least one snapshot on the target pool, make an incremental backup
    SNAPSHOT_NAME=$(echo "${LAST_SNAPSHOT_ON_TARGET}" | cut -d'@' -f2)
    PREVIOUS_SNAPSHOT_SOURCE="${JAILS_DATASET}@${SNAPSHOT_NAME}"
   
    echo "Parent snapshot found. Sending incremental backup."
    zfs send -i "${PREVIOUS_SNAPSHOT_SOURCE}" "${NEW_SNAPSHOT_SOURCE}" | zfs recv -F "${BACKUP_DATASET_FILES}"
   
    wait
    if [ $? -ne 0 ]; then
        echo "Error! Incremental backup failed."
        echo "Please manually check and fix the pool: '${BACKUP_DATASET_FILES}'."
        exit 1
    fi

    echo "Removing the old snapshot from the source pool."
    zfs destroy "${PREVIOUS_SNAPSHOT_SOURCE}"
else
    # If there are no snapshots on the target pool, make a full backup
    echo "Parent snapshot not found or this is the first run. Sending a full backup."
    zfs send -R "${NEW_SNAPSHOT_SOURCE}" | zfs recv -F "${BACKUP_DATASET_FILES}"
   
    wait
    if [ $? -ne 0 ]; then
        echo "Error! Full backup failed."
        exit 1
    fi
fi

# === Step 3: Clean up old backups on the target disk ===
RETENTION_SECONDS=$((RETENTION_DAYS * 24 * 60 * 60))
CURRENT_TIME=`date +%s`
echo "Deleting old backups in 'opt/backups/containers'..."
zfs list -t snapshot -o name -r "${BACKUP_DATASET_FILES}" | tail -n +2 | while read snap; do
    create_time=`zfs get -H -p -o value creation "${snap}"`
    ELAPSED_SECONDS=$((CURRENT_TIME - create_time))
   
    if [ "$ELAPSED_SECONDS" -gt "$RETENTION_SECONDS" ]; then
        echo "Delete the old snapshot: ${snap}"
        zfs destroy "${snap}"
    fi
done

echo "Backup and cleanup completed."

Good luck
 
I use zrepl from ports exactly for this. It tkes periodic snapshots of everything I need and when I want backups I connect my external NVME enclosure and send those snapshots there. Works perfectly and it's my backup solution from day one on Freebsd.
 
You must log in or register to reply here.
Back
Top