ZFS Simple zfs replication script

M

mokfarg

Hi all!
I presently manually perform replication pull from my backup server from my nas. Works great. I started exploring script options and either they are super elaborate or they seem out of date and do not work properly. One I liked that seemed simplistic that I tested was zxfer, however if I add -o readonly=on or -P it fails. From the github doesn't look like it's staying up to date with the newer versions of openzfs. I looked at others such as sanoid/syncoid, zrep, zrepl etc however they seem a bit elaborate. Openzfs has the native commands already, the more elaborate the more the chance in the future it doesn't work when Openzfs updates, also more room for another layer of bugs on top of zfs commands. Any suggestions on a simple replication script?

(Bonus, would like a diff log for the datasets as well automated preferably appended to one log)

This is my manual commands I ran initially then on going once a day:

Code: 
#@nas server
zfs snapshot -r zpool0@$(date +%Y%m%d)

#@backup server
Code: 
#first run
ssh 192.168.0.1 zfs send -LRPv zpool0@$(date +%Y%m%d) | zfs receive -o readonly=on zpool0/nas
#following runs
ssh 192.168.0.1 zfs send -RvI zpool0@$(date -v -1d +%Y%m%d) zpool0@$(date +%Y%m%d) | zfs receive zpool0/nas

#bonus diff each day!
zfs diff -HF zpool0@$(date -v -1d +%Y%m%d) zpool0@$(date +%Y%m%d) >> /var/log/zfs/zfs-diff-zpool0.$(date +%Y%m%d.log)
zfs diff -HF zpool0/dataset1@$(date -v -1d +%Y%m%d) zpool0/dataset1@$(date +%Y%m%d) >> /var/log/zfs/zfs-diff-zpool0-dataset1.$(date +%Y%m%d.log)
zfs diff -HF zpool0/dataset2@$(date -v -1d +%Y%m%d) zpool0/dataset2@$(date +%Y%m%d) >> /var/log/zfs/zfs-diff-zpool0-dataset2.$(date +%Y%m%d.log)

This is the zxfer command I tried to execute however couldn't get it work with Freebsd 14

Code: 
zxfer -Pv -O 192.168.0.1 -o readonly=on -R zpool0 zpool0/nas
 
Use the zxfer command with -I option.
Code: 
zxfer -I objsetid,keystatus,snapshots_changed,pbkdf2iters,special_small_blocks,encryptionroot,keystatus,keyformat,keylocation \
    -Pv -O 192.168.0.1 -o readonly=on -R zpool0 zpool0/nas

I hope the developer will be back to look at the issues and pull requests, but it is what we can do for now.
 
I had another thought, I noticed some automated snapshot apps follow a conventional backup strategy something like : 24 hourly, 14 daily, 8 weekly, 12 monthly, 2 yearly. Why is this? I am just curious, is it due to just to lower the number of snapshots to eliminate confusion? As far as I can tell space wise, doing this and just keeping a daily or even hourly for years isn't much difference.

edit: I did some reading I guess there are some performance issues when trying to zfs list many snapshots. I read that 10,000, you may start seeing some issues. That would still be okay for a year of hourly.
 
mokfarg said:
something like : 24 hourly, 14 daily, 8 weekly, 12 monthly, 2 yearly. Why is this?
Click to expand...
I think it's also to do with what you might need the backup for.

If someone mis-types a command (deletes files or database records) the mistake will be apparent almost immediately, so it's great to have the intra-day hourly backups to recover from.

Sometimes code with a bug goes into production, and the mistake isn't noticed for a few days or weeks - that's when those weekly or 14 day backups become useful.

Finallly if you are infected with malware or ransomware - you can go even further back in time - having those one-per-month means you can rewind to see how/where things went wrong, and at least you can recover SOME data.
 
You can set the number of snapshots with zfs-periodic.
Code: 
hourly_zfs_snapshot_enable="YES"
hourly_zfs_snapshot_pools="tank"
hourly_zfs_snapshot_keep=8800

Setting above in /etc/periodic.conf will keep hourly snapshots about a year. As long as the number is in the limit of an int, it should work.
But it is us, human beings that have difficulty dealing with them. Anyway you may always try.
 
My simple attempt at a script to zfs pull replicate from a nas server that has the zfs-periodic script installed and running with snapshots taken daily. My thought is if I replicate yesterday's snapshot that will give ample time for them to complete on source. Seems to work okay as long as the source has the snapshots present. I'll put it in a cron to run every day at 1am. Any advice or tips to make it better?

sh: 
#!/bin/sh
#snapshots
snap1=daily-$(date -v -2d +%Y-%m-%d)
snap2=daily-$(date -v -1d +%Y-%m-%d)
#replicate
ssh 192.168.0.1 zfs send -Rvi zp0@$snap1 zp0@$snap2 | zfs receive zp0/nas
#diff
#zfs diff -HF zp0/nas/ds0@$snap1 zp0/nas/ds0@$snap2 > /var/log/zfs-diff-zp0_nas_ds0.$snap2
#zfs diff -HF zp0/nas/ds12@$snap1 zp0/nas/ds12@$snap2 > /var/log/zfs-diff-zp0_nas_ds12.$snap2
#zfs diff -HF zp0/nas@$snap1 zp0/nas@$snap2 > /var/log/zfs-diff-zp0_nas.$snap2
#destroy any snapshot older than 1 year
zfs list -t snapshot -o name -S creation | grep zp0/nas@ | tail -n +366 | xargs -n 1 zfs destroy -rv
 
I may try to figure out how to email me the diffs and maybe any type of standard error if it fails. Would be nice if I knew how to check if the snapshots were present before execution.
 
mokfarg said:
I may try to figure out how to email me the diffs and maybe any type of standard error if it fails. Would be nice if I knew how to check if the snapshots were present before execution.
Click to expand...

If you put it in cron, the output will be emailed.
For the way to treat snapshot inexistence error, see sysutils/zxfer ( https://github.com/allanjude/zxfer ) (or just use it). Note that zxfer is behind zfs update and need to tell the propeties to skip copying by -I option.
 
Unfortunately I am still getting an error message when using zxfer shown below. I verified the dataset in question does not exist on destination. It appears zxfer is creating the dataset then erroring out stating it exists at some other step in the code.

sh: 
zxfer -I objsetid,keystatus,snapshots_changed,pbkdf2iters,special_small_blocks,encryptionroot,keystatus,keyformat,keylocation -Pv -O 192.168.0.1 -o readonly=on -R zpool zpool/nas
Creating destination filesystem "zpool/nas/zpool" with specified properties.
Sending zpool@hourly-2024-03-20-20 to zpool/nas/zpool.
cannot receive new filesystem stream: destination 'zpool/nas/zpool' exists
must specify -F to overwrite it
Error when zfs send/receiving.
 
As the error message says, you have to specify -F option for the first time to overwrite existing file systems. From the second try, it should go without -F option.
 
-F allowed me to replicate. I am using large block sizes of 1M. It seems to have transferred properly. I assume zxfer supports send/ receive large block sizes.
 
You must log in or register to reply here.
Back
Top