ZFS gzip --rsyncable option and zfs send

M

maxxfi

I plan to have a large gzip tarfile (a gitlab backup tarball, actually) that I will further replicate to a second backup location using the zfs send/receive mechanism.
The gzip gets regenerated every day. In order to reduce the incremental zfs replication transfers, would it benefit to create it with the --rsyncable option, or is that option only useful to rsync transfers?
I'm not familiar with the logic of how zfs calculates the incremental change, hence the question.

Thanks
 
zfs keeps track of the blocks that changed of a file and resend only those with an incremental zfs send.
As you regenerate the file complete probably all blocks are changed.
Use zfs send -v to see how much data is send actual.
Do you really need to create a zipped tarball ? Zfs knows compression.
I use zfs send incremental on a directory.
 
grahamperrin said:
How large, approximately, are the file and the containing dataset?



How frequently?
Click to expand...
File is about 100 GB and dataset has space for 2 TB, and there is a daily replication but anyway...
I did further tests and I don't think there is much to do.

Gitlab backup files are essentially tar archives of 'git bundles' of every repo in the gitlab server (there are maybe 20-30 of smaller ones, but the bulk is made by a 60 GB repo).
Starting from the backups generated on two consecutive days, when used as snapshots the incremental difference for zfs send is essentially the same as the whole backup.
I also tried to untar the older backup, snapshot, untar on top of it the newer one, snapshot again and check the diff and still it offers no savings from transferring the file as a whole, whether with zfs send or with rync, probably by the nature of the git bundle (which besides doesn't show any gain even in compressing it).

Thanks for your interest in my question, but at this point I'll work in some other way to optimize the transfer to the 2nd backup location.
 
You must log in or register to reply here.
Back
Top