zfs backup script for backup to disk

[jbeez]

So I wrote a little shell script I run in cron once a week, and it works great when I send a snapshot to a new disk and even send incrementals to that disk afterwards, however if i change backup disks I can't change back without it bombing. The main issue I don't understand is how to send new incremental snapshots when the destination is missing the last or last few, can zfs resend any older snaps to catch it up?

http://pastebin.com/ecszdfzm


This is how it's setup right now, 3 1.5TB in a raidz and a single 2TB for backup. I currently have an unused 1.5 and 2tb drive, im about to rework my storage raidz to be 4 1.5's to expand it.

Any ideas on how I can have multiple drives? The idea is to take one offsite for a week, then swap every week between 2 2TB drives I have for backup. Also once my dataset exceeds using 2TB I'll need to figure out how to backup to multiple drives I can slide in and out of the server.

Right now I use an icydock trayless sata cage that sits in a cdrom bay. I can get another one they have that goes upto 4 drives at once, I guess I could slide a whole raidz pool of drives in and then import it....

Anyone have any other ideas for backup to disk?

 

[AndyUKG]

The main issue I don't understand is how to send new incremental snapshots when the destination is missing the last or last few, can zfs resend any older snaps to catch it up?

The answer is, you can't. You cannot zfs receive an incremental stream to a zfs volume where the original volume doesn't exist or doesn't have the base snapshot that is required by the incremental stream.

This is how it's setup right now, 3 1.5TB in a raidz and a single 2TB for backup. I currently have an unused 1.5 and 2tb drive, im about to rework my storage raidz to be 4 1.5's to expand it.

Any ideas on how I can have multiple drives? The idea is to take one offsite for a week, then swap every week between 2 2TB drives I have for backup. Also once my dataset exceeds using 2TB I'll need to figure out how to backup to multiple drives I can slide in and out of the server.

Right now I use an icydock trayless sata cage that sits in a cdrom bay. I can get another one they have that goes upto 4 drives at once, I guess I could slide a whole raidz pool of drives in and then import it....

Anyone have any other ideas for backup to disk?

Well you are using ZFS, so the way to go above 2TB is to use RAIDz for your backup disks too. If your requirement is to have disks off site then the only way I can think of to achieve this is that you have 2 complete copies of your data for backups. So one week you have onsite backup copy A, to which you can sync via zfs send/receive, which you then offsite and run to backup copy B the next week.

The concept I think you need to remember is that zfs send/receive doesn't function like a traditional backup system, that you send data to tape then off site, and spread backup data across multiple tapes etc. Zfs send/receive is for replicating a zfs volume or pool to another volume or pool. They are very different systems that have different advantages/disadvantages.

 

[jbeez]

Thanks, I didn't think there was a way :/ I'll just adjust my backup schedule to work a little differently I think. I can do more incrementals and swap the disks only once a month and then take the other offsite. So I erase the second disk and do a full backup every month. I'll have to just swap that to be a raid setup once my data needs grow and buy more disks.

 

[AndyUKG]

So I erase the second disk and do a full backup every month.

You dont have to erase the other disk in order to use it. If you send a disk offsite for a month then bring it back onsite to resync, you can do just that, resync without erasing everything and copying alllll your data from scratch. The requirement is only that the last snapshot you used to sync to said disk is still on the live ZFS copy. Then all you need to do is an incremental "zfs send" from the old snapshot to the lastest one.