ZFS Backup on external HDD (>= 12 TB)

S

spectrum48

Hi.

I backup my workstation over my home internal network to 2 old PCs. I use two independent backups and alternate between the two.

All three machines must be replaced, and I'm trying to avoid buying 3 new PCs but I need to backup a 12 TB HDD so I don't think external HDDs connected via USB o eSATA are a viable option.

I'm concerned about heat dissipation and connection stability during large backups and ZFS scrubs on such large disks. In the past I've often had problems with external disks randomly disconnecting.

I wonder how you handle situations like that. Thanks!
 
spectrum48 said:
In the past I've often had problems with external disks randomly disconnecting.
Click to expand...
Same here, and with geli and zfs encryption this always break the gpt setup and I need to reformat it on a linux machine (loosing all data on the process) because FreeBSD stop recognize the dist.
 
I hate USB but I can't say that I had much disconnecting trouble lately. Of course mini-USB is completely unacceptable. If you get all new stuff then USB-C should be possible and it is a lot better wrt stability. Just get quality cables.

The large disks don't dissipate more heat than smaller ones. Of course external drives could do with a fan pointed at them since they have no ventilation of their own.
 
From at leas 2 years, on 3 servers, I am using 5 external USB3 disks for backup without big problems. I use geli encryption on these disks.
In my office I use an 8TB Seagate Desktop and I have reformatted this disk for bad sectors some months ago. For now this is the only problem with 5 disks.
 
Good quality external disk (I use a Seagate in their enclosure), reasonable connection speed (USB-3), with a quality cable, and no problems. With USB-1 and no-name enclosures I had lots of problems.
 
Thank you all for sharing your experiences.

I can try a good quality external disk with its own enclosure and USB-3 to avoid connection propblems as suggested.

I'm still worried about temperature since scrubbing a 12 TB HDD takes ages (12-24 hour I think). A disk crunching for a whole day in a small enclosure without fans seem riski to me. When I used external disks temperature used to reach critical values under heavy load.
 
SirDice said:
I suspect you created a partition then proceeded to use the full disk instead of the partition for ZFS.
Click to expand...
The process I usually do is:

Code: 
# gpart create -s GPT /dev/da0
# geli init -s 4096 /dev/da0
# geli attach /dev/da0
# zpool create tank0 /dev/da0.eli
# zfs create -o encryption=on -o keyformat=passphrase -o compression=zstd -o atime=off tank0/zbkp
# zfs load-key tank0/zbkp
# zfs mount tank0/zbkp
 
BobSlacker said:
The process I usually do is:
Click to expand...
geli would overwrite the partition table in that order. Then you attach the entire drive to a zpool, also overwriting the partition table. That partition table is completely useless if you use the whole drive. And that's why it's constantly corrupted. Which isn't a big deal, it gives you some annoying errors. It has no influence on the zpool (geli encrypted or not) itself (except when you try to recover that partition table, thereby destroying geli and/or zfs data). If you want to use partitions (useful for identifying what's on the disk), then create a partition; gpart add -a 4k -t freebsd-zfs -l tank0 da0 and encrypt it; geli init -s 4096 /dev/da0p1. Then create the pool from the encrypted partition; zpool create tank0 /dev/da0p1.eli.

If you really want to use the whole disk, without a partition, that's fine too. Just gpart destroy da0 to make sure there's NO partition table on the disk at all.

And there's very little point in adding ZFS native encryption, on top of a pool that's on an encrypted disk/partition. Use one or the other; geli or ZFS native encryption.
 
emmex said:
In my office I use an 8TB Seagate Desktop and I have reformatted this disk for bad sectors some months ago. For now this is the only problem with 5 disks.
Click to expand...
Do you use ZFS on this disk? If you do, do you monitor disk temperature during scrubs? That's what worries me more about ZFS on large external disks with no fans.
 
spectrum48 said:
Do you use ZFS on this disk? If you do, do you monitor disk temperature during scrubs? That's what worries me more about ZFS on large external disks with no fans.
Click to expand...
Yes, I use ZFS on external disks (today only one one of them). And yes, I monitor their temperature with smart. And sadly, they do get warmer. But not much. For example, my current external backup disk (a small 2.5" Seagate) is usually at 37 degrees, and during scrub or running a full backup it goes up to 41. Neither number of cause for concern. YMMV.
 
I looked into external USB enclosures, and the deeper I looked, the more concerned I became about cooling. In the end, I could not find a device that I thought was likely to endure a 12 hour hammering. But I never tried any, so I defer to those who have.

I routinely zfs-send my 12TB tank to backup disks, but install them into a properly cooled a hot swap enclosure.

I have, in the past, also used naked SATA to USB adapters. The cabling was not terribly secure, and the disks walked all over the desk, but at least I could point a fan at them. It's a handy solution for casual activities. I still use them to build systems in the office to take off-site.

If I had to go external USB again today, I'd consider a docking station. The cables are reasonably secure and you can supply your own quality disks, and add your own desk fan. Watch your USB standards very carefully. You generally want "3.x Gen 2" at both ends for maximum bandwidth.
 
My experience is that they work. Last time the subject came up, somebody said that FreeBSD did not support UASP, suggesting slower in some contexts.
 
ralphbsz said:
And sadly, they do get warmer. But not much. For example, my current external backup disk (a small 2.5" Seagate) is usually at 37 degrees, and during scrub or running a full backup it goes up to 41. Neither number of cause for concern. YMMV.
Click to expand...
That would be perfect for me. My experience with external disks (WD and Seagate) have always been horrible: after some minutes of high I/O temperature was above 55 degrees.
 
gpw928 said:
I looked into external USB enclosures, and the deeper I looked, the more concerned I became about cooling. In the end, I could not find a device that I thought was likely to endure a 12 hour hammering.
Click to expand...

That's exactly where I'm stuck at!

gpw928 said:
I routinely zfs-send my 12TB tank to backup disks, but install them into a properly cooled a hot swap enclosure.
Click to expand...

I like it, I could buy a single backup PC and swap disks instead of buying 2 backup PCs.

Dumb question: I'm wondering if swapping disks every day will damage STATA and power connectors: are they designed to be used about 1000 times during each disk lifetime?

gpw928 said:
I have, in the past, also used naked SATA to USB adapters. The cabling was not terribly secure, and the disks walked all over the desk, but at least I could point a fan at them. It's a handy solution for casual activities. I still use them to build systems in the office to take off-site.

If I had to go external USB again today, I'd consider a docking station. The cables are reasonably secure and you can supply your own quality disks, and add your own desk fan. Watch your USB standards very carefully. You generally want "3.x Gen 2" at both ends for maximum bandwidth.
Click to expand...

They are both good suggestions too, thank you!
 
spectrum48 said:
Dumb question: I'm wondering if swapping disks every day will damage STATA and power connectors: are they designed to be used about 1000 times during each disk lifetime?
Click to expand...
That's an excellent question, but I'll leave it to others to answer as I have have never exchanged disks at that sort of duty cycle.
 
spectrum48 said:
That would be perfect for me. My experience with external disks (WD and Seagate) have always been horrible: after some minutes of high I/O temperature was above 55 degrees.
Click to expand...
That probably depends on disk and enclosure. My external backup disk is small (2.5", 2TB), slow (5400 RpM), in a matching enclosure (designed as external USB by the disk manufacturer), and designed for backup workloads. Depending on what disk and enclosure you use, YMMV.
 
You must log in or register to reply here.
Back
Top