Dual ZFS mirror vs Raid-Z2 vs Raid-Z3

L

luckylinux

I have 6 x 2TB (2 x Samsung, 2 x Western Digital, 2 x Hitachi) I want to dedicate to my NAS for my personal data (no media!).

I plan to use encryption (GELI) with them and don't know if one of them is more suited than the others.
I would like to ask what partitioning scheme would be better to use for reliability or if one of them is definitively "overkill":
  1. Zpool with 1 x MIRROR (Samsung, WD, Hitachi) + 1 x MIRROR (Samsung, WD, Hitachi)
    This would give me 2 x 2TB = 4TB of storage roughtly with 2 HDDs of redundancy in each mirror allowing up to 4 consecutive disk failures. Maybe that's a bit overkill, but should provide good performance.
  2. Zpool with 1 x RAIDZ2 (2 x Samsung, 2 x WD, 2 x Hitachi)
    This would give me (6 - 2) x 2TB = 4 x 2TB = 8TB of storage roughtly with 2 HDDs of redundancy. This may be the best solution.
  3. Zpool with 1 x RAID-Z3(2 x Samsung, 2 x WD, 1 x Hitachi)
    This would give me (5-3) x 2 TB = 2 x 2TB = 4TB of storage roughtly with 3 HDDs of redundancy. Performance may be lower than the solutions above.

If you wonder why I bought disks from different brands (and for WD: even different models) it's because to lower the risk of simultaneous disks failures. I heard it's not so good to buy say 6 HHDs of same brand & model at the same time (even consecutive S/N) for use in RAID.

I already disabled the Intellipark feature of these WD Green / AV-GP drives. One of them already had LLC ~ 40000.

Based on this, which configuration would you reccomend?

For use with GELI, any good how-to related to raid-Zx / mirror (expecially what to do when a drive fails and needs to be replaced)?

Thank you very much.
 
Two 3 disk mirrors are overkill for a small home NAS but they are reliable and offer high read speed. If you have the disks and 4 TB usable storage is enough go for it. You left out the option of three 2 disk mirrors. Do you have any other copy of the data?
 
Crest said:
Two 3 disk mirrors are overkill for a small home NAS but they are reliable and offer high read speed.
Click to expand...
I don't really care about high speed. By "Data" I mean mostly text documents (Word / LaTEX), some "small" programming stuff (C++ or the likes, but nothing huge) or personal pictures (~ 10MB / file max). In my case speed isn't really that important for what concerns my "Data".

I'll make another zpool for media: in that case speed will be an issue to be addressed properly.

Crest said:
If you have the disks and 4 TB usable storage is enough go for it.
Click to expand...
At the moment I think my "Data" take less than 1TB (most of it is still on a 320GB HDD). Anyway I could expand pretty easily from there. The only "problem" I see is that when I'll need to expand I'll need to put my new HDDs inside my JBOD which goes into a PCIe controller. Therefore if 4TB are enough (and I think they are - at least for 2-3 years) I'd prefer to keep it directly attached to the server (for reliability reason).

If my SAS card fails I wouldn't be able to access my important data if they're split between the onboard SATA/SAS controller and the external one.

Crest said:
You left out the option of three 2 disk mirrors.
Click to expand...
I think that was intentional since it provides really no more than 1 faulty HDD protection. If 2 HDDs fail inside the same VDEV, I'm screwed. You're right however that if 1 HDD fail in each VDEV I can tolerate up to 3 HDDs failures.

Crest said:
Do you have any other copy of the data?
Click to expand...
I'm (a bit) ashamed to say no. Never really had a proper backup solution. I made backups by copy-paste from one HDD to another and now I can't really figure WHERE I stored what:\ That's the reason I wanted a NAS in the first place. From there I though I could backup to a remote location using Crashplan as well as a computer in my other house (rsync? bacula? amanda? backuppc? Haven't decided yet - any suggestions here? :()).
 
If it was me, I'd stripe across 3x2 disk mirror VDEVs.

Yes, if you have 2 failures in a single VDEV, you're hosed - but you have backups anyway, right?

On the flipside, up to half your disks can fail (if they're the right ones) and you're still good. Also, mirror rebuild is a fair bit quicker than parity raid rebuilds (it's a straight copy from the other disk in the mirror, rather than reading from all disks in the RAIDZ set, parity calc, and write to new disk) - so the likelyhood of secondary failure during rebuild is probably somewhat less.


edit:

No backups? I think if you're discounting the 3x mirror setup, yet have no backups, you're being overly paraniod against hardware failure vs human error/theft/etc.

Maybe do a stripe across 2 mirrors, and use the spare 2 disks in a different pool/different box for backup of critical data?
 
throAU said:
No backups? I think if you're discounting the 3x mirror setup, yet have no backups, you're being overly paraniod against hardware failure vs human error/theft/etc.
Click to expand...
No backups yet. I know that's bad and RAID is not a backup solution either.
I couldn't really get some free time to set it up though. I'll try ASAP.

throAU said:
Maybe do a stripe across 2 mirrors, and use the spare 2 disks in a different pool/different box for backup of critical data?
Click to expand...
Not really necessary. I already have other 6 (maybe more ... I'll have to check) x 2TB HDDs (bough before the flood :e).

Which backup solution would you reccomend though? Amanda / Bacula / Rsync / Backuppc / Others?

Thank you.
 
throAU said:
If you're running ZFS you could just ZFS-send a snapshot off to a different box.
Click to expand...

Is that really a backup solution?
I mean ... surely you can recover all your files (practical when a Zpools becomes corrupted / etc) but what about recovering one single file I accidentaly deleted?
 
throAU said:
If you're sending it to another zpool (a replica), then yes you just copy the file back :D
Click to expand...
Good to know, thank you.
I wonder however if it's really a good idea to have both boxes running ZFS. What if there is some problem/bug/vulnerability and both boxes get affected?

Wouldn't it be safer to use another OS with another FS?
 
luckylinux said:
Good to know, thank you.
I wonder however if it's really a good idea to have both boxes running ZFS. What if there is some problem/bug/vulnerability and both boxes get affected?

Wouldn't it be safer to use another OS with another FS?
Click to expand...

Keeping data in ZFS is much safer than others FS without ECC such as NTFS, EXT3.

I do zfs send to USB disk plugged to the FreeBSD storage.
 
Sorry to resurrect this thread already one month old, but I have yet one question.
I was assuming to dedicate 2 HDDs for "root" (system) drive, while using the 6 others for data (since my main server has 8 Hot-Swap drives). The other drives would be in a JBOD enclosure that I may want to power off when not in use (to save my electrical bill).

Would a 1 x RAIDZ-2 (3 Samsung, 3 Western Digital, 2 Hitachi) for 6*2TB = 12TB usable space or would a 2 (stripe) x RAIDZ-2 (2 Samsung, 1 Western Digital, 1 Hitachi) for 2*4TB = 8TB usable space be a better solution?

Then again I'm not sure what the limits of FreeBSD booting from RAIDZ-2 are (I seem to recall no more than one Vdev on it or something like that ...).


If space is not an issue, would performance of one of said configurations be "enough" to serve some Virtual machines via NFS/Samba (let's say saturate one or more of the gigabit links)? I think the 2xraidz-2(4 disks) may be able to achieve such performance. Any experience or other suggestion?

I think I'll just leave my data in the JBOD server since the virtual machines may need to be on all the time, but I won't access my documents while sleeping :D



Edit: but then again encryption using GELI may not work ...
Edit #2: maybe I can boot from one USB drive and then unlock GELI at will? Just have to make sure I won't do too many writes on the USB drive otherwise it'll wear out (have to put the ports collection on the HDDs :D)
Edit #3: forgot I paid so much for that server for having redundant PSU and KVM over IP (IPMI). That way there wouldn't be problems to unlock at boot, right? Even from remote location I'd just have to use an external VPN ...
 
luckylinux said:
Then again I'm not sure what the limits of FreeBSD booting from RAIDZ-2 are (I seem to recall no more than one Vdev on it or something like that ...).
Click to expand...

The only times I saw /boot/zfsboot or /boot/gptzfsboot fail its boot sequence was when there were multiple pools and it chose the wrong one (the one without the boot dataset containing the boot loader and kernel), or when a pool could not be accessed due to too many missing devices (they all have to be accessible through BIOS interrupt calls).

Other than that there should be no problem booting from complex pools, I have a server booting from a root pool with two raidz2 vdevs without issues, I believe this single-vdev restriction is a Solarism.
 
You must log in or register to reply here.
Back
Top