disk recommendation 2021

gpw928

Well-Known Member

Reaction score: 127
Messages: 377

Hi Robert, there's a lot of good advice above. Here are my random thoughts:
  • The Backblaze Hard Drive Data and Stats are a great resource, but never offer a complete view of all options, and often lag the market.
  • The introduction of SMR by Western Digital was sneaky (but WD seem to have responded to the bad press). Read the datasheets of any disk you contemplate, and avoid SMR.
  • SSD and NVME are great if they fit your capacity and budget needs. But at 20 TB+ might be a challenge.
  • Choosing very large disks has cost advantage, but an operational down-side. Re-silvering time for a 3 to 4 TB disk is measured in days. Disk access for other applications will be severely compromised (maybe unusable) while this is happening. The smaller the disk, the less the grief (which is why vendors like IBM are still selling truck loads of 500 GB disks into the enterprise market).
  • You largely get what you pay for.
  • My five CMR 3 TB WD Reds (which performed poorly in the Lifetime Backblaze stats) have been quite satisfactory in my ZFS server. One infant failure, replaced under warranty, and another very recent failure at ~8 years continuous service.
  • As a generalisation, RAIDZ gives you surprisingly good striping across the spindles (much better than mirrors), and (usually) better capacity than mirrors. RAIDZ2 gives you striping, plus better redundancy than mirrors. I expect that mirrors will probably perform better if you have a truly random small block I/O pattern.
  • I am slowly switching from 8-year old CMR 3 TB WD Reds to 3 and 4 TB Seagate EXOS (enterprise) disks for durability and reliability (I'm laying in spares as the WD Reds are pretty old).
  • Reliability is so important to me that I am switching my ZFS server from 5 x 3 TB consumer grade disks in RAIDZ1 to 7 x 3 TB enterprise grade disks in RAIDZ2.
My ZFS server backs up everything else on the network, but has too much data to back up itself. It warrants as much protection as I can sensibly give it. So, a UPS, data partitioning (to easily backup some of it to external media with a ZFS send), and multi-spindle redundancy with enterprise class disks are all in play.
 

olli@

Aspiring Daemon
Developer

Reaction score: 895
Messages: 862

Just a reminder: The purpose of a RAID is to improve performance, reliability and availability, but it is not a replacement for a backup. So, whatever kind of disk setup you have, always make sure you also have a good backup that you can store in a safe place, preferably at a different location, so it is safe from theft, fire and other adversities.
 

PMc

Aspiring Daemon

Reaction score: 421
Messages: 975

Just a reminder: The purpose of a RAID is to improve performance, reliability and availability, but it is not a replacement for a backup. So, whatever kind of disk setup you have, always make sure you also have a good backup that you can store in a safe place, preferably at a different location, so it is safe from theft, fire and other adversities.

This is indeed very important, because ZFS pools can go bad without the disks involved.

For instance, I am currently observing this (details removed):
Code:
  pool: bm
state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
config:

        NAME            STATE     READ WRITE CKSUM
        bm              ONLINE       0     0     1
          raidz1-0      ONLINE       0     0     2
            ada0p5.eli  ONLINE       0     0     0
            ada2p1.eli  ONLINE       0     0     0
            ada5p1.eli  ONLINE       0     0     0
        cache
          ada3p7.eli    ONLINE       0     0     0

errors: 18 data errors, use '-v' for a list
[...]


As we can see, these errors did not happen on any device. And I can always create new errors by creating a snapshot and trying to send that snapshot somewhere:

Code:
# zfs snapshot bm/bk/data@crap
# zfs send -R -PLcepv -I bm/bk/data@zmir.210112_172101 bm/bk/data@crap > /dev/null
warning: cannot send 'bm/bk/data@crap': Input/output error
# zpool status -v bm
config:

        NAME            STATE     READ WRITE CKSUM
        bm              ONLINE       0     0     2
          raidz1-0      ONLINE       0     0     4
            ada0p5.eli  ONLINE       0     0     0
            ada2p1.eli  ONLINE       0     0     0
            ada5p1.eli  ONLINE       0     0     0
        cache
          ada3p7.eli    ONLINE       0     0     0


So what to do with such a pool? Reboot didn't help. There is probably no other way than creating it new and restoring from backup.
 

Snurg

Aspiring Daemon

Reaction score: 343
Messages: 822

As others already said: the failures of drives are unpredictable.
But often have common causes (previous operating environment, handling mistakes, firmware bugs,...) that result in concurrent failure.
So according to my bad experience the worst thing for small users like me and the OP is to use several drives of same manufacturer/type and possibly even batch.

Personally I have had failed drives of every manufacturer, so I do not have any "preferences".
In general, my personal feeling is enterprise grade drives (heavy bricks) tend to be way more longlife than the other extreme, CCTV drives.

[USER=52756]PMc[/USER]
This really shocks me!
Is it really this easy to render ZFS pools unusable, just by trying to send a snapshot?
 

PMc

Aspiring Daemon

Reaction score: 421
Messages: 975

[USER=52756]PMc[/USER]
This really shocks me!
Is it really this easy to render ZFS pools unusable, just by trying to send a snapshot?
Short answer: no, not this way round. The snapshot-sending just triggered an error that was persistent in the pool.

A full scrub finally showed the file with the actual defect. (And as this file would be contained in any new snapshot, the snapshots also became erroneous.) Deleting that file and doing another scrub solved the matter.

But the important thing is: this error was not related to any disk failure. It was present in all mirrors, and probably caused by memory fault/power supply spike/whatever. In this case it concerned just a single file, which could be removed/replaced. But it is also possible for such a thing to happen within the pool's metadata, and then things become more difficult.

Bottomline: a zfs pool is not infallible, it can get lost, and there should be a desaster recovery plan for this.
 
Top