disk recommendation 2021

Hi Robert, there's a lot of good advice above. Here are my random thoughts:
  • The Backblaze Hard Drive Data and Stats are a great resource, but never offer a complete view of all options, and often lag the market.
  • The introduction of SMR by Western Digital was sneaky (but WD seem to have responded to the bad press). Read the datasheets of any disk you contemplate, and avoid SMR.
  • SSD and NVME are great if they fit your capacity and budget needs. But at 20 TB+ might be a challenge.
  • Choosing very large disks has cost advantage, but an operational down-side. Re-silvering time for a 3 to 4 TB disk is measured in days. Disk access for other applications will be severely compromised (maybe unusable) while this is happening. The smaller the disk, the less the grief (which is why vendors like IBM are still selling truck loads of 500 GB disks into the enterprise market).
  • You largely get what you pay for.
  • My five CMR 3 TB WD Reds (which performed poorly in the Lifetime Backblaze stats) have been quite satisfactory in my ZFS server. One infant failure, replaced under warranty, and another very recent failure at ~8 years continuous service.
  • As a generalisation, RAIDZ gives you surprisingly good striping across the spindles (much better than mirrors), and (usually) better capacity than mirrors. RAIDZ2 gives you striping, plus better redundancy than mirrors. I expect that mirrors will probably perform better if you have a truly random small block I/O pattern.
  • I am slowly switching from 8-year old CMR 3 TB WD Reds to 3 and 4 TB Seagate EXOS (enterprise) disks for durability and reliability (I'm laying in spares as the WD Reds are pretty old).
  • Reliability is so important to me that I am switching my ZFS server from 5 x 3 TB consumer grade disks in RAIDZ1 to 7 x 3 TB enterprise grade disks in RAIDZ2.
My ZFS server backs up everything else on the network, but has too much data to back up itself. It warrants as much protection as I can sensibly give it. So, a UPS, data partitioning (to easily backup some of it to external media with a ZFS send), and multi-spindle redundancy with enterprise class disks are all in play.
 
Just a reminder: The purpose of a RAID is to improve performance, reliability and availability, but it is not a replacement for a backup. So, whatever kind of disk setup you have, always make sure you also have a good backup that you can store in a safe place, preferably at a different location, so it is safe from theft, fire and other adversities.
 
Just a reminder: The purpose of a RAID is to improve performance, reliability and availability, but it is not a replacement for a backup. So, whatever kind of disk setup you have, always make sure you also have a good backup that you can store in a safe place, preferably at a different location, so it is safe from theft, fire and other adversities.

This is indeed very important, because ZFS pools can go bad without the disks involved.

For instance, I am currently observing this (details removed):
Code:
  pool: bm
state: ONLINE
status: One or more devices has experienced an error resulting in data
        corruption.  Applications may be affected.
config:

        NAME            STATE     READ WRITE CKSUM
        bm              ONLINE       0     0     1
          raidz1-0      ONLINE       0     0     2
            ada0p5.eli  ONLINE       0     0     0
            ada2p1.eli  ONLINE       0     0     0
            ada5p1.eli  ONLINE       0     0     0
        cache
          ada3p7.eli    ONLINE       0     0     0

errors: 18 data errors, use '-v' for a list
[...]

As we can see, these errors did not happen on any device. And I can always create new errors by creating a snapshot and trying to send that snapshot somewhere:

Code:
# zfs snapshot bm/bk/data@crap
# zfs send -R -PLcepv -I bm/bk/data@zmir.210112_172101 bm/bk/data@crap > /dev/null
warning: cannot send 'bm/bk/data@crap': Input/output error
# zpool status -v bm
config:

        NAME            STATE     READ WRITE CKSUM
        bm              ONLINE       0     0     2
          raidz1-0      ONLINE       0     0     4
            ada0p5.eli  ONLINE       0     0     0
            ada2p1.eli  ONLINE       0     0     0
            ada5p1.eli  ONLINE       0     0     0
        cache
          ada3p7.eli    ONLINE       0     0     0

So what to do with such a pool? Reboot didn't help. There is probably no other way than creating it new and restoring from backup.
 
As others already said: the failures of drives are unpredictable.
But often have common causes (previous operating environment, handling mistakes, firmware bugs,...) that result in concurrent failure.
So according to my bad experience the worst thing for small users like me and the OP is to use several drives of same manufacturer/type and possibly even batch.

Personally I have had failed drives of every manufacturer, so I do not have any "preferences".
In general, my personal feeling is enterprise grade drives (heavy bricks) tend to be way more longlife than the other extreme, CCTV drives.

PMc
This really shocks me!
Is it really this easy to render ZFS pools unusable, just by trying to send a snapshot?
 
PMc
This really shocks me!
Is it really this easy to render ZFS pools unusable, just by trying to send a snapshot?
Short answer: no, not this way round. The snapshot-sending just triggered an error that was persistent in the pool.

A full scrub finally showed the file with the actual defect. (And as this file would be contained in any new snapshot, the snapshots also became erroneous.) Deleting that file and doing another scrub solved the matter.

But the important thing is: this error was not related to any disk failure. It was present in all mirrors, and probably caused by memory fault/power supply spike/whatever. In this case it concerned just a single file, which could be removed/replaced. But it is also possible for such a thing to happen within the pool's metadata, and then things become more difficult.

Bottomline: a zfs pool is not infallible, it can get lost, and there should be a desaster recovery plan for this.
 
For reliability estimates see:

 
Charts don't seem useful by themselves, sorted by MTBF. One sample disks, don't know when the errors occurred, selective reporting bias, etc.
 
In fact I believe that most disks have about the same reliability, no matter what vendor. There are always people who tell you horror stories about arbitrary vendors – and the next person will tell you exactly the opposite. If there were vendors that were considerably worse than others, then those would be out of business very quickly.

The important thing is to buy drives that are suitable for the purpose. Do not put AV disks in a NAS RAID. Do not put a NAS disk as single disk in a desktop machine. Do not use server disks (meant for 24/7 duty) in environments were they’re spun down+up very often. And so on. Do not just buy the cheapest disk you can get. Think before you buy, and look at the disk’s specifications.

Another important thing is to watch the drive temperature. Many users make efforts to cool their CPU, but they forget about the disks. Run smartd(8) and configure it to alert you if the drive temperature gets too high.

Also, if a HDD has been used in a certain orientation (e.g. horizontal) for a long time, it should not be turned into a different orientation (e.g. vertical), because this might cause problems with the bearings. Been there, done that … – SSDs don’t have this problem, of course.

And finally, some people recommend to use different drives from different vendors when buying disks for a RAID, in order to spread the risk. This seems to be even more important for SSDs.
 
Back
Top