Chances of failure to drives on a 4 drive ZFS raidz1

ZFS is designed as 128 bit so it will be usable for 10 or even 20 years - but already the "current" generation of hard disks makes major changes in the source code(!!) necessary.

Even 1 Terabyte Sectors make sense with a 128 bit Filesystem!

Hmmm how strange is that?
 
fgordon said:
ZFS is designed as 128 bit so it will be usable for 10 or even 20 years - but already the "current" generation of hard disks makes major changes in the source code(!!) necessary.

Even 1 Terabyte Sectors make sense with a 128 bit Filesystem!

Hmmm how strange is that?



no, hard drive manufacturers need to give a shit about their customers and stop "lying" about the sector size.

ZFS's code is fine, except for one thing, it assumes the hard drive isn't lying about the sector size, go figure.
 
No it's not, just read what Phoenix wrote....

So the problem of ZFS is not disks "lying" about sector size the real problem ist ZFS has to be recompiled to work efficiently with any media > 512 byte sector size - as 512 byte is the "hard-coded" sector-size assumed by ZFS for ANY drive and cannot be changed without changing the source code and recompile it - and it seems the necessary changes are not that easy - if the drives would report 4k ZFS would not be any better...

Hehe ZFS is able to use Trillions of Terabyte.... but is designed to this on 512 sector size media only *g* I find that a bit strange of course one can patch ZFS when new sector sizes come - but this will make it very difficult to have 512, 4k, 8k sector sized media parallel in use.... like a big server and external drives...

I still think ZFS is great I just find this really strange for such an advanced fs-design.
 
fgordon said:
So the problem of ZFS is not disks "lying" about sector size the real problem ist ZFS has to be recompiled to work efficiently with any media > 512 byte sector size - as 512 byte is the "hard-coded" sector-size assumed by ZFS for ANY drive and cannot be changed without changing the source code and recompile it - and it seems the necessary changes are not that easy - if the drives would report 4k ZFS would not be any better...
If that's true, how are people using gnop as a workaround?
 
fgordon said:
No it's not, just read what Phoenix wrote....

So the problem of ZFS is not disks "lying" about sector size the real problem ist ZFS has to be recompiled to work efficiently with any media > 512 byte sector size - as 512 byte is the "hard-coded" sector-size assumed by ZFS for ANY drive and cannot be changed without changing the source code and recompile it - and it seems the necessary changes are not that easy - if the drives would report 4k ZFS would not be any better...

You are, of course wrong. ZFS is designed to use any sector size the underlying media says it has. And, you need to understand, that ZFS does not deal with only 'spinning disk media' but will all sorts of other storage devices, including network storage. Therefore, there is not any "512 byte sector" code compiled in.

The problem with those poor disks is the attitude of WD to restrict their usage to low-end desktop storage. The 'lie' is programmed in their microcode and WD are refusing to provide a fix for those operating systems that need to know the real geometry of the drive. The fact that it works with primitive OS like DOS/Windows does not make such lie acceptable.
In future version of Windows this might fail as miserably as it does with ZFS.

The truth is, that these drives do work with ZFS. They just don't performs to the user expectations, that are based on published spec. Thing is, this spec is only valid if you align writes to the assumed, but not reported 4k sector size.

There is nothing in ZFS to blame about this. Other file systems fail to utilize those drives 'performance' as well.

Or, let me ask it in a different way: everything with these WD drives works as designed. So where is the problem? If you need different behavior, buy different drives. There are so many on the market.
 
danbi said:
You are, of course wrong. ZFS is designed to use any sector size the underlying media says it has. And, you need to understand, that ZFS does not deal with only 'spinning disk media' but will all sorts of other storage devices, including network storage. Therefore, there is not any "512 byte sector" code compiled in.

Yes, there is. :) The "minimum block size" is a compile-time option, and is currently set to 512 B. This means, that ZFS will use variable-sized blocks for all writes, with the smallest block size being 512 B.

All of these "Advanced Format" drives advertise their block size as "512 B". Thus, ZFS will happily write 0.5, 1, 2, and 4 KB blocks, nicely destroying any manual partition alignment you've done. All it takes is writing out 1 little text file under 4 KB in size, to screw things up.

If the drive manufacturers fix their firmware to report 4 KB sectors, then things may work correctly.

Until then, you need to recompile all the ZFS tools to set the minimum block size to 4 KB.

The truth is, that these drives do work with ZFS. They just don't performs to the user expectations, that are based on published spec. Thing is, this spec is only valid if you align writes to the assumed, but not reported 4k sector size.

Which requires you to recompile all the ZFS tools to set the minimum block size to 4 KB. Without that, any small writes will be done using 0.5, 1, or 2 KB blocks, destroying your nicely aligned partitions.

There is nothing in ZFS to blame about this. Other file systems fail to utilize those drives 'performance' as well.

Never said ZFS was to blame, just that ZFS will not work with these 4 KB drives without either manually recompiling the ZFS tools to set the minimum block size to 4 KB, or manufacturers fixing their firmware to report 4 KB physical and logical sectors.

Or, let me ask it in a different way: everything with these WD drives works as designed.

They don't follow the ATA spec, which lists separate logical sector size and physical sector size parameters that the OS can query. These drives list 512 B for both.
 
fgordon said:
No it's not, just read what Phoenix wrote....

So the problem of ZFS is not disks "lying" about sector size the real problem ist ZFS has to be recompiled to work efficiently with any media > 512 byte sector size - as 512 byte is the "hard-coded" sector-size assumed by ZFS for ANY drive and cannot be changed without changing the source code and recompile it - and it seems the necessary changes are not that easy - if the drives would report 4k ZFS would not be any better...

Hehe ZFS is able to use Trillions of Terabyte.... but is designed to this on 512 sector size media only *g* I find that a bit strange of course one can patch ZFS when new sector sizes come - but this will make it very difficult to have 512, 4k, 8k sector sized media parallel in use.... like a big server and external drives...

I still think ZFS is great I just find this really strange for such an advanced fs-design.

You just don't know what you're talking about.

Next time someone explains the issue, before you go and make the same comment, try to do a google search.

A simple search for "ZFS 4k problem" or "4k drive lie" would have netted you a WEALTH of information on the subject.

The "patch" for FreeBSD isn't to fix any inherent flaw in ZFS, it's to make patch around the stupid 512b lie, and even then it's not even CLOSE to ideal.

Having a firmware which is "honest" is what we need. This is why i do not buy or advise buying any WD drives (nor have i for the past few years)

The ridiculousness of your comments is hilarious. I've seen ZFS work amazingly well on 4k drives which had 4k firmware (they DO exist).
 
phoenix said:
Yes, there is. :) The "minimum block size" is a compile-time option, and is currently set to 512 B. This means, that ZFS will use variable-sized blocks for all writes, with the smallest block size being 512 B.

All of these "Advanced Format" drives advertise their block size as "512 B". Thus, ZFS will happily write 0.5, 1, 2, and 4 KB blocks, nicely destroying any manual partition alignment you've done. All it takes is writing out 1 little text file under 4 KB in size, to screw things up.

Of course, ZFS has hard-coded "mimimum block size", but this is not really the "sector size". This minimum is to ensure that ZFS will write minimum of 512 bytes, even if the underlying media claims to support 128 bytes, for example. With smaller than 512 byte blocks, the filesystem will become too fragmented and metadata will occupy too much of the storage.

Wonder if this "Advanced Format" phenomenon will disappear soon and manufacturers will stop making such lying disks.
 
Back
Top