ZFS Choosing partition size to account for different disk sizes, how much gap?

[FzZzT]

Just ruminating and curious...

I will be using a new pool soon and was wondering about this. Normally, as I did initially with this new pool, I create a partition and use that to make my zpool. The reasons I do this (which might be stupid) are so that I can label them (e.g. gpt/DiskB1) and, to a lesser extent, align things on boundaries. Recently, I had to replace some disks in my main storage pool, and I kind of got lucky (I think) that the new disks are the same size as the old ones and I was able to create the same size partitions. But this assumes that drives of the same size (e.g. "2 TB") from different brands or models would have different block/sector counts. With the new pool, I was thinking about using whole raw disks this time around, but the label is pretty useful and I was concerned that if I did that I would be stuck using disks with that exact size (or larger tiers), which led me to this question.

I can easily create a partition that doesn't use the whole disk (or probably a zpool/zfs that doesn't, for the raw disk case). This seems to make some sense to guard against varying disk sizes, but what I am wondering is how much space to carve off? This new pool is using 1 TB SSDs and I created a 900 GB partition, but a 10%/100 GB gap between "1 TB" SSDs from different brands seems like a lot. Would a "1 TB" disk from one company really be 100 GB smaller than a "1 TB" disk from another brand? What percentage or amount makes sense to reserve, or is this all silly and I should just ignore it?

I have found some comments online that lean both ways -- that advertised sizes are standardized, and also that different brands can vary within the same size class.

 

[hedwards]

I wouldn't necessarily say that this is silly, however, the cost of getting this wrong is probably less than $300 if you have to buy a temporary disk to store the data on while resizing plus whatever cost downtime has. The $300 is just the first listing I found for a $12TB external drive at BestBuy, smaller amounts of data would be less and other stores may have better prices. I personally, haven't had much trouble with this sort of thing, but I tend to stick to the same brand and the same capacity disks unless I'm planning to expand in place.

YMMV, but if you're still concerned, you could just go with limiting it to 950 GB (Really, 950 before accounting for the portion that gets lost to the filesystem, not after) which would still be pretty conservative, but would make it highly unlikely that a disk you might buy later would be smaller than what you're already using. But, again, I personally haven't had any trouble, and if you have somewhere that you can temporarily hold the data while resizing, it's probably not an issue.

 

[cracauer@]

They definitely vary. I think 1 MB should be sufficient as a buffer.

 

[adorno]

By chance I just stumbled over this document providing some technical context: http://www.idema.org/wp-content/downloads/2169.pdf

I just ran the numbers on my laptop and found that Samsung scammed me by 40 (512B) sectors. Be that as it may, it indicates that the 1MB suggested by cracauer@ seems reasonable. If you want to be super safe just base your partitioning on the nominal disk size. Anything below that would just be wasteful.

 

[FzZzT]

Thanks for the responses. I think I ended up reserving way too much for a buffer, but for future readers: I did some more research and compared sector counts (from spec sheets/articles online) across a few brands and models (for 2 TB drives) and the spread from smallest to largest was around 5,000 sectors. By my math this is about a 20 MB difference for 4K sectors (0.001% for a 2 TB disk) or 2.5 MB for 512 byte (which I don't think anyone uses any more?). When I do it again I will probably reserve ~100 MB which is still only 0.01% of the disk (one hundredth of one percent).

 

[Oznaiust]

I don't know how much tolerance is within actual drive specifications but i have an 64GB USB stick that is missing approx. 1.14GB compared to another 64GB stick. Windows (sorry) shows after fresh format 57.7GB free space. Back to ZFS. I think i read something about that ZFS already handle that issue by itself and does not use the outer space to the maximum. If you had to replace disks in your main storage pool then the pool could be older, too small or faulty. My pool grew from one disk via mirror to raidz in yearly steps from different hdd companies. I never cared about detailed sector count and never had issues with that. Finally i ended up in buying bigger disks. The smallest disk limits the pool size and after replacement the pool grew with zpool online -e.

Today i see disks as consumables. If you have to do it again one more important point imho. To track the age of the disks i set the build date into the gpt label. But my disks from different companies have different warranty periods. My label scheme changed to slot-expiry-year-month e.g. gpt/D3e27oct. The trigger for that was an external HDD superblock failure and i opened the case for the suprise that the 1 TB disk was 12 years old.

For the whole disk question see whole-disks-versus-partitions. I use partitions only. It's flexible and give more control e.g. that size issue that we discuss. You may note the free space target of 10%. Meaning your pool will get slower at ~810GB filled. You could get all space if you use gpart to resize your freebsd-zfs partitions of the pool to maximum minus 100M and activate the space with zpool online -e. If you get the jackpot of a smaller disk then you could create a new pool on the smaller disk and "move" your datasets via zfs send|receive to the new pool. Then detach one of the old disks, wipe it with zpool labelclear and attach it to the new disk. Usual rules apply, backup first and it is helpful to test that within a VM first.