ZFS RAIDZ2 on 5 disks - specific settings?

[geos.pl]

I would like to create ZFS pool on 5 disks. Someone suggested that it could be bad idea because default 128KiB record would be divided into "ugly-sized chunks" when distributed over the disks, and gave me example:

https://forums.anandtech.com/threads/zfs-raidz-question.2353232/#post-35760300

This has to do with the recordsize of 128KiB that gets divided over the number of disks. Example for a 3-disk RAID-Z writing 128KiB to the pool:
disk1: 64KiB data (part1)
disk2: 64KiB data (part2)
disk3: 64KiB parity

Each disk now gets 64KiB which is an exact multiple of 4KiB. This means it is efficient and fast. Now compare this with a non-optimal configuration of 4 disks in RAID-Z:
disk1: 42,66KiB data (part1)
disk2: 42,66KiB data (part2)
disk3: 42,66KiB data (part3)
disk4: 42,66KiB parity

Now this is ugly! It will either be downpadded to 42.5KiB or padded toward 43.00KiB, which can vary per disk. Both of these are non optimal for 4KiB sector harddrives. This is because both 42.5K and 43K are not whole multiples of 4K. It needs to be a multiple of 4K to be optimal.

my question is: if the recordsize for a RAIDZ2 pool on 5 disks was set to 96KiB - would it be OK then? my understanding is that chunks would have "nice size" of 32KiB then. and another one: are there any ZFS settings which are better to have specific values for RAIDZ2 on 5 disks rather than default? if so, what these settings are and what values are optimal for RAIDZ2 5 disk setup?

thank you,
geos

 

[Eric A. Borisch]

Short answer? Don't make it smaller.

See here: https://www.delphix.com/blog/delphi...or-how-i-learned-stop-worrying-and-love-raidz
And here: https://docs.google.com/spreadsheets/d/13sJPc6ZW6_441vWAUiSvKMReJW4z34Ix5JSs44YXRyM

Extra overhead comes when ZFS needs to pad the record being saved to be (as required by design) a multiple of P+1 sectors -- so up to two sectors of padding per record for Z2 (P=2) pools. For smaller records this leads (in general) to move overhead. Also recognize that each record is <= recordsize; it is not "all records ARE this size." (The zpool's sector size == 2^ashift)

Be sure to read the "layout examples" on the google docs spreadsheet. It shows z3 examples, but sometimes a picture helps.