Biggest ZFS Misconfigurations and How to Fix Them: Part 1

@ Cracauer,
Setting it to 12 regardless disks seems a legit answer to me.

==================================================================================
Interesting URLs regarding virtualisation ,

https://klarasystems.com/articles/tuning-recordsize-in-openzfs/

https://openzfs.github.io/openzfs-docs/Performance and Tuning/Workload Tuning.html#virtual-machines

https://klarasystems.com/articles/zfs-virtualization-storage-backend-for-pros/

https://forum.proxmox.com/threads/s...atasets-on-zfs-2-2-0-mirror-vdev-pool.142584/

https://serverfault.com/questions/1117662/disadvantages-of-using-zfs-recordsize-16k-instead-of-128k

https://www.truenas.com/community/threads/vmware-6-7-datastore-block-size.76431/#post-531229

=================================================================================

My config,
Code: 
NAME                   PROPERTY    VALUE    SOURCE
SSD/var/db/prometheus  recordsize  128K     default
SSD/var/db/grafana     recordsize  128K     default
SSD/var/db/sqlite      recordsize  4K       local
SSD/var/db/influxd     recordsize  64K      local
SSD/var/db/mariadb     recordsize  16K      local
SSD/var/db/postgres    recordsize  32K      local
 
cracauer@ said:
Why wouldn't I set it to 12 regardless of current disks?
Click to expand...

ashift tells ZFS the physical sector size of a vdev, as a power of two: ashift=9 means 512-byte sectors, ashift=12 means 4K
Click to expand...

I figure best performance would be to match what the disk reports; I usually only see 512 but had a Hynix NVMe that could switch to native 4K (current Samsung 980 Gen3 only reports 512)

Not sure what it might affect on FreeBSD or ZFS, but Steam and Oculus (VR software) on Windows had interesting errors/didn't work if drive firmware was at 4K
 
Espionage724 said:
I figure best performance would be to match what the disk reports; I usually only see 512 but had a Hynix NVMe that could switch to native 4K (current Samsung 980 Gen3 only reports 512)

Not sure what it might affect on FreeBSD or ZFS, but Steam and Oculus (VR software) on Windows had interesting errors/didn't work if drive firmware was at 4K
Click to expand...

Well, my current disks are 512b. But this will be a long living array. What do I know what kind of replacement disks I might throw into it in 8 years or so.

(assuming I can afford replacement disks :))
 
the problem is you can't know what is really native. the firmware on the storage device reports whatever it likes. then the computational effort splits between the host cpu and the firmware.
which case is the better always depends on "stuff".
 
Rereading books and such I think if the device reports logical 512b blocks but internally uses 4k physical (a possible situation on SSDs), and you set ashift to 9, that's where you run into potential performance issues. ZFS knows about 512b, so to write 4k of data, ZFS queues up 8 write transactions (at least that's my understanding).
Smaller may be more efficient if the workload is storing lots of small files (same thing on UFS and pretty much every other filesystem)

The long living array: I think new devices, set ashift to 12 (4K) then add them. I think worst case would be new vdev gives different performance which may affect overall array performance.

Just my opinions based on my understanding which may be completely offbase
 
The virtual machine image file backing is also interesting.

There is no reason why you wouldn't have to split up into different datasets with different recordsizes just like a direct OS install. Of course doing this for a VM would be incredibly cumbersome as this would all be different filesystems inside and have to live in different datasets outside.

So you'd need a structure like:
- vms/recordsize64k/machineA
- vms/recordsize64k/machineB
- vms/recordsize1m/machineA
- vms/recordsize1m/machineB
[...]

At the very least mysql and postgres databases in VMs would need that kind of backing.
 
You must log in or register to reply here.
Back
Top