Firstly some praise: Thanks for this informative post. Previously the only thing I read about these non-512B sector disks is to avoid them for zfs, but your numbers look great.
And an addition: my system set up has 48 GB of ram and I found that zfs was only using around 600 MB until I raised the kmem size and other zfs tunables. You should definitely look into that. Here is basically what I used as a template: ZFSguru advanced tuning Before tuning, I found my numbers were 60-100% higher than yours (with more expensive hardware and 16 disks) with tests like yours (simple dd or cp with something other than /dev/zero though), after verifying no caching. But, when doing reading and writing at the same time, it went horribly slow... maybe 50-100 MB/s, which is slower than my raid0 fake raid desktop. After tuning the memory, it adds up to around 500, which is faster than an XFS 22 disk SAS RAID6 system we have.
And the main reason for my post... a question.
Does anyone know what sort of ashift/sector size an SSD should use? I am thinking it might affect my ZIL performance which I have on an SSD and I think it seems far too slow. I checked now, and it says ashift=9.
And an addition: my system set up has 48 GB of ram and I found that zfs was only using around 600 MB until I raised the kmem size and other zfs tunables. You should definitely look into that. Here is basically what I used as a template: ZFSguru advanced tuning Before tuning, I found my numbers were 60-100% higher than yours (with more expensive hardware and 16 disks) with tests like yours (simple dd or cp with something other than /dev/zero though), after verifying no caching. But, when doing reading and writing at the same time, it went horribly slow... maybe 50-100 MB/s, which is slower than my raid0 fake raid desktop. After tuning the memory, it adds up to around 500, which is faster than an XFS 22 disk SAS RAID6 system we have.
And the main reason for my post... a question.
Does anyone know what sort of ashift/sector size an SSD should use? I am thinking it might affect my ZIL performance which I have on an SSD and I think it seems far too slow. I checked now, and it says ashift=9.