Hi
Recently I have experienced low performance on 4k read benchmark in my VM with only 21-25MB on read, however, the read on host was saturated the SSD read which is 530MB/s during benchmark.
After investigation, I realized that the issue was due to read amplification on 128k recordsize filesystem, afterwards, I created a new VM on a 4k zfs volume and re-run the random 4k read benchmark again, the performance was aligned with the host which is > 150MB/s
I would like to convert my vm image volume from 128k recordsize to at least 16k with scarifying some read overhead(larger recordsize tend to better compression ratio and scrubbing performance), however, zfs send and receive won't do the job.
I have run few test without success
This is my test volume with the default 128k recordsize
zdb shows the file is created on 128k recordsize.
I changed the parent recordsize to 4k instead and all the subvolume will be inherited, and do the zfs send and receive.
New volume recordsize is 4k
However, zdb still showing the file is 128k recordsize.
The recordsize only has been changed by copying the file to the same volume again. (Object 9)
How can I alter the recordsize of individual file according to the parameter in zfs during send and receive?
Thank You.
Recently I have experienced low performance on 4k read benchmark in my VM with only 21-25MB on read, however, the read on host was saturated the SSD read which is 530MB/s during benchmark.
After investigation, I realized that the issue was due to read amplification on 128k recordsize filesystem, afterwards, I created a new VM on a 4k zfs volume and re-run the random 4k read benchmark again, the performance was aligned with the host which is > 150MB/s
I would like to convert my vm image volume from 128k recordsize to at least 16k with scarifying some read overhead(larger recordsize tend to better compression ratio and scrubbing performance), however, zfs send and receive won't do the job.
I have run few test without success
This is my test volume with the default 128k recordsize
Code:
NAME USED AVAIL REF VSIZE RSIZE COMP RATIO ATIME SYNC SNAP LATEST SNAP STATUS
vol 1.64T 1.84T 128K - 128K lz4 1.46x on standard 0
vol/test1 100M 1.84T 100M - 128K lz4 1.00x on standard 0
zdb shows the file is created on 128k recordsize.
Code:
# zdb vol/test1
Dataset vol/test1 [ZPL], ID 151461, cr_txg 18667515, 100M, 8 objects
Object lvl iblk dblk dsize lsize %full type
8 2 128K 128K 100M 100M 100.00 ZFS plain file
I changed the parent recordsize to 4k instead and all the subvolume will be inherited, and do the zfs send and receive.
zfs set recordsize=4k vol
zfs send vol/test1@2018-08-12_16_28 | zfs receive vol/test2
New volume recordsize is 4k
Code:
NAME USED AVAIL REF VSIZE RSIZE COMP RATIO ATIME SYNC SNAP LATEST SNAP STATUS
vol 1.64T 1.84T 128K - 4K lz4 1.46x on standard 0
vol/test2 100M 1.84T 100M - 4K lz4 1.00x on standard 1 2018-08-12_16_28
However, zdb still showing the file is 128k recordsize.
Code:
# zdb vol/test2
Dataset vol/test2 [ZPL], ID 151698, cr_txg 18667554, 100M, 8 objects
Object lvl iblk dblk dsize lsize %full type
8 2 128K 128K 100M 100M 100.00 ZFS plain file
The recordsize only has been changed by copying the file to the same volume again. (Object 9)
Code:
# cp test test2
# zdb vol/test2
Dataset vol/test2 [ZPL], ID 151698, cr_txg 18667554, 202M, 9 objects
Object lvl iblk dblk dsize lsize %full type
8 2 128K 128K 100M 100M 100.00 ZFS plain file
9 3 128K 4K 102M 100M 100.00 ZFS plain file
How can I alter the recordsize of individual file according to the parameter in zfs during send and receive?
Thank You.