Example: disk usage change in going from 512 to 4096 bytes

T

t1066

Yesterday, I migrated my storage from a pair of mirrors of 4 disks with 512 bytes (aaa) to 1 mirror of 2 disks with 4096 bytes (bbb) and finally to raidz2 of 6 disks with 4096 bytes (ccc). The storage usage changed as follows:

Code: 
NAME                     USED  AVAIL  REFER  MOUNTPOINT
aaa                      643G  1.16T    21K  none
aaa/ccache              2.03G  1.16T  2.03G  /home/ccache
aaa/jail                 346M  1.16T   346M  /home/jail
aaa/obj                   33K  1.16T    33K  /usr/obj
aaa/ports               1.62G  1.16T   262M  /usr/ports
aaa/ports/distfiles     1.37G  1.16T  1.37G  /usr/ports/distfiles
aaa/ports/packages        31K  1.16T    31K  /usr/ports/packages
aaa/src                  370M  1.16T   370M  /usr/src
aaa/ddd                  639G  1.16T  35.2G  /home/ddd
aaa/ddd/archive          269G  1.16T   269G  /home/ddd/archive
aaa/ddd/eee              115G  1.16T   115G  /home/ddd/eee
aaa/ddd/old             97.5G  1.16T  97.5G  /home/ddd/old
aaa/ddd/fff              123G  1.16T  33.0G  /home/ddd/fff
aaa/ddd/fff/hhh         22.1G  1.16T  22.1G  /home/ddd/fff/hhh
aaa/ddd/fff/iii         20.0G  1.16T  20.0G  /home/ddd/fff/iii
aaa/ddd/fff/ggg         47.6G  1.16T  43.5G  /home/ddd/fff/ggg
aaa/var                  126M  1.16T   126M  /var

bbb                      647G   267G   112K  none
bbb/ccache              2.60G   267G  2.60G  /home/ccache
bbb/jail                 386M   267G   386M  /home/jail
bbb/obj                  152K   267G   152K  /usr/obj
bbb/ports               2.23G   267G   880M  /usr/ports
bbb/ports/distfiles     1.37G   267G  1.37G  /usr/ports/distfiles
bbb/ports/packages       144K   267G   144K  /usr/ports/packages
bbb/src                  513M   267G   513M  /usr/src
bbb/ddd                  641G   267G  35.5G  /home/ddd
bbb/ddd/archive          269G   267G   269G  /home/ddd/archive
bbb/ddd/eee              115G   267G   115G  /home/ddd/eee
bbb/ddd/old             97.7G   267G  97.7G  /home/ddd/old
bbb/ddd/fff              123G   267G  33.4G  /home/ddd/fff
bbb/ddd/fff/hhh         22.2G   267G  22.2G  /home/ddd/fff/hhh
bbb/ddd/fff/iii         20.0G   267G  20.0G  /home/ddd/fff/iii
bbb/ddd/fff/ggg         47.9G   267G  43.7G  /home/ddd/fff/ggg
bbb/var                  210M   267G   210M  /var

ccc                      652G  2.93T   224K  none
ccc/ccache              3.76G  2.93T  3.76G  /home/ccache
ccc/jail                 466M  2.93T   466M  /home/jail
ccc/obj                  304K  2.93T   304K  /usr/obj
ccc/ports               3.01G  2.93T  1.64G  /usr/ports
ccc/ports/distfiles     1.37G  2.93T  1.37G  /usr/ports/distfiles
ccc/ports/packages       288K  2.93T   288K  /usr/ports/packages
ccc/src                  805M  2.93T   805M  /usr/src
ccc/ddd                  644G  2.93T  36.5G  /home/ddd
ccc/ddd/archive          270G  2.93T   270G  /home/ddd/archive
ccc/ddd/eee              115G  2.93T   115G  /home/ddd/eee
ccc/ddd/old             98.1G  2.93T  98.1G  /home/ddd/old
ccc/ddd/fff              124G  2.93T  33.9G  /home/ddd/fff
ccc/ddd/fff/hhh         22.2G  2.93T  22.2G  /home/ddd/fff/hhh
ccc/ddd/fff/iii         20.1G  2.93T  20.1G  /home/ddd/fff/iii
ccc/ddd/fff/ggg         48.0G  2.93T  43.8G  /home/ddd/fff/ggg
ccc/var                  364M  2.93T   364M  /var
 
@t1066

I´ve also noticed a slight increase in disk usage after migration to 4k drives. But it´s either that or god awful performance so the trade-off is acceptable.

Quick question though, exactly how did you migrate the data? Was it block-copy with zfs send|recv or file-copy with e.g. dump|restore?

/Sebulon
 
@t1066

Yeah, that´s how I moved my data as well. And since that procedure should be the most accurate, in terms reflecting the original data, that means that the increase we see is really caused by just the change in minimum IO size. This overhead is something I´ve seen discussed in many mail-threads between devs and admins all around, so I knew about this beforehand, and is something to be expected.

Odd to see it take up even more space on raidz compared to mirror though, but not something I´d find too alarming.

/Sebulon
 
I think reorganize the above data into user and system data will give a better picture of what is to be expected.

User data: Mostly multimedia files, zip and rar archives plus a few text files.

Code: 
NAME                     USED  AVAIL  REFER  MOUNTPOINT
aaa/ddd                  639G  1.16T  35.2G  /home/ddd
aaa/ddd/archive          269G  1.16T   269G  /home/ddd/archive
aaa/ddd/eee              115G  1.16T   115G  /home/ddd/eee
aaa/ddd/old             97.5G  1.16T  97.5G  /home/ddd/old
aaa/ddd/fff              123G  1.16T  33.0G  /home/ddd/fff
aaa/ddd/fff/hhh         22.1G  1.16T  22.1G  /home/ddd/fff/hhh
aaa/ddd/fff/iii         20.0G  1.16T  20.0G  /home/ddd/fff/iii
aaa/ddd/fff/ggg         47.6G  1.16T  43.5G  /home/ddd/fff/ggg

bbb/ddd                  641G   267G  35.5G  /home/ddd
bbb/ddd/archive          269G   267G   269G  /home/ddd/archive
bbb/ddd/eee              115G   267G   115G  /home/ddd/eee
bbb/ddd/old             97.7G   267G  97.7G  /home/ddd/old
bbb/ddd/fff              123G   267G  33.4G  /home/ddd/fff
bbb/ddd/fff/hhh         22.2G   267G  22.2G  /home/ddd/fff/hhh
bbb/ddd/fff/iii         20.0G   267G  20.0G  /home/ddd/fff/iii
bbb/ddd/fff/ggg         47.9G   267G  43.7G  /home/ddd/fff/ggg

ccc/ddd                  644G  2.93T  36.5G  /home/ddd
ccc/ddd/archive          270G  2.93T   270G  /home/ddd/archive
ccc/ddd/eee              115G  2.93T   115G  /home/ddd/eee
ccc/ddd/old             98.1G  2.93T  98.1G  /home/ddd/old
ccc/ddd/fff              124G  2.93T  33.9G  /home/ddd/fff
ccc/ddd/fff/hhh         22.2G  2.93T  22.2G  /home/ddd/fff/hhh
ccc/ddd/fff/iii         20.1G  2.93T  20.1G  /home/ddd/fff/iii
ccc/ddd/fff/ggg         48.0G  2.93T  43.8G  /home/ddd/fff/ggg

In this case, disk usage has only increased by less than 1%.

System data:

Code: 
NAME                     USED  AVAIL  REFER  MOUNTPOINT
aaa/ccache              2.03G  1.16T  2.03G  /home/ccache
aaa/jail                 346M  1.16T   346M  /home/jail
aaa/obj                   33K  1.16T    33K  /usr/obj
aaa/ports               1.62G  1.16T   262M  /usr/ports
aaa/ports/distfiles     1.37G  1.16T  1.37G  /usr/ports/distfiles
aaa/ports/packages        31K  1.16T    31K  /usr/ports/packages
aaa/src                  370M  1.16T   370M  /usr/src
aaa/var                  126M  1.16T   126M  /var

bbb/ccache              2.60G   267G  2.60G  /home/ccache
bbb/jail                 386M   267G   386M  /home/jail
bbb/obj                  152K   267G   152K  /usr/obj
bbb/ports               2.23G   267G   880M  /usr/ports
bbb/ports/distfiles     1.37G   267G  1.37G  /usr/ports/distfiles
bbb/ports/packages       144K   267G   144K  /usr/ports/packages
bbb/src                  513M   267G   513M  /usr/src
bbb/var                  210M   267G   210M  /var

ccc/ccache              3.76G  2.93T  3.76G  /home/ccache
ccc/jail                 466M  2.93T   466M  /home/jail
ccc/obj                  304K  2.93T   304K  /usr/obj
ccc/ports               3.01G  2.93T  1.64G  /usr/ports
ccc/ports/distfiles     1.37G  2.93T  1.37G  /usr/ports/distfiles
ccc/ports/packages       288K  2.93T   288K  /usr/ports/packages
ccc/src                  805M  2.93T   805M  /usr/src
ccc/var                  364M  2.93T   364M  /var

Excluding /usr/ports/distfiles, disk usage had more than doubled. In particular, /usr/ports had increased six folds. But this is to be expected as it consists of a lot of small text files.
 
You must log in or register to reply here.
Back
Top