ZFS raidz space

garig0

garig0

Hi all,

I'm running FreeBSD 8.2 with ZFS with RAIDZ configuration. I meet some problems with disk space.

Code: 
[root@srv-backup ~]# uname -a
FreeBSD srv-backup.company.com 8.2-RELEASE-p3 FreeBSD 8.2-RELEASE-p3 #0: Tue Sep 27 18:45:57 UTC 2011     
[email]root@amd64-builder.daemonology.net[/email]:/usr/obj/usr/src/sys/GENERIC  amd64

My pool contain 4x 2TO (+ 1 spare)
RAW space with raid : N-1 =~ 6TB
So I obtain a pool size of 5,33TB .

I've got 2 volumes, vol1 and vol2; vol2 is 3 TB size and it's presented in iSCSI. (type volume because fixed size); vol1 is expected to take the rest of the space: 2,33TB (type filesystem).

Nevertheless, I've a lack of space with my pool and vol1. When I run a zfs list, I've no space left on my pool01, and I've only 3,35TB USED.


Code: 
[root@srv-backup ~]# zfs list
NAME               USED  AVAIL  REFER  MOUNTPOINT
pool01            5.33T    0    2.02T  /pool01
pool01/vol1       325G     0    319G   /pool01/vol1
pool01/vol2       3T     3.01T  23.9K  -

I read some information about overhead and metadata, but I really don't understand why I've no available space.


Code: 
[root@srv-backup ~]# zpool list
NAME     SIZE   USED  AVAIL    CAP  HEALTH  ALTROOT
pool01  7.25T  3.12T  4.13T    43%  ONLINE  -



Thanks in advance for helping me :)


Regards,

garig0
 
@garig0

Perhaps you have quota or reservation on the pool or the file system
# zfs get -r all pool01
could sched some more light.

/Sebulon
 
@Sebulon

Hi Sebulon,

thanks for your reply, I've neither quota or reservation.
Here is the output of #zfs get -r all pool01

Code: 
[root@srv-backup ~]# zfs get -r all pool01
NAME              PROPERTY              VALUE                  SOURCE
pool01            type                  filesystem             -
pool01            creation              Mon Aug  1 17:21 2011  -
pool01            used                  5.33T                  -
pool01            available             0                      -
pool01            referenced            2.02T                  -
pool01            compressratio         1.00x                  -
pool01            mounted               yes                    -
pool01            quota                 none                   default
pool01            reservation           none                   default
pool01            recordsize            128K                   default
pool01            mountpoint            /pool01                default
pool01            sharenfs              off                    default
pool01            checksum              on                     default
pool01            compression           off                    default
pool01            atime                 on                     default
pool01            devices               on                     default
pool01            exec                  on                     default
pool01            setuid                on                     default
pool01            readonly              off                    default
pool01            jailed                off                    default
pool01            snapdir               hidden                 default
pool01            aclmode               groupmask              default
pool01            aclinherit            restricted             default
pool01            canmount              on                     default
pool01            shareiscsi            off                    default
pool01            xattr                 off                    temporary
pool01            copies                1                      default
pool01            version               4                      -
pool01            utf8only              off                    -
pool01            normalization         none                   -
pool01            casesensitivity       sensitive              -
pool01            vscan                 off                    default
pool01            nbmand                off                    default
pool01            sharesmb              off                    default
pool01            refquota              none                   default
pool01            refreservation        none                   default
pool01            primarycache          all                    default
pool01            secondarycache        all                    default
pool01            usedbysnapshots       0                      -
pool01            usedbydataset         2.02T                  -
pool01            usedbychildren        3.31T                  -
pool01            usedbyrefreservation  0                      -
pool01/vol1     type                  filesystem             -
pool01/vol1     creation              Mon Aug  1 17:31 2011  -
pool01/vol1     used                  325G                   -
pool01/vol1     available             0                      -
pool01/vol1     referenced            325G                   -
pool01/vol1     compressratio         1.00x                  -
pool01/vol1     mounted               yes                    -
pool01/vol1     quota                 none                   default
pool01/vol1     reservation           none                   default
pool01/vol1     recordsize            128K                   default
pool01/vol1     mountpoint            /pool01/vol1           default
pool01/vol1     sharenfs              on                     local
pool01/vol1     checksum              on                     default
pool01/vol1     compression           off                    default
pool01/vol1     atime                 on                     default
pool01/vol1     devices               on                     default
pool01/vol1     exec                  on                     default
pool01/vol1     setuid                on                     default
pool01/vol1     readonly              off                    default
pool01/vol1     jailed                off                    default
pool01/vol1     snapdir               hidden                 default
pool01/vol1     aclmode               groupmask              default
pool01/vol1     aclinherit            restricted             default
pool01/vol1     canmount              on                     default
pool01/vol1     shareiscsi            off                    default
pool01/vol1     xattr                 off                    temporary
pool01/vol1     copies                1                      default
pool01/vol1     version               4                      -
pool01/vol1     utf8only              off                    -
pool01/vol1     normalization         none                   -
pool01/vol1     casesensitivity       sensitive              -
pool01/vol1     vscan                 off                    default
pool01/vol1     nbmand                off                    default
pool01/vol1     sharesmb              off                    default
pool01/vol1     refquota              none                   default
pool01/vol1     refreservation        none                   default
pool01/vol1     primarycache          all                    default
pool01/vol1     secondarycache        all                    default
pool01/vol1     usedbysnapshots       0                      -
pool01/vol1     usedbydataset         325G                   -
pool01/vol1     usedbychildren        0                      -
pool01/vol1     usedbyrefreservation  0                      -
pool01/vol2  type                  volume                 -
pool01/vol2  creation              Fri Aug 26 18:04 2011  -
pool01/vol2  used                  3T                     -
pool01/vol2  available             3.01T                  -
pool01/vol2  referenced            23.9K                  -
pool01/vol2  compressratio         1.00x                  -
pool01/vol2  reservation           none                   default
pool01/vol2  volsize               3T                     -
pool01/vol2  volblocksize          8K                     -
pool01/vol2  checksum              on                     default
pool01/vol2  compression           off                    default
pool01/vol2  readonly              off                    default
pool01/vol2  shareiscsi            off                    default
pool01/vol2  copies                1                      default
pool01/vol2  refreservation        3T                     local
pool01/vol2  primarycache          all                    default
pool01/vol2  secondarycache        all                    default
pool01/vol2  usedbysnapshots       0                      -
pool01/vol2  usedbydataset         23.9K                  -
pool01/vol2  usedbychildren        0                      -
pool01/vol2  usedbyrefreservation  3.00T                  -

Thanks you.

garig0
 
@garig0

Code: 
pool01       usedbydataset         2.02T                  -
pool01/vol1  usedbydataset         325G                   -
pool01/vol2  usedbyrefreservation  3.00T                  -

2.02 + 0.325 + 3.00 = 5.345 TB

So the pool is full. You have 2TB stored under /pool01 and a zvol is immediately referenced when you create it. Unlike a zfs file system, a zvol is not "thin provisioned".

/Sebulon
 
@Sebulon

Thanks again for your reply. Why the zvol referenced immediately the space? Is there a way to reduce this space to set a new volsize at my vol1?


garig0
 
@garig0

A zvol isn´t "thin provisioned" by default, but you can turn into that by setting:
# zfs set refreservation=none pool01/vol2

Also, in the future if you want to create a thin zvol right away, you can do:
# zfs create [b]-s[/b] -V 3p pool01/petavol

/Sebulon
 
In fact, I totally understand why vol2 is taking 3TB of space. Because Ii fixed it to 3TB.

But my problem is why vol1 is not growing fill the pool01 automatically (the refreservation is set to none everywhere). Maybe I did an error during my configuration.

I wanted 1 volume non thin of 3TB. I think it's ok with vol2.
AND
1 mount point (filesystem type) which grow fill the pool. (so thin provisioned)

I show you an extract from "zpool history"

Code: 
zpool create pool01 raidz ada1 ada2 ada3 ada4
zpool add pool01 spare ada5
zfs create pool01/vol1
zfs set sharenfs=on pool01/vol1
zfs create -V 3TB pool01/vmbackups

Did I miss something?

Thank you again for your help.

garig0
 
@garigo

Your data is stored under mountpoint /pool01, not mountpoint /pool01/vol1
Code: 
[root@srv-backup ~]# zfs list
NAME               USED  AVAIL  REFER  MOUNTPOINT
pool01            5.33T    0    2.02T  /pool01
pool01/vol1       325G     0    319G   /pool01/vol1
pool01/vol2       3T     3.01T  23.9K  -
That is why you cannot "grow" vol1 any more, because it's already used directly under your pool pool01.

I would suggest you:
# zfs set refreservation=none pool01/vol2
# cd /pool01
# find . -xs | cpio -pv /pool01/vol1
# find . -xs -exec rm -rf {} \;
# cd /
# zfs set refreservation=3t pool01/vol2
# zfs set quota=2.3t pool01/vol1
# zfs set reservation=2.3t pool01/vol1
# zfs set mountpoint=none pool01
# zfs set mountpoint=/pool01/vol1 pool01/vol1

For you to achieve the layout ( I think ) you desire:
Code: 
[root@srv-backup ~]# zfs list
NAME               USED  AVAIL  REFER  MOUNTPOINT
pool01            5.33T    0    0      none
pool01/vol1       2.3T     0    2.3T   /pool01/vol1
pool01/vol2       3T     3.01T  23.9K  -

/Sebulon
 
@sebulon

Thank you for your help Sebulon.

Your suggestion was pretty good!
Now it's works as we want.

Have a good day.

garig0
 
You must log in or register to reply here.
Back
Top