ZFS: limit available space in /

O

oliver@

Developer
Hi,

I've set up a FreeBSD box using RootOnZFS.
I always liked the idea of having different filesystems before I used ZFS to prevent some stupid process filling up my complete harddisk by writing junk into /tmp.

With the standard configuration this seems to be possible in ZFS as all partitions are sharing the same pool so each "partition" can fill up the whole pool leaving no space to the other partitions:

Code: 
nudel# df -h
Filesystem                    Size    Used   Avail Capacity  Mounted on
zroot                         904G    332M    903G     0%    /
devfs                         1.0k    1.0k      0B   100%    /dev
zroot/tmp                     903G     36k    903G     0%    /tmp
zroot/usr                     904G      1G    903G     0%    /usr
zroot/usr/ports               904G    717M    903G     0%    /usr/ports
zroot/var                     903G    370k    903G     0%    /var
/dev/ufs/files                2.7T    576G    1.9T    23%    /mnt/files

Now I searched a bit on the internet and found the quota parameter for ZFS and used it:

Code: 
nudel# zfs set quota=512M zroot/tmp
nudel# zfs set quota=512M zroot/var
nudel# zfs set quota=20G zroot/usr/ports
nudel# zfs set quota=200G zroot/usr
dnudel# df -h
Filesystem                    Size    Used   Avail Capacity  Mounted on
zroot                         904G    332M    903G     0%    /
devfs                         1.0k    1.0k      0B   100%    /dev
zroot/tmp                     512M     70k    512M     0%    /tmp
zroot/usr                     199G      1G    198G     0%    /usr
zroot/usr/ports                20G    717M     19G     4%    /usr/ports
zroot/var                     512M    450k    511M     0%    /var
/dev/ufs/files                2.7T    576G    1.9T    23%    /mnt/files

But now, It would be still possible to write junk into my root partition and the free space would go away from the other partitions as well.
I'm not able to apply a quota of 1G to the root partition itself:

Code: 
nudel# zfs set quota=1G zroot
cannot set property for 'zroot': size is less than current used or reserved space

I also found out, that there is a reservation setting to just reservate the space for a partition so it could not be used by any other partition. But then the filesystem would appear as "full" in df when I wanted to reservate the size of the filesystem for the filesystem (check http://docs.huihoo.com/opensolaris/solaris-zfs-administration-guide/html/ch05s06.html)

Does a way exist to add a quota to the root filesystem as well So I would have a quota to every filesystem so that if the sum of each "avail" column of the df listing (=quota) is below the overall space of the zpool, it is guaranteed that this space is available to the partition whatever happens on another partition?!
 
The only way i know of is with
Code: 
zfs set reservation=<size>
and yes your
Code: 
df
is going to not be accurate. When using zfs i would not be very dependent on
Code: 
du
or
Code: 
df
. The zfs and zpool commands will provide more accurate information in those cases that you need to get data usage information.
 
Would it be possible to create a new ZFS called zroot/root?
Then move everything from zroot to zroot/root, mark zroot/root as bootfs, mount zroot/root to /.
Then "zroot" would be not in use anywhere.
Would zroot/root then be quota-able I guess?.
Is this a possible configuration? would FreeBSD boot from zroot/root?
Any idea how the migration could look like?

Would zroot/root then be quota-able?

Why am I asking the last question?
What I noticed is with a /usr zfs and a /usr/ports zfs is the following:

1st situation:
Code: 
nudel# df -h /usr /usr/ports
Filesystem                    Size    Used   Avail Capacity  Mounted on
zroot/usr                     199G    368M    199G     0%    /usr
zroot/usr/ports                20G    721M     19G     4%    /usr/ports

next step:
Code: 
nudel# zfs set quota=[color=blue]400G[/color] zroot/usr/ports
nudel# df -h /usr /usr/ports
Filesystem                    Size    Used   Avail Capacity  Mounted on
zroot/usr                     199G    368M    199G     0%    /usr
zroot/usr/ports               [color=blue]199G[/color]    721M    199G     0%    /usr/ports

next step:

Code: 
nudel# zfs set quota=[color=blue]100G[/color] zroot/usr
nudel# df -h /usr /usr/ports
Filesystem                    Size    Used   Avail Capacity  Mounted on
zroot/usr                      99G    368M     99G     0%    /usr
zroot/usr/ports                [color=blue]99G[/color]    721M     99G     1%    /usr/ports
nudel# 
nudel# zfs get quota zroot/usr zroot/usr/ports
NAME             PROPERTY  VALUE  SOURCE
zroot/usr        quota     100G   local
zroot/usr/ports  quota     400G   local
nudel# 
nudel# zfs list zroot/usr zroot/usr/ports
NAME              USED  AVAIL  REFER  MOUNTPOINT
zroot/usr        1.06G  98.9G   368M  legacy
zroot/usr/ports   722M  98.9G   722M  legacy

Is this just a display problem of df, or may I only 100G into /usr/ports because of the /usr quota even if I have a zroot/usr/ports quota of 400G? Are the quotas hierarchical?
 
Regarding to http://blogs.oracle.com/markm/ quota seems to be indeed hierarchical.

So having 200G quota in /usr and 70G quota in /usr/ports, it could be possible, that something fills up all 200G of /usr, and then /usr/ports will also have no data available as the quota of the upper filesystem is already reached.

Damn - this is kinda bad for my wanted configuration :(
 
And again me (oh I hate this....)

this could probably help:

Code: 
       refquota=size | none

           Limits the amount of space a dataset  can  consume.  This  property
           enforces  a hard limit on the amount of space used. This hard limit
           does not include space used by descendents, including file  systems
           and snapshots.

Code: 
nudel# zfs set quota=none zroot/usr
nudel# zfs set quota=none zroot/usr/ports
nudel# zfs set refquota=200G zroot/usr/ports
nudel# zfs set refquota=100G zroot/usr
nudel# df -h zroot/usr zroot/usr/ports
Filesystem                    Size    Used   Avail Capacity  Mounted on
zroot/usr                     100G    368M     99G     0%    /usr
zroot/usr/ports               200G    721M    199G     0%    /usr/ports
nudel#
 
I would advise first reorganize your filesystem as follows:

Code: 
#zfs create -o mountpoint=legacy zroot/root
#zfs set mountpoint=/tmp zroot/tmp
#zfs set mountpoint=/var zroot/var
#zfs create -o mountpoint=/usr/tmp zroot/local
#zfs rename zroot/usr/ports zroot/ports
#zfs set mountpoint=/usr/ports zroot/ports

Copy everything in /usr/local to /usr/tmp. Then

# zfs set mountpoint=/usr/local zroot/local

Now, it depends whether you want to compile from source or not.
I would recommend reading the following excellent tips if you decide to compile from source

https://forums.freebsd.org/showthread.php?t=6230

Post #5 is particularly relevant. Just remember to do a

# mount -t zfs zroot/root /tmproot

and point DESTDIR to /tmp/root.

Otherwise, you could
Code: 
#zfs snapshot zroot@now
#zfs send zroot@now | zfs recv zroot/root@now
#mount -t zfs zroot/root /tmproot

Then copy everything in /usr except /usr/local into /tmproot/usr.

In either cases, set the bootfs property on zroot and modify /tmproot/boot/loader.conf by changing zfs:zroot to zfs:zroot/root. Finally set the mountpoint of zroot to none (in single user mode if necessary).

I hope this covers all and there is no mistake except typos.
 
Thanks - but by using refquota, creating a new rootfs does not seem to be necessary anymore.
 
You must log in or register to reply here.
Back
Top