• This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.

Solved Reservations, quotas, and /tmpfs

mefizto

Well-Known Member

Thanks: 6
Messages: 372

#1
Greetings all,

learning more about zfs, I have been l wondering if it would not be a good idea to use reservations and quotas on some of the data-sets. The motivation would be to prevent filling the (common) pool by a runaway process (or an attack?).

I have been specifically contemplating about reserving about 2GB on the /, and perhaps some quota(s) on /var data-set(s). In this regard, I have been also wondering about /tmp.

As I understand it, there is a motivation to put /tmp on tmpfs(5) to use the RAM for fast access. However, it seems that by doing this (1) RAM resource is taken from other (perhaps more important) tasks and (2) my concern regarding filling the /tmp appears to have even more dire consequences. I do understand that the OS will use /swap to back the /tmp, but I just do not know how well the OS is at the memory management. Based on the foregoing, I wonder if, given the use of SSD, it is worth to use tmpfs(5) or use /tmp with a reasonable quota.

I have not been able to find much information regarding the issues, so any opinions/experiences would be appreciated.

Kindest regards,

M
 

ShelLuser

Son of Beastie

Thanks: 1,257
Messages: 2,671

#2
It definitely won't hurt. I have something likewise set up on my own servers. Whether or not this is usable and applicable depends on your setup. For example: I think that using this for the root filesystem is a bit overdoing it, also because it normally isn't used for storage. Of course this depends on the way you set it up. Also keep in mind that some reservation settings apply on the filesystem as well as any descendants. See the ZFS manualpage for more information,
 

phoenix

Administrator
Staff member
Administrator
Moderator

Thanks: 1,124
Messages: 3,944

#3
Setting a reservation on / or creating a separate filesystem with a reservation, is a good idea. That way, if something uses up all the space in the pool, you'll be able to lower the reservation to add free space to the pool, and then be able to delete items (free space is needed to delete items from the pool). I create a separate filesystem called do-not-delete for this purpose on all my ZFS pools. :)

I also set a quota on /tmp and a couple of other filesystems (security camera home directory, for instance) to prevent them from filling the pool. :)

Using tmpfs(5) works as well, especially if you have lots of RAM (16 GB+), or you don't use a lot of RAM for normal operations.

The OS will generally better at resource management that you will be, but setting limits can be useful. :)
 

mefizto

Well-Known Member

Thanks: 6
Messages: 372

#4
Gentlemen,

first thank you for the replies. In general, I do not see any harm in using quotas on zfs because should I misjudge the quota size, I can always increase it - provided, of course, that there is free space available. And to ensure that there is free space available, especially if the pool accidentally fills over 80% and locks, setting a reservation greater than needed so that it can be lowered, seems desirable.

ShelLuser,

can you please elaborate regarding your reasoning re the root? Looking at your setup in the How-To section, your root is part of a single pool. So even though / does not store data, if a process runs away, the root might be affected. Or, did I misunderstood and you disagree with the size of the reservation? But, please see phoenix' reasoning about lowering the reservation to recover the pool.

Also, could you post some generic ideas regarding your approach to reservations/quotas?

phoenix,

can you please elaborate regarding your creating a separate filesystem called do-not-delete helps? If the filesystem is part of a (single) pool, it might still be filled. Or does this filesystem have the reservation set?

Are you not using tmpfs(5)?

Kindest regards,

M
 

lkateley

Member

Thanks: 8
Messages: 21

#5
I almost always tell people who are new to zfs to use both quotas and reservations when you start.. But I very rarely find a need for them unless i want to control. With disk sizes being what they are, even a ssd might have plenty of space for /var.
 

phoenix

Administrator
Staff member
Administrator
Moderator

Thanks: 1,124
Messages: 3,944

#6
phoenix,

can you please elaborate regarding your creating a separate filesystem called do-not-delete helps? If the filesystem is part of a (single) pool, it might still be filled. Or does this filesystem have the reservation set?
The latter. It's a separate filesystem, set as readonly, with a 1 GB reservation set. It's not used for anything, other than to keep a forced set of space "available" in the pool for administrative purposes.

Basically, I ran a pool to 100% full a couple of times and could not delete snapshots to free up space (you need free space in the pool to track the metadata for the snapshots). I got lucky, and had a different filesystem in the pool without any snapshots and I was able to remove a file from there, freeing up enough space to allow for snapshot deletion. After that happened a second time (again, getting lucky in having a snapshot-free filesystem I could delete files from), I came up with the do-not-delete filesystem with a 1 GB reservation. (Pretty sure I stole the idea from a mailing list post or something like that.) :)

Now, every ZFS pool I manage has that filesystem on it. And it's saved my bacon multiple times per year on our backups storage systems.

Very new versions of OpenZFS support this internally now (supposedly), where they reserve a tiny fraction of the pool "for administrative purposes", such that a pool that is 100% full will still allow you to delete things. However, I don't trust that fully, and continue to keep the do-not-delete filesystem around. :)

Are you not using tmpfs(5)?
Not on my home system, no. At work, I am.
 

mefizto

Well-Known Member

Thanks: 6
Messages: 372

#7
Hi lkateley,

interesting approach. I would think that initially people would start with the default as installed and only after getting some experience/different needs start with more advanced topics. As to your statement re disk sizes, although I agree, please that my motivation is not due to lack of space, but prevention or at least mitigation of potential problems.

Hi phoenix,

thank you for the explanation and your experience.

Kindest regards,

M
 
Top