ZFS Running swap on ZFS

Hi all,

After participating in Thread 50987 I have a question my "google-foo" can't seem to find answer to and didn't want to run the mentioned thread off topic.

I've been under the impression for a couple years now that running swap on ZFS was a bad idea due to possible memory exhaustion and instead should use a swap partition. I've not seen anywhere that this has changed and was wondering if anyone had any concrete knowledge they could share in regards to this. This doesn't affect me now as I've maxed out the memory in my system and run/test CURRENT so want a swap partition for dumps, however it would be nice to know for sure for future systems I may put together.
 
Thanks for this posting, we definitively should know more about swap on Zfs.
zfs(8) does not know the word swap, but there is an option that could be used to create it:
Code:
 # zfs create -V seize \
                    [b]-o org.freebsd:swap=on[/b] \
                    -o checksum=off \
                    -o compression=off \
                    -o dedup=off \
                    -o sync=disabled \
                    -o primarycache=none \
                    poolname/swap
What does it mean that even the manual does not know swap? Is it a hidden feature or is the manual not complete?

I do use swap on Zfs for quite some time now, but I do not need it for core dumps as they are disabled here. Hint from junovitch:
if you need to do core dumps in the event of a kernel panic the kernel cannot write to a real file system as it could cause corruption. You would need a swap partition to capture the core dump.
The first time I noticed that my swap on Zfs got used was due to a misconfiguration of Poudriere (the tread protocelt refers to) which freezes the system.
It is still not known what actually caused the system going out of memory. Was it the program running or the swap on Zfs? I tend to the first, but it has to be tested which I do not know how to do.

PS:
I found this on the topic:
http://www.schmidp.com/2014/01/05/freebsd-10-does-swap-work-on-a-zvol/
 
The zfs(8) manual page doesn't know about swap because swap on ZFS is just a clever application of a more general concept, ZVOLs that behave like block devices. The org.freebsd:swap property is the "glue" that allows the system to use the swap automatically, the /etc/rc.d/zvol script handles the initialization on boot up.
 
The zfs(8) manual page doesn't know about swap because swap on ZFS is just a clever application
A "clever application" should have a clever documentation, which also should give some information whether this kind of swap is suitable for dumps or not.
 
A "clever application" should have a clever documentation, which also should give some information whether this kind of swap is suitable for dumps or not.

Proper place to document it would be in my opinion swapon(8) since it's the utility that handles turning swap on. Its manual page even makes mention of md(4) devices so ZVOLs are not that far off.

Maybe a new manual page in category 7 that would explain the ins an outs of swap on anything, files, ZVOLs, younameit. swap(7).
 
Thanks for this posting, we definitively should know more about swap on Zfs.
zfs(8) does not know the word swap, but there is an option that could be used to create it:
Code:
 # zfs create -V seize \
                    [b]-o org.freebsd:swap=on[/b] \
                    -o checksum=off \
                    -o compression=off \
                    -o dedup=off \
                    -o sync=disabled \
                    -o primarycache=none \
                    poolname/swap
What does it mean that even the manual does not know swap? Is it a hidden feature or is the manual not complete?

I do use swap on Zfs for quite some time now, but I do not need it for core dumps as they are disabled here. Hint from junovitch:

The first time I noticed that my swap on Zfs got used was due to a misconfiguration of Poudriere (the tread protocelt refers to) which freezes the system.
It is still not known what actually caused the system going out of memory. Was it the program running or the swap on Zfs? I tend to the first, but it has to be tested which I do not know how to do.

PS:
I found this on the topic:
http://www.schmidp.com/2014/01/05/freebsd-10-does-swap-work-on-a-zvol/
In fact I had originally added that link in my original reply to your thread but after thinking about it, retracted it due to it being the only recent link I could find. My guess seems to be so far that the potential issues with swap on ZFS has not changed and depending on your setup and use case should still seriously consider using a swap partition instead.
 
Proper place to document it would be in my opinion swapon(8) since it's the utility that handles turning swap on. Its manual page even makes mention of md(4) devices so ZVOLs are not that far off.

Maybe a new manual page in category 7 that would explain the ins an outs of swap on anything, files, ZVOLs, younameit. swap(7).
The man page is the first place I looked for an answer. This would certainly be helpful.
 
I used to run my swap on ZFS (zvol) on most of my zroot installations. IMHO performance is better if you don't. But I never experienced any problems whatsoever.
 
Running swap on zvol is a bad idea, it will eventually crash the server when trashing happens.

You can simulate the trashing by running the following Perl script, it will eventually fill up all your available memory.
Code:
#!/usr/bin/perl
my $count=0;
my @data;
my $temp_data;
for(my $i=0;$i<10000000;$i++) {
        $temp_data.="1234567890abcdefghijklmnopqrstuvwxyz";
}


while(1) {
        $data[$count++]=$temp_data;
}
Tested on FreeBSD 10.1 stable
With zvol swap - my 8GB ram FreeBSD server will stall within 1 minutes.
without swap or dedicated swap disk/partition - server will automatically kill that Perl process.
 
I would be happy to file a PR on this myself, however I don't feel qualified enough to be honest. I can only look at this from a simple user's perspective. Is this a technical limitation of FreeBSD's implementation of ZFS or bug that there is intention of fixing or can be fixed at some point? Is this a technical limitation of ZFS itself that is accepted and thus a possible documentation issue? I don't know.
 
To jump in 7 plus months later, it looks like it can be used with some sysctl parameters, judging by comment 16 of the PR. Documentation still seems lacking. Is there anywhere where the option ... -o org.freebsd:swap=on is explained? I'm looking though the zfs(8) and swapon(8) man pages and don't see it listed.
(I haven't given it any serious testing, just added a couple of gigs of swap on a somewhat underpowered machine. but at least the machine hasn't become unresponsive.)
 
Last edited by a moderator:
Is there anywhere where the option ... -o org.freebsd:swap=on is explained?

This seems to be the only place:
Code:
% grep org.freebsd /etc/rc.d/zvol
  # Enable swap on ZVOLs with property org.freebsd:swap=on.
  zfs list -H -o org.freebsd:swap,name -t volume | \
  # Disable swap on ZVOLs with property org.freebsd:swap=on.
  zfs list -H -o org.freebsd:swap,name -t volume | \
 
Last edited by a moderator:
Technically:
Dataset is the generic term for a ZFS file system, volume, snapshot or clone.
A ZFS file system is just one type of ZFS dataset, so he is correct (although putting zvol would of probably been more appropriate as it's the only type of dataset that should be used for swap). There's confusion because people often use "zfs dataset" or "zvol / zfs volume" when they really mean a "zfs file system" or "zvol". Both are datasets.

Edit: If anyone if wondering what this is going on about, it's in reference to a since deleted post.
 
To jump in 7 plus months later, it looks like it can be used with some sysctl parameters, judging by comment 16 of the PR. Documentation still seems lacking. Is there anywhere where the option -o org.freebsd:swap=on is explained? I'm looking though the zfs and swapon man pages and don't see it listed.
(I haven't given it any serious testing, just added a couple of gigs of swap on a somewhat underpowered machine. but at least the machine hasn't become unresponsive.)

It's a user-defined property set on the ZFS dataset. User-defined properties are mentioned in the man pages. The "best-practise" for naming user-defined properties is to use reverse-domain notation, where you use your web domain in reverse, then a colon, then the name of the property. So, for properties defined by the FreeBSD project, they would start with org.freebsd.

User-defined properties are not used by ZFS directly. They are just meta-data attached to datasets that users can then query to do different things with.

For example, I've been toying with using user-defined properties to set the expiration date for snapshots at snapshot creation time, and then just running a cronjob that checks if today's date is greater than the expiration date to determine whether to delete the snapshot or not.
 
Back
Top