Swap and ZFS

C

ctengel

Quick question I'd like to throw out there: Is it (at this point in time, on FreeBSD 9.0-RELEASE) a good idea to put swap on a ZFS Zvol?

I've read alot either way, and I'm wondering if maybe it used to be a bad idea but now it's not due to some fix. I can understand why it wouldn't be, but at the same time if it works well nowadays it would be convenient for me. We do it on Solaris 10 extensively.

Right now I have 8GB of RAM. The bsdinstall program alloted 190MB of swap automatically from my 4GB rootdisks. Honestly I'm not sure how much I need for my purposes, but I feel like 190MB may not be enough. Years ago I remember a rule of thumb of doubling the memory to determine swap, but I'm pretty sure that's overkill. In an event that I do need more, but zvols are a bad idea I can just partition some off from the media zpools, but I would have to do that now, before I put anything on it... ;)
 
No, you should not put swap on ZFS. You will crash the system. Search the forums for the gory details of why it crashes.
 
I use swap with zfs on FreeBSD 9 with zpool v28. No problem on my AMD quad core with 8GB RAM.
At work, I was using FreeBSD-8-STABLE with zpool v28. System had 3GB ram. Still no problem. Even with ~500MB swap usage
 
I used a separate partition, just to be safe.
Code: 
dice@williscorto:~>zfs list
NAME             USED  AVAIL  REFER  MOUNTPOINT
zroot           13.8G   211G   766M  legacy
zroot/usr       13.0G   211G  3.23G  /usr
zroot/usr/home  9.73G   211G  9.73G  /usr/home
zroot/var       48.4M   211G  34.4M  /var
zroot/var/log   13.9M   211G  13.9M  /var/log
dice@williscorto:~>gpart show 
=>       34  488397101  ada0  GPT  (232G)
         34        128     1  freebsd-boot  (64k)
        162    8388608     2  freebsd-swap  (4.0G)
    8388770  480008365     3  freebsd-zfs  (228G)

There's 2GB of memory in this machine, I've set the swap to 4GB. Instead of wasting that space I've enabled tmpfs(5) in /etc/fstab:
Code: 
tmpfs	/tmp	tmpfs	rw,mode=1777	0	0

Code: 
dice@williscorto:~>df -h
Filesystem                         Size    Used   Avail Capacity  Mounted on
zroot                              211G    766M    210G     0%    /
devfs                              1.0k    1.0k      0B   100%    /dev
tmpfs                              3.3G     12k    3.3G     0%    /tmp
procfs                             4.0k    4.0k      0B   100%    /proc
fdescfs                            1.0k    1.0k      0B   100%    /dev/fd
linprocfs                          4.0k    4.0k      0B   100%    /compat/linux/proc
zroot/usr                          213G    3.2G    210G     2%    /usr
zroot/usr/home                     220G    9.8G    210G     4%    /usr/home
zroot/var                          210G     34M    210G     0%    /var
zroot/var/log                      210G     13M    210G     0%    /var/log
 
I have 3 FreeBSD servers with ZFS swap running on these, 2 of the servers storing backup keep crashing randomly but another one housing VMWARE virtual machine still survives until now for more than 14 days.
 
graudeejs said:
I use swap with zfs on FreeBSD 9 with zpool v28. No problem on my AMD quad core with 8GB RAM.

At work, I was using FreeBSD-8-STABLE with zpool v28. System had 3GB ram. Still no problem. Even with ~500MB swap usage
Click to expand...

Watch your Free, Cached, and Inactive RAM in top. When those get near to 0, and you start swapping like crazy (meaning you are actually out of RAM, and need to use swap), things will go BOOM!

As long as you have Free, Cached, or Inactive RAM, you are not "out of memory", and swap is still optional (basically only used for stuff that hasn't been accessed in a *long* time).

Fire something up that needs a tonne of RAM (like 2x RAM, so it will definitely swap like mad), and you'll crash the system.

Swap-on-ZVol is very stable ... until you actually hammer the swap subsystem. :) Then things go very awry due to ZFS needing RAM for ARC to track swap usage ... but not having any RAM due to swapping.
 
So zfs-based swap would become stable if you could "somehow" reserve/protect the portion of RAM that zfs uses = can't touch this!
 
Hmm
So I guess (or I hope!) Solaris has some sort of protections to allow a Zvol as swap. I think I understand now how it wouldn't be much help on FreeBSD under heavy load.

I guess my next question (which may be beyond the scope of this thread) is whether I even need any or if I do how much (originally I was just going to push this question off by just creating swap Zvols as necessary...) Should I be following the 2*RAM rule? The only reason I have so much RAM (8GB) is basically just for ZFS's cache (which obviously swap wouldn't help much with). If I limit ZFS's use of RAM to maybe say 6GB, would I have any issues with no swap at all?

And secondarily, is there any reason to mirror swap on a non-mission-critical system? What's the worst that can happen?
 
I'd create something like an 8GB swap and setup tmpfs(5). You don't want to turn off swap completely, modern operating systems all use some swap. If only for some housekeeping. By using tmpfs(5) you're not really wasting that space as it will get mounted on /tmp/ where it can be used.
 
Hmm OK I think I will probably partition some off from my zpool drives for swap. Unfortunately though I will not likely be able to use /tmp for much, since I don't trust my power that much!

Is it a good idea to mirror swap, with, say gmirror(8)?
 
ctengel said:
Is it a good idea to mirror swap, with, say gmirror(8)?
Click to expand...
That depends, can you handle a crash if one of the drives containing swap dies? For me it doesn't really matter so I don't use it.
 
Yes I think I can deal with a crash, as long as no data loss.

gmirror sounds like it does not have the same issues with requiring memory to do it's thing as is the case with ZFS, correct?
 
OK thanks. Wasn't sure if maybe it did block-level caching or something. In any event I've partitioned off some space from my zpool disks, totaling two times my RAM, so I think that should be enough. Haven't activated it yet, but haven't really started doing anything yet either.
 
You must log in or register to reply here.
Back
Top