Is swap on zfs safe?

V

VK

New Member


Messages: 2

Hello,

I've read a couple of years ago that swapping on zfs wasn't safe [at that time] due to the fact zfs itself might need allocating memory for writing to the swap dataset. I can't find anything related to this issue right know.

Is swap over zfs safe on FreeBSD?

Thanks.
 
gkontos

gkontos

Daemon

Reaction score: 488
Messages: 2,160

The only disadvantage is that it can not be used as a dump device.

Other than that it is absolutely safe. Just make sure you set the checksum property to off.

George
 
graudeejs

graudeejs

Son of Beastie

Reaction score: 707
Messages: 4,617

gkontos said:
The only disadvantage is that it can not be used as a dump device.

Other than that it is absolutely safe. Just make sure you set the checksum property to off.

George
Click to expand...

That is not required
 
graudeejs

graudeejs

Son of Beastie

Reaction score: 707
Messages: 4,617

gkontos said:
Sorry, what is not required ?
Click to expand...

It is not required to turn off checksum property.
However (turning it off might increase performance. I never noticed anything. So I decided to leave it on. At least I'll know when my disk start to die, even if it's swap area. I might be wrong :) )
 
gkontos

gkontos

Daemon

Reaction score: 488
Messages: 2,160

graudeejs said:
It is not required to turn off checksum property.
However (turning it off might increase performance. I never noticed anything. So I decided to leave it on. At least I'll know when my disk start to die, even if it's swap area. I might be wrong :) )
Click to expand...
I have seen cases where the checksum has caused weird messages and false alarms. IMHO you are much better off without it.

George
 
fluca1978

fluca1978

Aspiring Daemon

Reaction score: 77
Messages: 788

Sorry, I cannot see the real advantage of having swap over ZFS. I mean, ok, ZFS can help finding out disk problems, but is it really needed to have swap over such an important file system? Or are you talking about adding swap files on a ZFS file system?
 
fluca1978

fluca1978

Aspiring Daemon

Reaction score: 77
Messages: 788

graudeejs said:
I don't see disadvantages either
Click to expand...

Of course there are no drawbacks, even if having the capability to snapshot swap, do a checksum on it, and all the other features that ZFS provides sounds a bit too much to me.
 
K

kpa

Beastie's Twin

Reaction score: 1,848
Messages: 6,318

The advantage is that you don't have to make separate partition for swap. You may not even have partitions on the disks that make up your zfs pool and your system disk is so minimal that you don't want to create more than one partition on it. For example on my fileserver the OS is on two 2GB usb memory sticks configured as gmirror(8) RAID1, there just isn't enough room for swap and I didn't want to make extra partitions on the zfs pool disks for swap so I use swap on a ZVOL.
 
B

bbzz

Aspiring Daemon

Reaction score: 134
Messages: 874

Exactly; for me the only reason to have it on ZFS is because I don't have to slice up disk.
 
C

Crest

Active Member

Reaction score: 67
Messages: 211

Not checksumming swap is braindead (unless you trust your disks for some reason and want to save a few cycles) as it contains. If ZFS complains about "strange checksum errors" it most likly found disk I/O errors.
 
gkontos

gkontos

Daemon

Reaction score: 488
Messages: 2,160

There have been many arguments regarding the use of ZFS swap. Either way has its pros and cons. Allocating swap space in a ZFS pool has many advantages. For example you don't need to create separate partitions nor to load any other software mirroring module. The only disadvantage is that you can't store core dumps. This could be easily bypassed by allocating a separate device.
On the other hand, there is no point in using checksums for swap. On the contrary you are adding an extra meaningless burden. You don't rely on swap checksums to monitor the health of your drives.

PS @crest be a little more careful with your expressions.
 
phoenix

phoenix

Administrator
Staff member
Administrator
Moderator

Reaction score: 1,290
Messages: 4,099

Not sure if this has been fixed yet or not, but there's one little niggle to using swap on ZFS: you need memory to track disk usage and whatnot in the ARC. So, if you get into a situation where you need to swap something to disk, but don't have any non-wired memory to use to track the writes and whatnot ... you end up with a locked up system.

This was a big problem with ZFSv6 and ZFSv14. Not sure if it's still a problem.
 
Crivens

Crivens

Moderator
Staff member
Moderator

Reaction score: 1,681
Messages: 2,539

gkontos said:
I have seen cases where the checksum has caused weird messages and false alarms. IMHO you are much better off without it.
George
Click to expand...

Have you heard that they are now removing the FriendOrFoe alert from missile systems? There have been weird beebing noises and false alarms... ;)

Seriously, for all disks which gave out in my systems in the last years ZFS got the message way before SMART was able to find anything. The only benefit for turning off checksums for swap would be that a read error from the disk does not shoot up the system. Maybe the page is to be written, then you are save. But I do not play russian roulette. If it does not fail at once you get panics, maybe hours or days later, and have no idea where they come from.

I prefer a faulty system to halt at once so I can see what the trouble is and so I am motivated to fix it. Turning that off is IMHO like playing russian roulette. With an automatic.

Sorry if this seems harsh, but I see what a certain "this will work, 'guv" stance can get you into each day @work. Do the right thing, do not overdo it, consider each part to be a fail-waiting-to-happen. Including yourself.
 
gkontos

gkontos

Daemon

Reaction score: 488
Messages: 2,160

Crivens said:
Have you heard that they are now removing the FriendOrFoe alert from missile systems? There have been weird beebing noises and false alarms... ;)
Click to expand...

No, I haven't but I take your word for it since you seem to be the expert here ;)

Crivens said:
Seriously, for all disks which gave out in my systems in the last years ZFS got the message way before SMART was able to find anything. The only benefit for turning off checksums for swap would be that a read error from the disk does not shoot up the system. Maybe the page is to be written, then you are save. But I do not play russian roulette. If it does not fail at once you get panics, maybe hours or days later, and have no idea where they come from.
Click to expand...

SMART has nothing to do with ZFS. SMART will report a failure only after the failure has occurred. ZFS also doesn't get any message from disks. It simply rely on checksums to check the integrity of data.

Your point is that while your data is being constantly checked by checksums, you rely on SWAP which is being used as an alternative to memory, as a tool for data integrity.

In addition you are adding extra hogs to a system at a time when the system is over loaded.

Crivens said:
Sorry if this seems harsh, but I see what a certain "this will work, 'guv" stance can get you into each day @work. Do the right thing, do not overdo it, consider each part to be a fail-waiting-to-happen. Including yourself.
Click to expand...

My opinion and implementation is based upon technical facts and experience. Of course I could be wrong. But I at least I am not arrogant. I see the arrogance in some peoples work each time I am called to fix their catastrophes ;)
 
Crivens

Crivens

Moderator
Staff member
Moderator

Reaction score: 1,681
Messages: 2,539

gkontos said:
No, I haven't but I take your word for it since you seem to be the expert here ;)
Click to expand...
Not for ZFS, there are others here who know more about what to do with it. Phenix comes to mind, for example.

The joke is that the missile systems, at least those I had the questionable honor to provide service for, do not have safety measures. This reduces parts and also makes certain that whoever operates the damned thing does not rely on safety catches and is widely awake while operating it.
SMART has nothing to do with ZFS. SMART will report a failure only after the failure has occurred. ZFS also doesn't get any message from disks. It simply rely on checksums to check the integrity of data.
Click to expand...
Yes, it checks your data. And it does this IMHO much better than the SMART code in the HD. You get the notification from ZFS that something does not work out before the HD sees anything wrong. You are free to ignore such warnings if you want.
Your point is that while your data is being constantly checked by checksums, you rely on SWAP which is being used as an alternative to memory, as a tool for data integrity.
Click to expand...
No. My point is that if you can have a warning that the disks are about to fail, I want that warning ASAP. Your suggestion to disable checksums to shave a few cycles would help how exactly?
In addition you are adding extra hogs to a system at a time when the system is over loaded.
Click to expand...
Yes, and proudly. There is a fundamental difference between a slow system and a crashed system. Also, if the pool which is hosting the swap device or file is supplying redundancy you can survive even a read error upon page-in. Again, it can do that only when checking is enabled.
My opinion and implementation is based upon technical facts and experience. Of course I could be wrong. But I at least I am not arrogant. I see the arrogance in some peoples work each time I am called to fix their catastrophes ;)
Click to expand...
Which is more or less exactly what I wrote.
 
graudeejs

graudeejs

Son of Beastie

Reaction score: 707
Messages: 4,617

BTW, it I never notice when system writes to swap when I use ZFS, however I remember how I felt lags when I was using swap partition
 
S

Slurp

Member

Reaction score: 2
Messages: 67

To those who say that they use swap on ZFS and it's safe, did you try to stress it?
I.e. write a program that randomly accesses 2x physical memory for a couple of minutes.
 
fluca1978

fluca1978

Aspiring Daemon

Reaction score: 77
Messages: 788

Uhm...if the point of having swap on ZFS is that it can help finding problems on disks, why not doing a memory disk using ZFS? That will help finding errors in memory too. I'm not joking, that is one of the ZFS strength.
However, I agree that using ZFS for swap means using a fat system when the whole machine is overloaded, so it does not seems the right choice to do. It is also true that, my experience, people tend to use ZFS on a set of disks, not a single disk, and here the RAID could split I/O and make swapping less overloading that the single-partition counterpart.
Is there any real stress-test/comparison about using a slice vs zfs swap? That could give us some real answers...
 
graudeejs

graudeejs

Son of Beastie

Reaction score: 707
Messages: 4,617

Right, and you expect swap to be use all the time... I don't I expect swap to not be used... If it's used too much, it means that you need more RAM
 
S

Slurp

Member

Reaction score: 2
Messages: 67

graudeejs said:
Right, and you expect swap to be use all the time... I don't I expect swap to not be used... If it's used too much, it means that you need more RAM
Click to expand...

I have a workload that could benefit from petabytes of RAM. I can't afford it, so I push what I have. For this reason, in the predictable future my swap will be used heavily.:)
 
Crivens

Crivens

Moderator
Staff member
Moderator

Reaction score: 1,681
Messages: 2,539

Slurp said:
I have a workload that could benefit from petabytes of RAM. I can't afford it, so I push what I have. For this reason, in the predictable future my swap will be used heavily.:)
Click to expand...

Fluid dynamics simulations? (I know when my system starts to hit the page store ;))
 
You must log in or register to reply here.
Top