ZFS What to do with extra NVMe SSD after adding two HDDs that mirror each other?

D

Deleted member 65953

Guest
I have a desktop that has a 512 GB NVMe SSD. I am planning to repurpose the desktop as a file server by adding two 10 TB hard disks, and installing FreeBSD with ZFS on those two 10 TB hard disks (RAID 1, i.e. two disk mirror). It seems that I no longer have any use for the 512 GB SSD in this desktop, since I don't have anything to mirror its data to. Is there anything I can safely use the SSD for in this desktop? Could it be used to somehow speed up ZFS? Could it safely be used for /tmp?
 
you can use it as L2ARC (cache), ZIL (intent log) or both depending on zpool load type
see zpoolconcepts(8)
 
If you can boot off the NVMe device, why not use that for the OS (with ZFS so you have boot environments) and then the 10TB mirror for the data you are going to serve?
I've always tried to separate OS and Data devices even if the OS device is not redundant.
 
you can use it as L2ARC (cache), ZIL (intent log) or both depending on zpool load type
see zpoolconcepts(8)
Isn't that risky for data integrity in the two 10 TB hard disks? What happens if the computer crashes or if the NVMe SSD fails? That NVMe SSD is not mirrored.
 
Usage pattern matters a bit. L2ARC is mostly read data (just like regular ARC), ZIL is write data. If the usage is mostly read of the same data (say streaming content) using some of the SSD as L2ARC may speed things up. I would still partition it so you have the OS on it then use some for L2ARC.

"Grossly simplified opinion follows"
ZIL can be thought of as roughly a "persistent write cache". You are writing to a zpool that has ZIL attached to it, the data gets written to ZIL first, then to the zpool. If the computer crashes mid write to the zpool, the zpool should be consistent, then when it next gets imported the remaining data from ZIL should get pushed to the zpool.
If you have a catastrophic failure of the SSD, then yes you probably would lose the data that was not yet pushed to the zpool. If you have mirrored ZIL, then yes you should be able to not lose anything, but you may suffer performance because "a write to mirrored devices is not complete until they are both done"

My opinion would be along the lines of if you can partition the SSD, chunk off say 128G for OS (should be more than big enough), then play with the rest as either L2ARC or ZIL based on the usage pattern.
 
There is a fundamental assumption that the ZIL is reliable, in the sense that an acknowledged write is safe. The caches used by non-enterprise class SSDs break that assumption. A sudden power loss could cause data loss and corruption of the pool. The only place I would put a separate ZIL is on an enterprise class SSD (which will have "power loss protection"). But a separate ZIL should have redundancy at least as good as the pool it serves, so a mirror'd pair of enterprise class SSDs. You will get NO BENEFIT from a separate ZIL unless you are doing synchronous writes, which are not all that common.

The L2ARC is a cache, and can be regenerated if lost, so it does not need redundancy. However the benefits of an L2ARC depend on your particular usage. You might not get any benefit...

You might consider adding a "special" VDEV, to contain the metadata and, optionally, small files for your large pool. Putting this in fast access storage is likely a win, BUT it really should have redundancy at least as good as the pool it serves, so, also a mirror.

If it were my system, I wold probably buy another NMVe SSD, identical to the first, and create mirrors for boot, zroot, swap, and "special" VDEV, plus leave space for testing an L2ARC stripe.
 
A single SSD is good for /usr/ports, /usr/src (if you don't have local modifications that need protection) and /usr/obj. /usr/local, too, if you don't have too many ports installed with changed options and the like.

L2arc cache is not a universally good idea:
 
A single SSD is good for /usr/ports, /usr/src (if you don't have local modifications that need protection) and /usr/obj. /usr/local, too, if you don't have too many ports installed with changed options and the like.

L2arc cache is not a universally good idea:
Will I be able to use a single SSD for both /usr/ports/ and /usr/src/?
 
Back
Top