ZFS ZFS Compress over GELI, is it recommendable?

freezr

freezr

Hi folks,

I wonder if I can enable lz4 compression on a mirror over geli.
I am concerned that I am going to have quite CPU overhead and therefore poor performance.

What do the expert say?

Thanks ?
 
The lz4(1) manual page says that:

lz4 offers compression speeds > 500 MB/s per core, linearly scalable with multi-core CPUs. It features an extremely fast decoder, offering speed in multiple GB/s per core, typically reaching RAM speed limit on multi-core systems.​

Clearly, the make and model of the core matters, so you may need to investigate further.

If your disks write faster than 500 MB/s, it may be an issue (lucky you).

I'm not sure of the overhead of GELI(8). Maybe AESNI(4) helps? But you are paying that cost anyway, what ever it is.
 
freezr said:
I wonder if I can enable lz4 compression on a mirror over geli.
Click to expand...
Also keep in mind that when enabling ZFS compression, a file is only compressed "byte-by-byte" when deemed compressable by ZFS. For example, your movies, in all likelyhood, will be deemed non-compressable by ZFS. As a practical matter, non-compressable files incur zero CPU processing overhead.
 
Erichans said:
Also keep in mind that when enabling ZFS compression, a file is only compressed "byte-by-byte" when deemed compressable by ZFS. For example, your movies, in all likelyhood, will be deemed non-compressable by ZFS. As a practical matter, non-compressable files incur zero CPU processing overhead.
Click to expand...

Well, it takes some CPU time to determine that a block is not compressible.

But it really should be no problem, especially if the storage is spinning rust.
 
I agree (as a technical matter, absolutely), that's why I wrote "As a practical matter". Hmm, on rethinking: "incur no CPU overhead" would probably have been a better phrasing.
 
My disks are SSD and my system is a laptop, is compression not recommeded for SSD?

I have ZFS on an arm board with rust disks, but is not FreeBSD, and I believe is not quite powerful to handle compression, but perhaps I am wrong, performance aren't important for this sistem.
 
You were correct to ask about lz4 in the context of GELI.

lz4 is not usually a problem with spinning disks because the disks can't write fast enough for the lz4 to slow them down.

However, the very latest NVMe SSDs on the very latest PCIe bus variants can, almost certainly, write a lot (ballpark 10 times) faster than "ordinary" CPUs can compress data for writing with lz4.

Note that GELI will slow down your disk access, potentially quite a lot, which is why it's essential to thoroughly investigate the aesni(4) module. This post is 10 years old, but might be a good place to start.

Once you know how much throughput the GELI provider is costing, you can consider the cost of lz4 in the ZFS layer (because the lz4 compression will only have to keep up with the GELI provider)..

So it's a complex ecosystem, impacted by CPU, AES-NI acceleration, PCIe bus, M.2, and SSD hardware; plus the GELI and lz4 software layers.

There is no simple answer if you have very fast SSDs.

Bottom line: if you have installed and configured GELI, and are not sure about lz4, it's not hard to enable lz4 selectively and run a disk I/O benchmark.

Do remember the old axiom that "90% of applications spend 90% of their time reading (or waiting to read)", and lz4 is really fast at decompressing. So, although writing will be slower, it's rarely the bottleneck.
 
Last edited:
Note that ARC is compressed by default. Your reads and writes are already being compressed if you haven't turned that off.

One thing I don't know is whether ARC can cache compressed blocks without recompression. If it can then dataset compression actually removes the overhead of compressing each time data comes back into ARC.
 
freezr said:
Hi folks,

I wonder if I can enable lz4 compression on a mirror over geli.
I am concerned that I am going to have quite CPU overhead and therefore poor performance.

What do the expert say?

Thanks ?
Click to expand...
You can.

That is also the default for FreeBSD install when You enable GELI in the installer in Auto (ZFS) mode.

Code: 
% zfs get compression zroot
NAME   PROPERTY     VALUE           SOURCE
zroot  compression  lz4             local
 
vermaden said:
You can.

That is also the default for FreeBSD install when You enable GELI in the installer in Auto (ZFS) mode.

Code: 
% zfs get compression zroot
NAME   PROPERTY     VALUE           SOURCE
zroot  compression  lz4             local
Click to expand...

So basically it is already enabled, isn't it?
 
You must log in or register to reply here.
Back
Top