ZFS ZFS compression recommendations for encrypted files

Kaminar

Kaminar

What compression type do you recommend for a ZFS dataset that will contain files encrypted with PEFS or another encryption method?
Does compression provide any practical benefit in this case, or is it better to disable it entirely?
 
you will have random data which is basically not compressible. I would suggest no compression or lz4 which is very fast in compression and decompression of data as well as detecting whether data is compressible.
 
If you have some original files, why not manually try the different compression algorithms? You may find one that does the best (you define what best means) or you may find, they all do nothing.
 
Below is a simple test of ZFS compression methods. All tests were performed in VirtualBox, always starting from the same clean snapshot, each in a separate ZFS dataset. For file encryption, PEFS was used.

Sample set contents

File typeNumber of filesTotal size (bytes)
video
14​
3,150,528,644​
PDF
51​
166,362,504​
Total
65​
3,316,891,148​

Results (columns show ZFS properties compression and used for the tested dataset)

compressionused (bytes)
off
3921088512​
zle
3945467904​
lz4
3951259648​
 
Its not worth compression compressed files.
But CPU overhead for compression compressed files is little.
So i put everything compression "lze" as default.
Note the opposite , i dont compress log files in /var/log by syslog. But the dataset has compression "lze".
 
Kaminar said:
Below is a simple test of ZFS compression methods. All tests were performed in VirtualBox,
Click to expand...
I wouldn't rate the results too high, since VirtualBox already compresses its virtual drives, and you don't really get much compression on already compressed data.
In contrary, depending on what kind of data the compression is used on - if not bitmap any picture, if not .wav any most audio and above all all video files are already in a compressed state, which you cannot gain much more (lossless) compression. In worst case a combination of compression algorithms - compress already compressed data that already is compressed... - can produce even larger files than without any compression at all.
If you want to get real comparable results on compression I recommend to use real ZFS pools on real drives.
 
Flac is lossless compression. Meaning audio is bit-exact.
With mp3 you can reduce file size but you will always remove a bit of quality.
When i can i always try to use flac for storing audio.
Putting simple uncompressed .wav or compressed .flac .mp3 on zfs with "lze-compression" will have no effect.
 
You must log in or register to reply here.
Back
Top