Solved Should I turn compression on for the file system store VM images?

  • Thread starter Thread starter Deleted member 63539
  • Start date Start date
D

Deleted member 63539

Guest
I use the file system zroot/vm to store VM images, both vm-bhyve's .img images and VirtualBox's .vdi images. The .vdi images are dynamic allocated. Should I turn compression on for the file system store VM images? Currently I have compression turned off.

Given I already have images on zroot/vm, if I turned on compression, does it compress the already there data or only newly created data? And how to force it to compress the already there data too? Thanks.

p/s: which algorithm should be chosen? Is the FreeBSD's default lz4 algorithm fine?
 
sysctl said:
I use the file system zroot/vm to store VM images, both vm-bhyve's .img images and VirtualBox's .vdi images. The .vdi images are dynamic allocated. Should I turn compression on for the file system store VM images? [...]
Click to expand...
Yes. Usually compression makes sense, especially on a (slow) rotating disk device, because the decompression is so fast that it will speed up read-mostly filesystems, i.e. the time saved to read/write less data to the disk is greater than the time it takes to de/compress. Even for SSDs (see below mdcomp_disable). Thus the default setting is compression=on for the root dataset, inherit compression for it's children, and only set it off for selected datasets.
Given I already have images on zroot/vm, if I turned on compression, does it compress the already there data or only newly created data? And how to force it to compress the already there data too? Thanks.
Click to expand...
No. ZFS is a copy-on-write filesystem. You can compress data that has already been written uncompressed by copying to another dataset where compression is active. E.g. zfs create -o compression=on zroot/vm-x, cpdup -dI /vm /vm-x, then zfs destroy zroot/vm & zfs rename zroot/vm-x zroot/vm.
p/s: which algorithm should be chosen? Is the FreeBSD's default lz4 algorithm fine?
Click to expand...
Yes, it is very fast & one of the main advantages is that it detects uncompressable data very quickly, aborts and does not waste resources in trying to compress it. Thus there is rarely a good reason to set compression off.
 
Hi. What about zstd? If I want to use zstd, do I need to use the OpenZFS from ports? Thanks.
 
According to the mailing list OpenZFS with ZSTD is now in CURRENT so will be in FreeBSD 13. I don't know if it's in the existing port but it's not in any existing release.

Personally I just enable lz4 on pretty much everything.
 
usdmatt said:
According to the mailing list OpenZFS with ZSTD is now in CURRENT so will be in FreeBSD 13. I don't know if it's in the existing port but it's not in any existing release.

Personally I just enable lz4 on pretty much everything.
Click to expand...
I'm on 11.4-p2 so I will stick with lz4. Thanks.
 
Why is the adoption of this first attempt by facebook to publish open/free code so uncritically passed around?

In my tests with the largest compressed package I had, comparing xz (utilizing the -T option for multithreading) zstd was deficient in size even in its slowest setting, slightly faster but unless we are talking about huge data sets a few milieseconds here and there don't hurt anyone. And ram use was also significantly different. At the end was marginally comparing an old trusty tool meant for compression that still beats zstd handsdown, with something facebook produced the day before yesterday. I urge anyone to do tests on their own and decide for themselves if it is worthwhile.
 
fungalnet said:
Why is the adoption of this first attempt by facebook to publish open/free code so uncritically passed around?
Click to expand...
Who says it's passed around "uncritically"? Have you looked at the long list of issues? Those issues wouldn't be reported if people were "uncritical" of it.
 
fungalnet said:
I urge anyone to do tests on their own and decide for themselves if it is worthwhile.
Click to expand...
That is always true for tuning storage systems, in particular for compression: whether to use it, what compression algorithm to use if they are selectable. People who do storage for a living tend to spend months or years selecting and tuning compression algorithms.

But: that doesn't mean that ZFS now having some form of compression enabled, with a sensible default, is a bad thing. Your ranting about Facebook is uncalled for. And the statement that this is the "first attempt by facebook to publish open/free code" is nonsense.
 
You must log in or register to reply here.
Back
Top