ZFS FreeBSD 11, ZFS and lz4 compression woes.

[salival]

Hi guys,

I recently built a new server with 6xWD40 red drives in a raidz2 and thought I would try out lz4 compression.

From reading the online tutorials they all said to use:

zfs compression=lz4 storage

I copied all the data off my old pool (7TB worth) to this pool but my compression ratio is still 1.00, Have I done something wrong or am I missing something?

[root@gate ~]# zfs get all storage
NAME     PROPERTY              VALUE                  SOURCE
storage  type                  filesystem             -
storage  creation              Thu Dec  1 22:05 2016  -
storage  used                  6.17T                  -
storage  available             7.86T                  -
storage  referenced            6.17T                  -
storage  compressratio         1.00x                  -
storage  mounted               yes                    -
storage  quota                 none                   default
storage  reservation           none                   default
storage  recordsize            128K                   default
storage  mountpoint            /storage               default
storage  sharenfs              off                    default
storage  checksum              on                     default
storage  compression           lz4                    local
storage  atime                 on                     default
storage  devices               on                     default
storage  exec                  on                     default
storage  setuid                on                     default
storage  readonly              off                    default
storage  jailed                off                    default
storage  snapdir               hidden                 default
storage  aclmode               discard                default
storage  aclinherit            restricted             default
storage  canmount              on                     default
storage  xattr                 off                    temporary
storage  copies                1                      default
storage  version               5                      -
storage  utf8only              off                    -
storage  normalization         none                   -
storage  casesensitivity       sensitive              -
storage  vscan                 off                    default
storage  nbmand                off                    default
storage  sharesmb              off                    default
storage  refquota              none                   default
storage  refreservation        none                   default
storage  primarycache          all                    default
storage  secondarycache        all                    default
storage  usedbysnapshots       0                      -
storage  usedbydataset         6.17T                  -
storage  usedbychildren        19.4M                  -
storage  usedbyrefreservation  0                      -
storage  logbias               latency                default
storage  dedup                 off                    default
storage  mlslabel                                     -
storage  sync                  standard               default
storage  refcompressratio      1.00x                  -
storage  written               6.17T                  -
storage  logicalused           6.18T                  -
storage  logicalreferenced     6.18T                  -
storage  volmode               default                default
storage  filesystem_limit      none                   default
storage  snapshot_limit        none                   default
storage  filesystem_count      none                   default
storage  snapshot_count        none                   default
storage  redundant_metadata    all                    default

Cheers.

 

[salival]

Compression is likely to be inherited. When did you set compression, before or after copying?
Check again zfs get all | grep compressratio.
Maybe you need to set compression on each dataset.

Hey,

I set compression BEFORE I copied all the data across, I definitely made sure of this.

storage                                          compressratio         1.00x                  -
storage                                          refcompressratio      1.00x                  -

I may be confused by your "dataset" statement, but I don't use slices (if that's the term for it). My ZFS storage is mounted as /storage so I just use it like that, I don't have a /storage/downloads etc.

Maybe that could be my problem? I assumed that setting it on storage would mean it would use it.

Cheers.

 

[asteriskRoss]

Slightly confusingly, ZFS pools have a ZFS dataset at their root. ZFS pools may contain more than one dataset but they always contain at least one. So, salival, I believe your ZFS pool storage contains a single dataset also referred to as storage.

Getting back to your original question... I see on this page on ZFS in the FreeBSD Handbook under the compression heading:

The biggest advantage to LZ4 is the early abort feature. If LZ4 does not achieve at least 12.5% compression in the first part of the data, the block is written uncompressed to avoid wasting CPU cycles trying to compress data that is either already compressed or uncompressible.

However, I have not been able to find mention of the feature in any of the manual pages. What data were you copying? If the files were encrypted or already compressed (for example many common audio and video file formats) I can see it would be possible that the abort feature would be triggered. Perhaps you would be able to test with different files that are definitely compressible (for example text files such as the FreeBSD source code)?