zfs upgrade -V 5000 doesn't work

C

curlymo

If have a ZFS test pool running v28. I want to upgrade that pool to version 5000, but with the highest compatibility.

When I normally upgrade my pool to v5000, all feature flags are activated by ZFS automatically.
Code: 
[root@server /home/ssh]# zpool status test
  pool: test
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on software that does not support feature
        flags.
  scan: none requested
config:

        NAME           STATE     READ WRITE CKSUM
        test           ONLINE       0     0     0
          mirror-0     ONLINE       0     0     0
            gpt/test1  ONLINE       0     0     0
            gpt/test2  ONLINE       0     0     0

errors: No known data errors
[root@server /home/ssh]# zpool upgrade test
This system supports ZFS pool feature flags.

Successfully upgraded 'test' from version 28 to feature flags.
Enabled the following features on 'test':
  async_destroy
  empty_bpobj
  lz4_compress
  multi_vdev_crash_dump
  spacemap_histogram
  enabled_txg
  hole_birth
  extensible_dataset
  bookmarks

The problem is that the multi_vdev_crash_dump isn't supported in zfsonlinux and ZFS-OSX. So, I want to upgrade the pool without all feature activated by default.

The manual tells me that I should use the -V parameter:
Code: 
     zpool upgrade [-V version] -a | pool ...

         Enables all supported features on the given pool.  Once this is done,
         the pool will no longer be accessible on systems that do not support
         feature flags.  See zpool-features(7) for details on compatability
         with system sthat support feature flags, but do not support all fea-
         tures enabled on the pool.

         -a      Enables all supported features on all pools.

         -V version
                 Upgrade to the specified legacy version. If the -V flag is
                 specified, no features will be enabled on the pool.  This
                 option can only be used to increase version number up to the
                 last supported legacy version number.

However, when I recreate my test pool and upgrade using the -V parameter it still activates all feature flags.

Code: 
[root@server /home/ssh]# zpool status test
  pool: test
 state: ONLINE
status: The pool is formatted using a legacy on-disk format.  The pool can
        still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
        pool will no longer be accessible on software that does not support feature
        flags.
  scan: none requested
config:

        NAME           STATE     READ WRITE CKSUM
        test           ONLINE       0     0     0
          mirror-0     ONLINE       0     0     0
            gpt/test1  ONLINE       0     0     0
            gpt/test2  ONLINE       0     0     0

errors: No known data errors
[root@server /home/ssh]# zpool upgrade -V 5000 test
This system supports ZFS pool feature flags.

Successfully upgraded 'test' from version 28 to feature flags.
Enabled the following features on 'test':
  async_destroy
  empty_bpobj
  lz4_compress
  multi_vdev_crash_dump
  spacemap_histogram
  enabled_txg
  hole_birth
  extensible_dataset
  bookmarks

Is this a bug? I assumed the whole goal of the new feature flags is that we can just pick those features we want to use so we can balance between maximum compatibility between OpenZFS versions and bleeding edge features.

In my specific situation, I just want to use the lz4_compress feature which is supported in all OpenZFS implementations.
 
Since there's only one version; v5000, an upgrade will upgrade to that version, regardless of the features. To turn features on or off have a look at the zpool-features(7) man page.
 
The problem is that all feature are enabled by default, regardless of the command given. Therefor, (theoretically) a pool created / upgraded in FreeBSD 11 can't be imported anymore in FreeBSD 10 although both should support v5000. I haven't tested it, but the description of the feature flags seems clear. A pool can only be imported when all feature flags are also supported. Since, we can't start of with a "blank" v5000 (with all feature disabled), you can't import a pool into a previous OpenZFS version, once upgraded.

More specifically:
Code: 
     Since most features can be enabled independently of each other the
     on-disk format of the pool is specified by the set of all features marked
     as active on the pool.
This is exactly the problem. They can't be enabled independently of each other, because all are enabled/actived by default in an zfs upgrade.
 
I've never tried it but the man page states:
Code: 
To	enable a feature on a pool use the upgrade subcommand
     of	the zpool(8) command, or set the feature@feature_name property to
     enabled.

I would guess you can also disable features using this method.
 
I will try right away :)

The issue is that the -V parameter of zpool upgrade should upgrade with everything disabled. That's what at least the man pages say it does. However, that is not what's happening.
 
-V version
Upgrade to the specified legacy version. If the -V flag is
specified, no features will be enabled on the pool. This
option can only be used to increase version number up to the
last supported legacy version number.
Click to expand...

I believe the last 'legacy' version is 28. v5000 is the feature flags version so I suspect upgrading to 5000 is basically telling it that you want to enable the feature flags.

Personally I'm not sure if there's a way to disable specific feature flags during upgrade or after pool creation. (I think -o can be used during creation although I'm not certain). Would be interesting to find out from someone with more v5000 experience. I remember being concerned when I first learnt about feature flags that it seems to enable everything be default, making some features difficult if not impossible to get away from. This seems to go against one of its primary advantages for me - the ability to choose the features I want in order to make sure I don't enable something that isn't going to work on systems running an older or different OS.
 
I've tested some more with the upgraded pool.
Code: 
[root@server /home/ssh]# zpool get all test
NAME  PROPERTY                       VALUE                          SOURCE
[...]
test  feature@async_destroy          enabled                        local
test  feature@empty_bpobj            enabled                        local
test  feature@lz4_compress           enabled                        local
test  feature@multi_vdev_crash_dump  enabled                        local
test  feature@spacemap_histogram     active                         local
test  feature@enabled_txg            active                         local
test  feature@hole_birth             active                         local
test  feature@extensible_dataset     enabled                        local
test  feature@bookmarks              enabled                        local
[root@server /home/ssh]# zpool set feature@lz4_compress=disable test
cannot set property for 'test': property 'feature@lz4_compress' can only be set to 'enabled'

Destroyed the pool again but created it with all features disabled:
Code: 
[root@server /home/ssh]# zpool create -d -f test mirror gpt/test1 gpt/test2
[root@server /home/ssh]# zpool get all test
NAME  PROPERTY                       VALUE                          SOURCE
[...]
test  feature@async_destroy          disabled                       local
test  feature@empty_bpobj            disabled                       local
test  feature@lz4_compress           disabled                       local
test  feature@multi_vdev_crash_dump  disabled                       local
test  feature@spacemap_histogram     disabled                       local
test  feature@enabled_txg            disabled                       local
test  feature@hole_birth             disabled                       local
test  feature@extensible_dataset     disabled                       local
test  feature@bookmarks              disabled                       local
[root@server /home/ssh]# zpool set feature@lz4_compress=enabled test
[root@server /home/ssh]# zpool get all test
NAME  PROPERTY                       VALUE                          SOURCE
[...]
test  feature@async_destroy          disabled                       local
test  feature@empty_bpobj            disabled                       local
test  feature@lz4_compress           enabled                        local
test  feature@multi_vdev_crash_dump  disabled                       local
test  feature@spacemap_histogram     disabled                       local
test  feature@enabled_txg            disabled                       local
test  feature@hole_birth             disabled                       local
test  feature@extensible_dataset     disabled                       local
test  feature@bookmarks              disabled                       local
[root@server /home/ssh]# zpool set feature@lz4_compress=disable test
cannot set property for 'test': property 'feature@lz4_compress' can only be set to 'enabled'

So, it is possible to activate individual features in v5000, but you can't disable them. I understand you can't disable it. You never could downgrade in zfs, and this is essentially the same. However, manually enabling specific features was what i believed the whole purpose of v5000, that apparently doesn't work when upgrading. And of which i believe is a bug in zfs.
 
Yes, I think this should be raised as a bug. The upgrade process should have the same ability as zpool create to start with all features disabled. Otherwise it's impossible to move an existing pool to the new version if you need to maintain compatibility with another OS that doesn't support all the same features.
 
I already filed a bug report @ freebsd. I'm not sure how long it takes to arrive, but i wanted to make sure i'm wasn't doing anything weird in the first place.
 
I've got an answer from the OpenZFS developers mailing list. There is a way to upgrade the pool with just one feature enabled. Not very logical in my opinion:
Code: 
[root@server /home/ssh]# zpool create -f -o version=28 test mirror gpt/test1 gpt/test2
[root@server /home/ssh]# zpool get all test
NAME  PROPERTY                       VALUE                          SOURCE
[...]
test  feature@async_destroy          disabled                       local
test  feature@empty_bpobj            disabled                       local
test  feature@lz4_compress           disabled                       local
test  feature@multi_vdev_crash_dump  disabled                       local
test  feature@spacemap_histogram     disabled                       local
test  feature@enabled_txg            disabled                       local
test  feature@hole_birth             disabled                       local
test  feature@extensible_dataset     disabled                       local
test  feature@bookmarks              disabled                       local
[root@server /home/ssh]# zpool set feature@lz4_compress=enabled test
[root@server /home/ssh]# zpool get all test
NAME  PROPERTY                       VALUE                          SOURCE
[...]
test  feature@async_destroy          disabled                       local
test  feature@empty_bpobj            disabled                       local
test  feature@lz4_compress           enabled                        local
test  feature@multi_vdev_crash_dump  disabled                       local
test  feature@spacemap_histogram     disabled                       local
test  feature@enabled_txg            disabled                       local
test  feature@hole_birth             disabled                       local
test  feature@extensible_dataset     disabled                       local
test  feature@bookmarks              disabled                       local
[root@server /home/ssh]# zpool status test
  pool: test
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
        still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
        the pool may no longer be accessible by software that does not support
        the features. See zpool-features(7) for details.
  scan: none requested
config:

        NAME           STATE     READ WRITE CKSUM
        test           ONLINE       0     0     0
          mirror-0     ONLINE       0     0     0
            gpt/test1  ONLINE       0     0     0
            gpt/test2  ONLINE       0     0     0

errors: No known data errors
 
You must log in or register to reply here.
Back
Top