Solved zfs remove geli encryption

byrnejb

byrnejb

I have a zfs pool which was originally setup as encrypted. Due to some hardware errors I replaced an encrypted member with its unencrypted counterpart. In other words: zpool replace zroot /dev/ada0p4.eli /dev/ada0p4. A second drive was also replaced and so I now have this:
Code: 
[root@vhost03 ~ (master)]# zpool status zroot
  pool: zroot
 state: ONLINE
status: Some supported and requested features are not enabled on the pool.
    The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
    the pool may no longer be accessible by software that does not support
    the features. See zpool-features(7) for details.
  scan: resilvered 553G in 16:58:51 with 0 errors on Fri May 26 03:28:29 2023
config:

    NAME            STATE     READ WRITE CKSUM
    zroot           ONLINE       0     0     0
      raidz2-0      ONLINE       0     0     0
        ada1p4.eli  ONLINE       0     0     0
        ada0p4      ONLINE       0     0     0
        ada2p4.eli  ONLINE       0     0     0
        ada3p4      ONLINE       0     0     0

errors: No known data errors

So, my thinking is that as this system no longer hosts what needed to be encrypted I would remove the encryption from the other drives as well. However, I am missing something when I try to do this and get an error instead:
Code: 
[root@vhost03 ~ (master)]# zpool offline zroot /dev/ada2p4.eli

[root@vhost03 ~ (master)]# zpool status zroot
  pool: zroot
 state: DEGRADED
status: One or more devices has been taken offline by the administrator.
    Sufficient replicas exist for the pool to continue functioning in a
    degraded state.
action: Online the device using 'zpool online' or replace the device with
    'zpool replace'.
  scan: resilvered 464K in 00:00:02 with 0 errors on Fri May 26 08:46:11 2023
config:

    NAME            STATE     READ WRITE CKSUM
    zroot           DEGRADED     0     0     0
      raidz2-0      DEGRADED     0     0     0
        ada1p4.eli  ONLINE       0     0     0
        ada0p4      ONLINE       0     0     0
        ada2p4.eli  OFFLINE      0     0     0
        ada3p4      ONLINE       0     0     0

errors: No known data errors

[root@vhost03 ~ (master)]# zpool replace zroot /dev/ada2p4.eli /dev/ada2p4
cannot replace /dev/ada2p4.eli with /dev/ada2p4: no such pool or dataset

[root@vhost03 ~ (master)]# ls -l /dev/ada2*
crw-r-----  1 root  operator  0x99 Apr 20 11:01 /dev/ada2
crw-r-----  1 root  operator  0xb5 Apr 20 11:01 /dev/ada2p1
crw-r-----  1 root  operator  0xb7 Apr 20 11:01 /dev/ada2p2
crw-r-----  1 root  operator  0xb9 Apr 20 11:01 /dev/ada2p3
crw-r-----  1 root  operator  0xd8 Apr 20 11:01 /dev/ada2p3.eli
crw-r-----  1 root  operator  0xbb Apr 20 11:01 /dev/ada2p4
crw-r-----  1 root  operator  0xe5 Apr 20 11:01 /dev/ada2p4.eli

What am I missing?
 
I did not need to geli kill ada0p4 nor ada3p4. I only needed to zpool offline and then zpool replace. Why is geli kill needed now?
 
byrnejb said:
I did not need to geli kill ada0p4 nor ada3p4. I only needed to zpool offline and then zpool replace. Why is geli kill needed now?
Click to expand...

If they had experienced failures, they were likely completely closed out by ZFS. Assuming you haven't changed geli_autodetach — documented in rc.conf(5); it defaults to "YES" — this means the .eli device will go away (geli detach completes) once ZFS closes it, so the geli kill won't be required.

I'm a little surprised offline didn't fully close it out, but maybe it keeps it open if there have been no device errors to be able to online it again if requested?
 
byrnejb said:
I did not need to geli kill ada0p4 nor ada3p4. I only needed to zpool offline and then zpool replace. Why is geli kill needed now?
Click to expand...

Perhaps you geli(8) detached ada0p4 and ada3p4 first before zpool-replace(8)?

As long as a geli(8) provider is attached it's not possible to replace it from a pool with the same partition the geli(8) provider is created from. It needs to be detached first.

I've used geli kill (and zpool replace) because it's less input than zpool offline, geli detach, zpool replace.
 
Thank you. The two previous drives had been removed and faulted before replacing them in the pool. So that must have invisibly taken care of the geli attachments. ada2p4 is re-silvering now..
 
You must log in or register to reply here.
Back
Top