ZFS Lost mirror while detach

urosgruber

urosgruber

Looks like I did a mistake with detaching failed drive in mirror first instead of attaching new drive and after resilver detaching failed one. I didn't check that doing this way mirror actually became single drive, so after I run zpool add with this new drive I created even more idiotic situation. No before I make my pool unreadable I would like to ask if there is a way out from this.

My current output from zpool status

Code: 
    NAME           STATE     READ WRITE CKSUM
    sys            ONLINE       0     0     0
     gpt/disk1    ONLINE       0     0     0
     mirror-1     ONLINE       0     0     0
       gpt/disk3  ONLINE       0     0     0
       gpt/disk4  ONLINE       0     0     0
     mirror-2     ONLINE       0     0     0
       da3p2      ONLINE       0     0     0
       da4p2      ONLINE       0     0     0
     gpt/disk2    ONLINE       0     0     0

But right after drive failed my setup was

Code: 
    NAME                     STATE     READ WRITE CKSUM
    sys                      DEGRADED     0     0     0
     mirror-0               DEGRADED     0     0     0
       gpt/disk1            ONLINE       0     0     0
       6302187020726097327  UNAVAIL      0     0     0  was /dev/gpt/disk2
     mirror-1               ONLINE       0     0     0
       gpt/disk3            ONLINE       0     0     0
       gpt/disk4            ONLINE       0     0     0
     mirror-2               ONLINE       0     0     0
       da3p2                ONLINE       0     0     0
       da4p2                ONLINE       0     0     0

Is there a way to get back to this setup?
 
But first you will need to destroy the vdev you made with disk2. Make sure you haven't written any data there first.
Click to expand...

Unfortunately it's not that simple. You can't remove a top level vdev from a pool. The only easy way out of this situation is to add 2 more disks and turn both of the single disk vdevs into mirrors. (of course that relies on having 2 spare disk ports and buying the disks)

There's nothing really wrong with detaching a failed disk, then attaching a new one. Doesn't seem any more dangerous than physically replacing the failed disk while it's still a member of the pool, then running a zpool replace. The big mistake of course was adding the new disk to the pool (thus creating a new vdev) rather than attaching it to the one already there, converting it from single disk to mirror.
 
urosgruber,

if what´s in "sys" fits into two disks, I´d rebuild like this. First boot with an install media in Live CD mode and then:
Code: 
# sh
# mkdir /mnt/sys
# zpool import -d /dev/gpt -o cachefile=/tmp/zpool.cache -o altroot=/mnt/sys sys
# zpool detach sys gpt/disk3
# zpool detach sys da3p2
# mkdir /mnt/sys2
# zpool create -o cachefile=/tmp/zpool.cache -o -o altroot=/mnt/sys2 sys2 gpt/disk3 da3p2
# zfs snapshot -r sys@now
# zfs send -R sys@now | zfs recv -du sys2
BOOTFS=$(zpool get -H -o value bootfs sys)
# zpool destroy sys
# zpool export sys2
# zpool import -d /dev/gpt -o cachefile=/tmp/zpool.cache -o altroot=/mnt/sys sys2 sys
# zpool set bootfs=${BOOTFS} sys
# zpool attach -f sys gpt/disk3 gpt/disk4
# zpool attach -f sys da3p2 da4p2
# zpool add -f sys mirror gpt/disk1 gpt/disk2  <-- Here´s where you did wrong, forgetting the "mirror" part

And that´s all I can guess by what you´ve given so far. There´s more to do after this though:
# cp /tmp/zpool.cache /mnt/sys/foo/bar/boot/zfs/

This needs to be corrected for your setup!

After a reboot you should be back on track, hopefully:) But as I initially stated, this procedure depends on fitting everything you now have into just two disks so check that first!
 
You must log in or register to reply here.
Back
Top