Problem replacing drive in ZFS array

G

garrettmoore

I have an 8-drive ZFS array consisting of WD15EADS drives. One of my disks has started to fail, so I got a replacement disk. I have replaced a disk before by:

zpool offline tank /dev/da5
shutting down, swapping from old disk to new disk
booting
zpool replace tank /dev/da5

This worked fine.

This time the failing disk was da3, and I tried the same thing:
zpool offline tank /dev/da3
zpool status showed da3 offline.
shut down, swapped old disk to new disk.

When I booted again, I got:
Code: 
        NAME        STATE     READ WRITE CKSUM
        tank        UNAVAIL      0     0     0  insufficient replicas
          raidz1    UNAVAIL      0     0     0  corrupted data
            da0     ONLINE       0     0     0
            da1     ONLINE       0     0     0
            da2     ONLINE       0     0     0
            da3     ONLINE       0     0     0
            da4     ONLINE       0     0     0
            da5     ONLINE       0     0     0
            da6     ONLINE       0     0     0
            da7     ONLINE       0     0     0

I switched back to the old disk and booted again and then I could access my data again, and da3 still showed as offline. I tried 'zpool online tank /dev/da3' and after a few seconds resilvering completed and all 8 drives are back online again, but with the 'dying' disk as da3 still.

I tried shutting down WITHOUT first offlining /dev/da3, and swapping the disks, and when I booted I again got 'insufficient replicas'.

Why am I getting this error, and how come it worked ok the last time I replaced a disk?

Thanks.
 
Turns out to have been a problem with the disk controller re-ordering the disks when I swapped a disk.

Instead of shutting down to switch disks, I hotswapped them, using camcontrol.

# zpool offline tank da3
# camcontrol stop da3
swap drive
# camcontrol rescan all
"da3 lost device, removing device entry"
# camcontrol rescan all
"da3 at mpt0 ..."
# zpool replace tank da3

Done. (actually I have some other issues with the replacement disk but those are unrelated to this thread)
 
You must log in or register to reply here.
Back
Top