HOWTO: Add full disk labels to an existing zfs system

[peetaur]

HOWTO: Add full disk labels to an existing zfs system

To create labels:
for a list of commands type:

# glabel
​

Why create labels?

When you reboot, sometimes the disks move around and change numbers. Maybe they are reliable with some specific hardware or driver, but don't count on it.

When a data disk from my pool would switch places with another data disk from the same pool, zfs would automatically handle it. But when a hotspare or something else switched places with a data disk, it would have an error seen in zpool status.

<some big number>       UNAVAIL 0 0 0 was /dev/da#

Option #1 (live system conversion, possible corruption if you make an error or a disk has a URE; do a backup!)

One disk at a time, replace a disk, label it, add it back, and resilver the newly labeled device into the pool where the unlabeled one was.

Practice first on a virtual machine, and then on the live system.

*** Create a backup ***
If something goes wrong, raid will not save you.

Scrub because you don't want to remove a good disk when another is bad and you don't know.

# zpool scrub share

Wait...

At this point, decide if you want to go to single user mode or if you want to do it live. For a production system, I would recommend single user mode or someone can put data on there, and then you could destroy your pool, restore your backup, and their data is gone.

# glabel label spare1 <your spare device>

For all disks you can list in /dev/ad* {
Make the disk light blink, and decide what to name it based on position, then hit CTRL+c)
# dd if=/dev/ada4 of=/dev/null

Replace with your spare
# zpool replace share ada4 label/spare1

Set the label
# glabel label share1d1 ada4

Replace the spare with your original disk now with a label.
# zpool replace share label/spare1 label/share1d1

Wait for resilver...
}
​

Option #2 (experimental, no idea if you lose data this way, bad idea unless you are not too worried about failing and restoring from backup):

# zpool export poolname
# glabel label poolname1d1 ad8
# glabel label poolname1d1 ad10
...
# zpool import poolname

In my test virtual machine, it automatically found the pool and didn't have any errors. I have no idea if it is safe though. I don't have any data on my test disks.
​

Option #3 (restore a backup):

scrub
back up the pool
destroy the pool
label
recreate the pool with labels
restore the pool
​

Here is an example of the above option #1, done in a FreeBSD virtual machine in VirtualBox, using a script.

# zpool status big

pool: big
    state: ONLINE
    scan: none requested
    config:

            NAME        STATE     READ WRITE CKSUM
            big         ONLINE       0     0     0
            raidz1-0    ONLINE       0     0     0
                ad8     ONLINE       0     0     0
                ad10    ONLINE       0     0     0
                ad12    ONLINE       0     0     0
                ad14    ONLINE       0     0     0

    errors: No known data errors

# vim relabeltest.bash

spare=ad16
    tolabel="ad8 ad10 ad12 ad14"
    IFS=' '
    labels=(big1 big2 big3 big4)
    i=0
    IFS=' '
    for disk in $tolabel; do
        label=${labels[$i]}
        echo "=============================="
        echo "Renaming disk $disk to $label"
        
        zpool replace big $disk ad16
        glabel label $label $disk
        zpool replace big ad16 label/$label

        zpool status big
        
        sleep 2
        while zpool status big | grep "DEGRADED" >/dev/null 2>&1; do
            echo Still waiting...
            sleep 1
        done

        echo "=============================="
        let i++
    done

# chmod u+x relabeltest.bash
# ./relabeltest.bash
...
# zpool status

pool: big
    state: ONLINE
    scan: resilvered 43.5K in 0h0m with 0 errors on Tue Dec  6 10:43:35 2011
    config:

            NAME            STATE     READ WRITE CKSUM
            big             ONLINE       0     0     0
            raidz1-0        ONLINE       0     0     0
                label/big1  ONLINE       0     0     0
                label/big2  ONLINE       0     0     0
                label/big3  ONLINE       0     0     0
                label/big4  ONLINE       0     0     0

    errors: No known data errors

DISCLAIMER: I would love to hear about all the wonderful experiences the above information brings you, but won't take any blame. And this is a work in progress.