HOWTO: Add full disk labels to an existing zfs system

peetaur · Dec 6, 2011

HOWTO: Add full disk labels to an existing zfs system

To create labels:
for a list of commands type:

# glabel

Why create labels?

When you reboot, sometimes the disks move around and change numbers. Maybe they are reliable with some specific hardware or driver, but don't count on it.

When a data disk from my pool would switch places with another data disk from the same pool, zfs would automatically handle it. But when a hotspare or something else switched places with a data disk, it would have an error seen in zpool status.

Code:

<some big number>       UNAVAIL 0 0 0 was /dev/da#

Option #1 (live system conversion, possible corruption if you make an error or a disk has a URE; do a backup!)

One disk at a time, replace a disk, label it, add it back, and resilver the newly labeled device into the pool where the unlabeled one was.

Practice first on a virtual machine, and then on the live system.

*** Create a backup ***
If something goes wrong, raid will not save you.

Scrub because you don't want to remove a good disk when another is bad and you don't know.

# zpool scrub share

Wait...

At this point, decide if you want to go to single user mode or if you want to do it live. For a production system, I would recommend single user mode or someone can put data on there, and then you could destroy your pool, restore your backup, and their data is gone.

# glabel label spare1 <your spare device>

For all disks you can list in /dev/ad* {
Make the disk light blink, and decide what to name it based on position, then hit CTRL+c)
# dd if=/dev/ada4 of=/dev/null

Replace with your spare
# zpool replace share ada4 label/spare1

Set the label
# glabel label share1d1 ada4

Replace the spare with your original disk now with a label.
# zpool replace share label/spare1 label/share1d1

Wait for resilver...
}

Option #2 (experimental, no idea if you lose data this way, bad idea unless you are not too worried about failing and restoring from backup):

# zpool export poolname
# glabel label poolname1d1 ad8
# glabel label poolname1d1 ad10
...
# zpool import poolname

In my test virtual machine, it automatically found the pool and didn't have any errors. I have no idea if it is safe though. I don't have any data on my test disks.

Option #3 (restore a backup):

scrub
back up the pool
destroy the pool
label
recreate the pool with labels
restore the pool

Here is an example of the above option #1, done in a FreeBSD virtual machine in VirtualBox, using a script.

# zpool status big

Code:

pool: big
    state: ONLINE
    scan: none requested
    config:

            NAME        STATE     READ WRITE CKSUM
            big         ONLINE       0     0     0
            raidz1-0    ONLINE       0     0     0
                ad8     ONLINE       0     0     0
                ad10    ONLINE       0     0     0
                ad12    ONLINE       0     0     0
                ad14    ONLINE       0     0     0

    errors: No known data errors

# vim relabeltest.bash

Code:

spare=ad16
    tolabel="ad8 ad10 ad12 ad14"
    IFS=' '
    labels=(big1 big2 big3 big4)
    i=0
    IFS=' '
    for disk in $tolabel; do
        label=${labels[$i]}
        echo "=============================="
        echo "Renaming disk $disk to $label"
        
        zpool replace big $disk ad16
        glabel label $label $disk
        zpool replace big ad16 label/$label

        zpool status big
        
        sleep 2
        while zpool status big | grep "DEGRADED" >/dev/null 2>&1; do
            echo Still waiting...
            sleep 1
        done

        echo "=============================="
        let i++
    done

# chmod u+x relabeltest.bash
# ./relabeltest.bash
...
# zpool status

Code:

pool: big
    state: ONLINE
    scan: resilvered 43.5K in 0h0m with 0 errors on Tue Dec  6 10:43:35 2011
    config:

            NAME            STATE     READ WRITE CKSUM
            big             ONLINE       0     0     0
            raidz1-0        ONLINE       0     0     0
                label/big1  ONLINE       0     0     0
                label/big2  ONLINE       0     0     0
                label/big3  ONLINE       0     0     0
                label/big4  ONLINE       0     0     0

    errors: No known data errors

DISCLAIMER: I would love to hear about all the wonderful experiences the above information brings you, but won't take any blame. And this is a work in progress.

HOWTO: Add full disk labels to an existing zfs system

peetaur