Hi my old zpool has 8 2T disk and I'd like to replace all of them to 4T disk to expand my zpool.
old zpool.
First, I just pull out da0, then zpool show disk removed. It's expected
Then I added the new 4T disk, and run
Then new disk was added into zpool and start resilvering.
Then I realize I didn't create any partition on it, so it's using the entire disk, which means there will be no swap, no EFI partition.
After completed.
Then I offline da0.
And try to create the partition based on existing disk info.
Then checked gpart info.
Then try to resize it.
Then check the partition.
Looks like the new 4T disk is starting from 34 not 40, is this a problem?
And then I could not replace the disk
But the wired thing is, I can still make da0 online.
And at this time, if run gpart, then da0 has no partition.
But if mark da0 offline, then gpart show da0 has partition
So what's the right step to replace the disk in root-on-zfs zpool?
should I manually create the partition to make sure it's 4K aligned?
old zpool.
Code:
zroot ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
da0p4 ONLINE 0 0 0
da1p4 ONLINE 0 0 0
da2p4 ONLINE 0 0 0
da3p4 ONLINE 0 0 0
da4p4 ONLINE 0 0 0
da5p4 ONLINE 0 0 0
da6p4 ONLINE 0 0 0
da7p4 ONLINE 0 0 0
First, I just pull out da0, then zpool show disk removed. It's expected
Code:
NAME STATE READ WRITE CKSUM
zroot DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
da0p4 REMOVED 0 0 0
da1p4 ONLINE 0 0 0
da2p4 ONLINE 0 0 0
da3p4 ONLINE 0 0 0
da4p4 ONLINE 0 0 0
da5p4 ONLINE 0 0 0
da6p4 ONLINE 0 0 0
da7p4 ONLINE 0 0 0
Then I added the new 4T disk, and run
Code:
zpool replace zroot da0p4 da0
Then new disk was added into zpool and start resilvering.
Code:
zroot DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
replacing-0 DEGRADED 0 0 0
da0p4 REMOVED 0 0 0
da0 ONLINE 0 0 0 (resilvering)
da1p4 ONLINE 0 0 0
da2p4 ONLINE 0 0 0
da3p4 ONLINE 0 0 0
da4p4 ONLINE 0 0 0
da5p4 ONLINE 0 0 0
da6p4 ONLINE 0 0 0
da7p4 ONLINE 0 0 0
Then I realize I didn't create any partition on it, so it's using the entire disk, which means there will be no swap, no EFI partition.
After completed.
Code:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1p4 ONLINE 0 0 0
da2p4 ONLINE 0 0 0
da3p4 ONLINE 0 0 0
da4p4 ONLINE 0 0 0
da5p4 ONLINE 0 0 0
da6p4 ONLINE 0 0 0
da7p4 ONLINE 0 0 0
Then I offline da0.
Code:
zroot DEGRADED 0 0 0
raidz1-0 DEGRADED 0 0 0
da0 OFFLINE 0 0 0
da1p4 ONLINE 0 0 0
da2p4 ONLINE 0 0 0
da3p4 ONLINE 0 0 0
da4p4 ONLINE 0 0 0
da5p4 ONLINE 0 0 0
da6p4 ONLINE 0 0 0
da7p4 ONLINE 0 0 0
And try to create the partition based on existing disk info.
Code:
gpart backup da1 | gpart restore -F da0
Then checked gpart info.
Code:
=> 40 3907029088 da1 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 34 7814037101 da0 GPT (3.6T)
34 6 - free - (3.0K)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 3907008143 - free - (1.8T)
Then try to resize it.
Code:
# gpart resize -i 4 da0
da0p4 resized
Then check the partition.
Code:
=> 40 3907029088 da1 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 34 7814037101 da0 GPT (3.6T)
34 6 - free - (3.0K)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 7805113999 4 freebsd-zfs (3.6T)
Looks like the new 4T disk is starting from 34 not 40, is this a problem?
And then I could not replace the disk
Code:
# zpool replace -f zroot da0 da0p4
invalid vdev specification
the following errors must be manually repaired:
/dev/da0p4 is part of active pool 'zroot'
But the wired thing is, I can still make da0 online.
Code:
# zpool online -e zroot da0
# zpool status
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
da0 ONLINE 0 0 0
da1p4 ONLINE 0 0 0
da2p4 ONLINE 0 0 0
da3p4 ONLINE 0 0 0
da4p4 ONLINE 0 0 0
da5p4 ONLINE 0 0 0
da6p4 ONLINE 0 0 0
da7p4 ONLINE 0 0 0
And at this time, if run gpart, then da0 has no partition.
Code:
# gpart show da0
gpart: No such geom: da0.
gpart show
=> 40 3907029088 da6 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 40 3907029088 da7 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 40 3907029088 da4 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 40 3907029088 da5 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 40 3907029088 da3 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 40 3907029088 da2 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
=> 40 3907029088 da1 GPT (1.8T)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 3898105856 4 freebsd-zfs (1.8T)
3907028992 136 - free - (68K)
But if mark da0 offline, then gpart show da0 has partition
Code:
# zpool offline zroot da0
# gpart show da0
=> 34 7814037101 da0 GPT (3.6T) [CORRUPT]
34 6 - free - (3.0K)
40 532480 1 efi (260M)
532520 1024 2 freebsd-boot (512K)
533544 984 - free - (492K)
534528 8388608 3 freebsd-swap (4.0G)
8923136 7805113999 4 freebsd-zfs (3.6T)
So what's the right step to replace the disk in root-on-zfs zpool?
should I manually create the partition to make sure it's 4K aligned?