Replacing all disks in a RAID10-syle pool

I currently have four 4TB disks and an 80GB SSD which were configured in RAID10 (striped and mirrored) with the SSD as a cache device. This is the only pool on the server and contains root (it was configured with the FreeBSD 11.0-RELEASE installer with mostly default options). The pool name is data.

I'm running short on storage lately and so I'm looking at upgrading the whole set to 8TB disks. But before making my purchase I want to make sure I'm understanding this correctly. This is my first time replacing disks in a pool with higher capacity disks.

From what I can see in the Handbook, I need to do the following:
1.run zpool replace data ada1p3 ada2p3 (replacing ada1p3 with old disk and ada2p3 with new)
2. wait until resilvering is complete
3. run zpool online -e
4. repeat 1-3 for each disk until new disks are all in place

At this point everything should be good and I would have ~16TB of usable space instead of my original ~8TB. Is this correct? Or am I missing steps here?

Now, two questions that I'm having a hard time learning from the handbook.
1. The handbook seems to assume that the new disk and old disk will be active at the same time, but I don't have enough SATA slots for that so I need to pull the old disk before adding the new one. Should i just unplug the the old disk and plop the new one into place? Or is there some kind of prep that needs to be done on the disk first?

2. The handbook gives this example as something that has to be done after an operation like this: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1. However, I don't understand what needs to be done exactly. Do I just randomly select a disk in place of ada1? Do I give a pool name? Or what?

My zpool: https://paste.fedoraproject.org/513742/14828585/

This is a potentially dangerous undertaking and being my first time doing something like this, I need to be absolutely sure it's done right the first time. I appreciate any input or clarification you can provide.
 
I've only done this with a RAID-Z set but it should work the same way. You indeed need to replace each disk, one by one. Once everything is replaced you may need to enable autoexpand property on the pool. It took some time to replace 4 x 2 TB with 4 x 3 TB but it was easy to do.
 
You might also have a look at the zpool split command. This way you can replace half of the disks at once and keep the other half as backup during the process.
Either enable the autoexpand property or just use zpool online -e on the new providers.
 
You might also have a look at the zpool split command. This way you can replace half of the disks at once and keep the other half as backup during the process.
Either enable the autoexpand property or just use zpool online -e on the new providers.

Looking at Oracle's ZFS documentation Here, the split command can't be used on a root pool. Since my root does exist on this zpool, wouldn't that mean I can't use it in my case? Or has ZFS in FreeBSD been changed in such a way as to allow this to be done?
 
Back
Top