Replace a drive having a partition in a ZFS pool - how do I deal with the REST of the drive?

rwv37

rwv37

I have a 13.2-RELEASE-p4 machine with four hard drives. They are all partitioned just like this one:

Code: 
# gpart show ada0
=>         40  35156656048  ada0  GPT  (16T)
           40       532480     1  efi  (260M)
       532520         2008        - free -  (1.0M)
       534528     33554432     2  freebsd-swap  (16G)
     34088960  35122565120     3  freebsd-zfs  (16T)
  35156654080         2008        - free -  (1.0M)

The swap partitions and the ZFS partitions are encrypted (with GELI). The GELI views of the ZFS partitions are joined as a RAID-Z2 pool.

I would like to replace these drives with larger ones so as to expand my pool. I know (or at least know that I can once again figure out) how to do this from the ZFS point of view, one at a time and waiting ages for resilvering. But I'm concerned about my lack of knowledge regarding setting up the other parts of the drives; much or all of it was automagically done for by the FreeBSD installer. Specific things I am concerned about:
  • My machine can currently boot from any of the four drives. I would like to keep it that way. Is there anything that I need to do in order to accomplish this beyond just a gpart create of the EFI partition, with type "efi" and the appropriate byte offset and size? For example, maybe I have to somehow copy boot code there?
  • As I mentioned earlier, swap is encrypted on my current drives. I'd like it to be encrypted on my new drives, too. I have no idea how to do this. To be clear, I do know (or, again, at least can figure out once again) how to do it for the ZFS partitions, but I don't know how to do it for swap.
  • All four drives currently have the same GELI password. When I boot up, I only have to enter the password once, and it unlocks all four drives. I'd like it to keep behaving like this. Is there anything I need to do to accomplish this, beyond simply using the same password for the new drive?
Thanks in advance for any help.
 
OK, thanks. I think I've figured out (how to figure out) the encrypted swap thing - it's all apparently in a four-drive mirror, and there's a "gmirror" command that I was not previously consciously aware of. And I guess I don't really care that much about the same GELI password working for all the drives in one fell swoop - I guess it might just continue to work that way (assuming I give the new drive the same password), but if not, it's not so horrible. So what remains of my concern is the EFI stuff:

First, tingo, thanks for the suggestion. Unfortunately, the man page for gpart is, from my point of view, extremely cryptic. It's full of lots of jargon with which I am not familiar, and I feel like I'll have to read a book (or five) before even beginning to have some not-terribly-shaky grasp on what it's saying. So I'm hoping instead for a something that can be done without a particularly deep understanding.

Second, cracauer, thanks for the dd suggestion. That seems like it would probably fit the bill for the EFI part? I want to make sure I'm understanding some things first, though:

First, I imagine dding an entire 16 TB disk would take a long time (and four of them four times as long), and since I'd like to continue using the machine for most of that time, I'd rather just let resilvering do its thing at leisure. Given that, it seems like dding the entire disk would largely be time-expensive uselessness. So I'm thinking, can't I just dd the EFI partitions instead? So something like:
  • Shut down the machine.
  • Physically replace one drive (let's say /dev/ada3).
  • Boot from a thumb drive or whatever.
  • gpart to add the three partitions on /dev/ada3 (and specifically for the EFI partition, ada3p1, not worrying about anything beyond "-t efi" and the appropriate byte bounds).
  • dd if=/dev/ada0p1 of=/dev/ada3p1
  • Whatever gmirror stuff adds /dev/ada3p2 to the swap mirror
  • Reboot from /dev/ada3p1
  • Assuming that works, initialize geli for /dev/ada3p3
  • Replace the old /dev/ada3p3.eli with the new /dev/ada3p3.eli in the ZFS pool
  • Go about my life for the next several decades while resilvering happens, then repeat all this for the second drive, then the third, then the fourth.
Does that all seem right? Specifically, I am mostly concerned about this:

Am I correct when I imagine that the dd if=/dev/ada0p1 of=/dev/ada3p1 will take care of all the necessary bootability things, without me having to worry about actually comprehending how to "really" do it (i.e. with gpart bootcode and so forth?

Thanks again to you both.
 
Sounds reasonable. As far as EFI boot is concerned, dding the partition should be sufficient, if you created the partition in an identical manner in gpart.
 
rwv37 said:
First, tingo, thanks for the suggestion. Unfortunately, the man page for gpart is, from my point of view, extremely cryptic. It's full of lots of jargon with which I am not familiar, and I feel like I'll have to read a book (or five) before even beginning to have some not-terribly-shaky grasp on what it's saying. So I'm hoping instead for a something that can be done without a particularly deep understanding.
Click to expand...
Unfortunately, doing advanced stuff requires reading and understanding. It comes with the territory, you can say. Learning the jargon is necessary - every different field uses its own "tribal language".
If you have trouble understanding parts of the documentation, I find that constructing examples and asking questions about those examples sometimes help.
 
One of the really nice things about FreeBSD is the geom(8) architecture, which allows you to seamlessly deploy different storage classes in identical roles.

You can, for instance, use md(4) memory disks (just about) anywhere you can use a "real" disk. So you could make some small memory disks, configure them to mimic your real disk layout, and test your plans on the memory disks. [You can't boot hardware from memory disks, but that's about the only limitation.]

I would also think about your recovery strategy if you mis-step in the upgrade. If a larger ZFS pool mean you need to change your backup strategy, maybe make that change sooner, so you have better or more secure backups before you do anything risky. I once used my new high capacity backup media to hold a full copy of my ZFS pools while I re-laid everything.
 
You must log in or register to reply here.
Back
Top