ZFS Did my new install just waste half the space on my unequal sized zfs mirror root drives?

ThePythonicCow

ThePythonicCow

I just completed my first ever FreeBSD install on a network attached disk storage box I am building out of recycled desktop PC parts. I've got several big spinning rust disks in it, one 500 Gb SATA attached SSD, and one 1000 Tb NVME drive (and a delightful excess of RAM and fast cores <smile>.)

I told the installer to use the SSD and NVME as a ZFS mirror for my root (and EFI, swap) partitions.

Now I see I ended up with my root zpool on a mirror of 450 Gb on the SSD, and 950 Gb on the NVME (the rest of those two drives being swap and EFI).

Does that mean that I am now wasting the 950-450 == 500 Gb of my NVME? Surely a simple two way mirrored zpool can't use more than the smallest vdev (450 Gb in this case) for the useful size of each vdev in the mirror.

I would like to have the 500 Gb on that NVME unused for now, available for (unmirrored) scratch space in the future.

Should I have used a more manual disk configuration during the install, to get this, rather than just "hoping" the installer woudn't waste space making an unbalanced mirror?

Is my best remedy now to remove the NVME half of my root mirror, use gpart to make a 450 Gb partition on that NVME, and then reattach that newly created, smaller, partition back into my root mirror?

If so, would that be using zpool-detach/zpool-attach, or zpool-remove/zpool-add ?

Thanks for keeping the classic Unix way of doing things going strong all these years.
 
Would be useful to see the zpool status and gpart show {nvme device} output to confirm.

But yes, if the installer has created a single partition for the OS on each disk and mirrored them, you will only get the space of the smaller disk. You should be able to use zpool detach {pool} {device} to remove the nvme device, re-partition and re-attach.
 
ThePythonicCow said:
I would like to have the 500 Gb on that NVME unused for now, available for (unmirrored) scratch space in the future.
Click to expand...
Aha - I should keep that 500 Gb of my NVME as spare. Failures on spinning rust tend to take out the entire drive, but one common failure scenario for flash memory is wearing the bits out from too many writes ... so another vdev of the same size, even if on the same physical drive, resting quietly as a spare, could be put to use if either of the two active vdevs in the mirror started flaking out with tired old bits. That would give me some time to repopulate that mirrored root hardware with some new SSD or NVME flash drives, good for another long time.
SirDice said:
Bad idea. It's going to be as slow as the SATA SSD, you're not getting the benefits of the fast NVMe drive.
Click to expand...
Agreed - as slow as the slowest drive in the mirror.

But that didn't matter to me. Once the system gets booted up and running, it will be doing its one job of being a networked file store for the big data on the big spinning rust drives almost entirely out of RAM, limited only by size of the RAM caches, and the speeds of the spinning rust drives and my modest 1 Gb network. A few maintenance cron jobs will probably page in and out from the NVME+SSD root drive, but if they spend 10 minutes doing what should have been done in 10 seconds (*), I'll never notice. I only threw in the NVME because it was sitting in my used parts box, with some life still left in it, and I had one open NVME slot on the also used old motherboard I'm using.

(*) I exaggerate obviously. Those SSD's are only a few times slower than a PCIe3 NVME, not a few dozen times.
 
ThePythonicCow said:
1000 Tb NVME
Click to expand...
Obvious typo - it's a 1 Tb or 1000 Gb drive - maybe in another 10 or 20 years I'll be able to get a 1000 Tb drive ;).

In my lifetime, I've gone from 80 column punch cards, to 160 Kb floppies to spinning rust drives of 5 Mb, 1 Gb, 1 Tb, and now 12 Tb (or a bit more if I wanted), with solid state storage closing the gap. It's been an awesome time to be a storage glutton.
 
One of advantages of software RAID is ability to use part of the drive. You can create a mirror of your SATA SSD and half of the NVME, and use other half of the NVME as a non-redundant volume or swap.

Note, that 500 GB is an overkill for both root & swap.
 
>> You can create a mirror of your SATA SSD and half of the NVME, and use other half of the NVME

Yes - I've experimented with such splits. My initial posting of this thread was partly in surprise that the installer used up all my 1000 Gb NVME to mirror with my 500 Gb SSD, rather than leaving half my NVME out of the mirror, so available for other such purposes.

>> Note, that 500 GB is an overkill for both root & swap.

That depends on how much stuff one wants on one's fastest drive. I keep both root and /home on my best drive.

I have lots of data that I like having rapid access to under /home.

I have been in the habit of keeping my root+home drive <= 50% used for many decades now, as often I've been on devices or file systems that stayed faster (less fragmented, typically) if kept less fully utilized.
 
You must log in or register to reply here.
Back
Top