ZFS Single zpool with two mirrors or two zpool mirrors?

[semi-ambivalent]

I have a machine with a zfs bootable root comprised of a two-disk mirror plus a spare. I added three more disks as a three-disk mirror in a separate zpool. One of those disks was then also detached and re-added as a spare for that mirror only. No disks in the second zpool have an OS or bootcode. zpool status just shows them as buckets with a single GEOM partition; ada0, ada1, ada2.

This is all just experimentation but, compared to what I have now (rpool mirror+spare and dpool mirror + spare), would there have been any benefits to have instead created the second pool as a second mirror on the boot pool (rpool mirror-0 and mirror-1)? I'd like to think dpool is physically transportable in the event that rpool is wiped out but really, what is the possibility or need of that. I'm also finding ZFS to be almost too flexible.

 

[PMc]

would there have been any benefits to have instead created the second pool as a second mirror on the boot pool (rpool mirror-0 and mirror-1)?

Then, if one of mirror-0 or mirror-1 gets completely broken, all data is gone. In case of two pools, if one breaks, only that one is dead. That is the advantage of multiple pools. The downside is, you cannot move diskspace around between pools, while within one pool all files share the same space provisioning.
Anonther thing to consider is spinning-down of disks. If a pool gets accessed, all the active disks in the pool must spin. So, for data that is only occasionally used, it can make sense to put it in it's own pool.

 

[ekingston]

I maintain a separate boot pool and data pool. So far it came in handy twice.

First time was when I messed up and toasted the root pool, requiring that I re-built it (not yet in production).

Second time was a hardware upgrade. The data pool was exported from the old system and then imported into the new system. No need to copy or backup/restore the data.

But, as you point out, ZFS is flexible. So, you can do it whichever way you prefer.