ZFS Recovery of Encrypted ZROOT Mirror in degraded state

P

petrl

Hi Guys,

I am quite new to FreeBSD and I'd like to ask you for help / recommendation for general ZFS implementation scenarios for personal workstation.

Basically, my current and desired setup is Encrypted ZROOT on two mirrored internal SATA drives. I was able to successfully install FreeBSD 11 on top of mirrored+encrypted ZFS device. Even simple test - removing physically one drive from pool - proved that ZROOT will survive the single disk outage. Nevertheless, my current problem is how to recover degraded pool where encrypted devices were present.

Current setup and state is following:

zroot
mirror-0
ada0p3.eli - unavailable ( primary ie. sata0 drive disconnected during test )
ada1p3.eli - online ( secondary ie. sata1 alive serving data )

Simple reboot will trigger re-silvering operation and fix the pool, so far so god, but I'd like to know how to recover it manually. I had no success with replace, scrub, history etc. commands.

Please can you advice how to fix "it" manually or maybe where to find what the "zfs" is exactly doing during boot / recover operation regarding to re-silvering procedure and eli devices ?

Thank you,

Petr.
 
No offense, but why do you pick something so advanced, if your not familiar with FreeBSD and ZFS?
I'm not a ZFS pro myself but have used it for quite some time now and I think there isn't THAT command that will do it in ZFS.
You sometimes have to "replace" a provider, another time you can just "online" it. The way to fix a problem always depends on what has happend to your pool.

Since the drive is unavailable i'd try:
Code: 
zpool zroot attach ada0p3.eli
or if that won't do it, try:
Code: 
zpool online zroot ada0p3.eli
You should also consider learning to use gpart and how to label your drives, to avoid pulling the wrong drive from an allready degraded mirror in the future. The ZFS basics are in the Handbook: https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/zfs.html

Even for me as a long time FreeBSD, ZFS was overwhelming in the beginning (still is).
Bought this good book called "FreeBSD Mastery: ZFS" from Michael W. Lucas and Allan Jude

And please take a look at the formatting guidelines: Thread 49535 ;)

cheers
Matthias
 
Hi Matthias,

good question and yes my knowledge of FreeBSD and ZFS is very very limited, nevertheless I think, I gained a certain amount of knowledge over last two decades as a Storage Architect / Engineer, therefore it could explain why I am "fiddling" with such a setup even as a complete FreeBSD/ZFS/GELI beginner :)

I think "I may say" I know what I am doing or heading to, what are the expected results of my doing, but I am bit of lost in ZFS/GELI, for sure. It is not that straight forward in comparison to for example working with VGs/LVs etc. on AIX, but it could be my subjective feeling based on lack of experience of ZFS/GELI.

Further "DR" tests explained a bit that I was wrong in my understanding of how the mirror works together with GELI, since same test without encrypted drivers succeeded and I was able to put mirror online on-fly without any problems. I think the main problem is that before I try to instruct ZFS to "recover" mirror I must simply involve GELI to open slot/partition ada0p3.eli to be able to recover mirror.

This also explain, why it worked automatically after reboot. During boot, once I have entered password, ada0p3.eli was opened for ZFS to start/handle recovery ops.

Well, I definitely need to take a better look on Encryption and ZFS.

Thank you for information provided and hint on ZFS book, I think you just helped my wife find interesting Xmas present for me :)

Have nice day.

Petr
 
Hei Petr,

sounds like you allready made the experience with encryption that I havn't had the patience to learn until now.

I would also like to use GELI on my server but figured out it's better for me to become more experience with ZFS first and use GELI later.
You seem to be in the right business to learn all that quick, though :)

Glad I could help with a useful christmas present for you ;)

Matthias
 
One thing to keep in mind with GEOM and ZFS is that ZFS doesn't care what the underlying providers are and is absolutely oblivious to how they work or how they are configured. All it cares is that all of the devices listed in the pool label(s) are available at the pool import time, if they aren't it's going to give you a generic error that will not tell you how to fix the problem with underlying providers, especially if the underlying providers are GELI providers ZFS will not know what's wrong with the GELI configuration.
 
Hi Matthias, kpa,

it is exactly as kpa described in post above. The ZFS is not aware of GELI layer, it expects /dev/ada0p3.eli as member of mirror-0, in comparison to simple non-encrypted /dev/some-device. Once the /dev/ada0p3.eli is "constructed" all ZFS healing actions can take place.

I don't know why, but I somehow expected GELI native part of ZFS, which was a wrong assumption :)

Regards,

Petr.
 
You must log in or register to reply here.
Back
Top