UFS gmirror degraded - how to fix?

pww

pww

Hello all,

I'm currently using FreeBSD 10.0-p18 with a set of gmirror'd drives. I'm getting this error from gmiror status:
Code: 
  Name  Status  Components
mirror/boot  COMPLETE  ada0p1 (ACTIVE)
[INDENT][INDENT][INDENT][INDENT][INDENT][INDENT]   ada1p1 (ACTIVE)[/INDENT][/INDENT][/INDENT][/INDENT][/INDENT][/INDENT]
mirror/swap  COMPLETE  ada0p2 (ACTIVE)
[INDENT][INDENT][INDENT][INDENT][INDENT][INDENT]   ada1p2 (ACTIVE)[/INDENT][/INDENT][/INDENT][/INDENT][/INDENT][/INDENT]
mirror/root  DEGRADED  ada0p3 (ACTIVE)
[INDENT][INDENT][INDENT][INDENT][INDENT][INDENT]   ada1p3 (ACTIVE)[/INDENT][/INDENT][/INDENT][/INDENT][/INDENT][/INDENT]
I recently had trouble with drive ada1, so I tried gmirror remove ada1p3, then newfs the partition, and then gmirror insert root /dev/gpt/root1.

gmirror status says root is complete, and starts synchronizing. A few hours later, the status message is shown above.

Any ideas on how to fix?

Thanks
-phil
 
SirDice said:
This should help: 19.3.5. Recovering from Disk Failure
Click to expand...

Thanks for the reply, but that was the first thing I tried (multiple times, I might say). I even tried to newfs the partition (ada1p3) to get rid of the metadata, and re-inserting; the gmirror status switches to synchronizing, goes through the completion percentages, then says "degraded" when its done.

Eventually, I fixed the problem, by destroying the entire mirror and recreating it. It was a harrowing experience, since I had to boot single user from the mountroot command since I didn't remember that the mount points for booting still had the references to the old destroyed mirror.
 
Just to affirm, SirDice's link is correct. (I just tested this the other week, updating some of my company's documentation and it should be as simple as those two commands given in his link)
 
scottro said:
Just to affirm, SirDice's link is correct. (I just tested this the other week, updating some of my company's documentation and it should be as simple as those two commands given in his link)
Click to expand...

Oh, I know it is right, too. I've used that procedure may times before on my other systems when I had disk failures. It is that this time the procedure didn't work out, and I don't know why.
 
newfs(8) will not erase metadata, because the metadata is outside the partition containing the filesystem. Unless something unsafe has been done, that is.

This is not one mirror but three. Using gmirror(8) and GPT partitioning can be tricky. If the third partition overlapped the mirror metadata, it would have exactly the effect you describe--the mirror would rebuild, and the very last block would overwrite the metadata, making the mirror degraded. If the procedure you followed to create the mirror messed with the GEOM "debugflags" sysctl, please stop using that procedure. The procedure in the Handbook is tested and does not use GPT because of metadata conflicts.
 
wblock@ said:
newfs(8) will not erase metadata, because the metadata is outside the partition containing the filesystem. Unless something unsafe has been done, that is.

This is not one mirror but three. Using gmirror(8) and GPT partitioning can be tricky. If the third partition overlapped the mirror metadata, it would have exactly the effect you describe--the mirror would rebuild, and the very last block would overwrite the metadata, making the mirror degraded. If the procedure you followed to create the mirror messed with the GEOM "debugflags" sysctl, please stop using that procedure. The procedure in the Handbook is tested and does not use GPT because of metadata conflicts.
Click to expand...

Well, I didn't use any "debugflags" sysctl trickery to make my mirrors. These are the steps that I used:
Code: 
gpart create -s gpt ada0
gpart add -s 64k -t freebsd-boot -l boot0 ada0
gpart add -s 8G -t freebsd-swap -l swap0 ada0
gpart add -t freebsd-ufs -l root0 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada0
(then do the same thing for ada1)
gmirror label boot /dev/gpt/boot0 /dev/gpt/boot1
gmirror label swap /dev/gpt/swap0 /dev/gpt/swap1
gmirror label root /dev/gpt/root0 /dev/gpt/root1
(and then do the newfs stuff)
It's my impression from reading the man pages, that the gmirror metadata is stored on the last sector of the provider (i.e. the gpt partition), not the disk itself, so I think should be ok.

From the gmirror(8)man page:
The gmirror utility uses on-disk metadata (stored in the provider's last sec-
tor) to store all needed information.
Click to expand...

Is my interpretation wrong? Will I eventually have a problem?
 
Last edited by a moderator:
That procedure doesn't have anything obviously wrong. It does create three mirrors, which can be a problem when a drive dies and is replaced. The three mirrors will be resynchronized simultaneously on the new drive, so there will be very high usage and head contention. That will be slow, and will place high stress on the old drive that is still working but might be the same age as the one that failed and nearing failure itself.
 
You must log in or register to reply here.
Back
Top