[Solved] Upgrade to 8.2 - GPT became corrupt?

CumpsD · #1 February 27th, 2011, 11:20

Yesterday I tried to do a binary upgrade to 8.2-RELEASE (using freebsd-update), coming from 8.1-RELEASE-p2. The upgrade process went fine, up until the point I rebooted using the GENERIC kernel (nextboot -k GENERIC).

After rebooting, I was unable to mount my root filesystem. After some searching I found this to be the cause:

Code:

GEOM: mirror/gm0: corrupt or invalid GPT detected.
GEOM: mirror/gm0: GPT rejected -- may not be recoverable

Which is strange, since I then rebooted again using my old (custom) kernel, and I could boot again and everything is running.

However, something I had on my old kernel was this:

Code:

GEOM_MIRROR: Device mirror/gm0 launched (2/2).
GEOM: mirror/gm0: the secondary GPT table is corrupt or invalid.
GEOM: mirror/gm0: using the primary only -- recovery suggested.

I had this as soon as I set up a mirror raid, but everything worked.

Now I'm at a loss. I didn't change anything (shown from the fact my old kernel still boots), but the new GENERIC stops at the GPT part. I know 8.2 did some stuff for GPT, but I don't understand how it goes from "recovery suggested" to plain "corrupt and I will refuse to boot".

This is how I partitioned my disk (I'm new to FreeBSD and used one big root partition, sorry)

Code:

[root@cc001 ~/gptbackup]# gpart show /dev/mirror/gm0
=>       34  156301421  mirror/gm0  GPT  (75G)
         34        128           1  freebsd-boot  (64K)
        162       1886              - free -  (943K)
       2048    8388608           2  freebsd-swap  (4.0G)
    8390656  147910799           3  freebsd-ufs  (71G)

And my mirror:

Code:

[root@cc001 ~/gptbackup]# gmirror status
      Name    Status  Components
mirror/gm0  COMPLETE  ad4
                      ad6

When I installed 8.1, I first partitioned ad4, after which I created the mirror, which synced everything to ad6.

Would appreciate if anyone had an idea how I can manage to boot using the 8.2 GENERIC so I can finish the upgrade process

jem · #2 February 27th, 2011, 13:41

If you partitioned your disk before setting up the mirror, then GPT will have placed its secondary header in the last sector of the disk.

When you then set up a gmirror, it will have stored its metadata also in the last sector of the disk, clobbering the secondary GPT header.

In 8.1, GPT header checksumming wasn't fully implemented, but it is now in 8.2 which is probably why you're now seeing complaints about corruption.

The solution is to create your mirror from the raw disks, BEFORE partitioning the resulting gmirror device. You'll notice that the gmirror device is one sector smaller than the two disks it's made from because it hides it's own metadata sector. When GPT puts its secondary header at the end of the mirror device, it'll actually end up on the second from last sector of the physical disks.

Then, gmirror metadata and GPT secondary header won't trample all over eachother.

CumpsD · #3 February 27th, 2011, 17:22

Yes, I partitioned it before the mirror :/

Is it possible to repartition my current disks *without* losing any data on it?

My guess I came up with while thinking about it today, after reading your answer:

- I need to boot using fixit mode
- Somehow remove the current partitioning scheme from the mirror
- Break up the mirror
- Check if the harddisks itself don't have any partitioning left
- Restore the mirror
- Repartition the mirror, using the same sectors as before
- Data magically was saved?

What's the safe way to go about this making sure I can keep everything I got on the disks right now?

jem · #4 February 27th, 2011, 19:31

In the same situation, I'd try the following:

Detach ad6 from your existing gm0 mirror, leaving it running on just ad4.
Create a new, second gmirror device with just ad6 initially
GPT partition your new mirror, install bootblocks etc (might be a good time to revise your partition layout)
Move data from your old gmirror to your new gmirror, with dump and restore
Attempt to boot the system from the new mirror
If successful, destroy the old mirror and attach ad4 to the new one

Note that this is just off the top of my head. There may well be problems I haven't thought of with this process.

CumpsD · #5 February 27th, 2011, 21:55

I'm halfway there

Just got one last question I'm unsure about (found different sources using different statements)

When you say "install bootblocks", would this do?

  gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 mirror/gm0

Or do I need different switches?

**DutchDaemon** · #6 February 28th, 2011, 00:02

CumpsD, use proper formatting, thanks.

jem · #7 February 28th, 2011, 09:14

Quote:

Originally Posted by CumpsD

I'm halfway there

Just got one last question I'm unsure about (found different sources using different statements)

When you say "install bootblocks", would this do?

  gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 mirror/gm0

Or do I need different switches?

Spot on, although gm0 is your old mirror isn't it? You'll need the bootblocks on the new mirror.

When you attempt to test booting from your new mirror, you'll probably need to use your BIOS's boot menu to boot from your second disk (ad6).

I'd strongly advise having the memstick image written to a USB flash drive to aid in recovery if something goes wrong.

CumpsD · #8 February 28th, 2011, 20:00

Yes, I did it with the new mirror, but due to some mistakes on my side, gm0 was my new (and only) mirror

I accidentally (don't ask how), managed to not be able to boot into my old mirror anymore, so I went in with a livecd and fixit, broke up the mirror, repartitioned ad6 with the above partition table, and booted into it (after changing the bios to boot from ad6). By miracle it worked, I learned a lot by doing this

(But was scared everything was lost sometimes :p)

Then I recreated the mirror with ad4 in it, and partitioned the mirror device (gm0), I installed the bootcode using the above command, and restored ad6 to gm0 and rebooted. Which worked!

Right now I'm adding ad4 to the mirror as well, which should have it back into the same condition.

However, and this is something that I'm worried about, when I booted, just before I got the options menu (normal boot, safe mode, single user, etc..) I noticed a warning about an invalid backup GPT header. So I'm afraid I'll have the same problems all over again

I'll be able to tell in a few, when the mirror is restored and I'll reboot (using the old 8.1 kernel first to see if I get any GPT complaints)

*fingers crossed*

jem · #9 February 28th, 2011, 20:10

No, don't worry about that initial message about an invalid backup header.

When FreeBSD first starts booting it is accessing a single disk directly, not a mirror device, as it hasn't loaded the geom_mirror module yet. Because you created your GPT partitioning on the mirror device, the secondary GPT header is on the second from last sector of the physical disk. GPT considers that a problem and complains about it.

However, as soon as the gmirror module loads and the last sector of the disks (containing the gmirror metadata) are hidden from view, the secondary GPT header appears to be on the last sector of the gmirror device, which is OK.

CumpsD · #10 February 28th, 2011, 20:15

Indeed, the only GPT warning was during boot, but nothing in dmesg anymore, sounds good. Now to try the 8.2 GENERIC

(Sorry DutchDaemon, I kept an eye out but missed the formatting of devices)

**phoenix** · #11 February 28th, 2011, 20:19

On the -stable mailing list right now, there's a discussion about this very issue. There appears to be a bug in the way GEOM tastes different providers, with the GPART class overriding the GMIRROR class. Or, at least with the way the gptboot code does the GEOM tasting.

The best way to describe the issue is like so:

Here's the entire disk before you do anything to it:

Code:

-----------------------------
|          ad0              |
-----------------------------

Here's the disk after you use gmirror; the gmirror metadata is put into the last sector of ad0, and creates a new provider gm0 that's 1 sector smaller than ad0:

Code:

-----------------------------
|          ad0           |MD|
-----------------------------
|          gm0           |
--------------------------

Then you partition gm0 using gpart, and it stores it's metadata in the first and last sectors of gm0:

Code:

-----------------------------
|          ad0           |MD|
-----------------------------
|GPT|      gm0       |GPT|
--------------------------
|          gpt/p1    |
----------------------

At this point, everything is configured and stored correctly.

The problem comes when you boot, and the GPTBOOT codes tastes the ad0 GEOM provider. It sees the GPT in the first sector of the disk (which is also the first sector of gm0, and the first sector of gpt/p1), and then automatically looks in the last sector of the disk (ad0) for the secondary GPT ... and finds the gmirror meta-data instead. Thus, it complains about a corrupt secondary GPT.

There's no fix as yet.

CumpsD · #12 February 28th, 2011, 20:22

Ah, thanks for the explanation.

I'm happy it works now, overjoyed

Now rebuilding my ports, kernel and I'm up and running again, ready for the next challenge, trying to freebsd-update my jails

Thanks for the great help all!

CumpsD · #13 February 28th, 2011, 20:24

(Can't edit my last reply, nor the first, not long enough here yet, but if someone could mark it as solved? Thanks)

jem · #14 February 28th, 2011, 20:49

Quote:

Originally Posted by phoenix

Code:

-----------------------------
|          ad0           |MD|
-----------------------------
|GPT|      gm0       |GPT|
--------------------------
|          gpt/p1    |
----------------------

A picture tells a thousand words! Nice illustration of what I was trying to communicate!

gour · #15 April 3rd, 2011, 08:47

Hello,

I am new Free/PC-BSD user (just migrated from Linux after using it since '99) and your post saved me some more hours of troubleshooting while I was fighting with 9.0 PC-BSD snapshot install and decided to use just one HD.

Quote:

Originally Posted by jem

The solution is to create your mirror from the raw disks, BEFORE partitioning the resulting gmirror device. You'll notice that the gmirror device is one sector smaller than the two disks it's made from because it hides it's own metadata sector. When GPT puts its secondary header at the end of the mirror device, it'll actually end up on the second from last sector of the physical disks.

Then, gmirror metadata and GPT secondary header won't trample all over eachother.

Now, I'm curious what would be the recommended way to go from one-disk setup to mirror?

I've two identical hard drives and the first one has small UFS /boot, swap and the rest is ZFS tank with {/,/usr,/var}. How to proceed?

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
[Solved] metadata corrupt ?!	nORKy	Installation and Maintenance of FreeBSD Ports or Packages	29	December 15th, 2011 01:10
ZFS GEOM: mfid1: corrupt or invalid GPT detected	nbari	General	8	September 7th, 2011 08:39
snapshot is corrupt	SeaHag	Installation and Maintenance of FreeBSD Ports or Packages	1	February 6th, 2010 08:56
migrating zfs from opensolaris to freebsd (corrupt gpt)	trash	Installing & Upgrading	2	October 16th, 2009 15:50
GEOM: adX: the secondary GPT table is corrupt or invalid	FestusHagen	General	6	June 14th, 2009 06:24

The Following User Says Thank You to phoenix For This Useful Post:
jb_fvwm2 (March 1st, 2011)