1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Upgrade to 8.2 - GPT became corrupt?

Discussion in 'Installing and Upgrading FreeBSD' started by CumpsD, Feb 27, 2011.

  1. CumpsD

    CumpsD New Member

    Messages:
    7
    Thanks Received:
    0
    Yesterday I tried to do a binary upgrade to 8.2-RELEASE (using freebsd-update), coming from 8.1-RELEASE-p2. The upgrade process went fine, up until the point I rebooted using the GENERIC kernel (nextboot -k GENERIC).

    After rebooting, I was unable to mount my root filesystem. After some searching I found this to be the cause:
    Code:
    GEOM: mirror/gm0: corrupt or invalid GPT detected.
    GEOM: mirror/gm0: GPT rejected -- may not be recoverable


    Which is strange, since I then rebooted again using my old (custom) kernel, and I could boot again and everything is running.

    However, something I had on my old kernel was this:
    Code:
    GEOM_MIRROR: Device mirror/gm0 launched (2/2).
    GEOM: mirror/gm0: the secondary GPT table is corrupt or invalid.
    GEOM: mirror/gm0: using the primary only -- recovery suggested.


    I had this as soon as I set up a mirror raid, but everything worked.

    Now I'm at a loss. I didn't change anything (shown from the fact my old kernel still boots), but the new GENERIC stops at the GPT part. I know 8.2 did some stuff for GPT, but I don't understand how it goes from "recovery suggested" to plain "corrupt and I will refuse to boot".

    This is how I partitioned my disk (I'm new to FreeBSD and used one big root partition, sorry)
    Code:
    [root@cc001 ~/gptbackup]# gpart show /dev/mirror/gm0
    =>       34  156301421  mirror/gm0  GPT  (75G)
             34        128           1  freebsd-boot  (64K)
            162       1886              - free -  (943K)
           2048    8388608           2  freebsd-swap  (4.0G)
        8390656  147910799           3  freebsd-ufs  (71G)


    And my mirror:
    Code:
    [root@cc001 ~/gptbackup]# gmirror status
          Name    Status  Components
    mirror/gm0  COMPLETE  ad4
                          ad6


    When I installed 8.1, I first partitioned ad4, after which I created the mirror, which synced everything to ad6.

    Would appreciate if anyone had an idea how I can manage to boot using the 8.2 GENERIC so I can finish the upgrade process :)
     
  2. jem

    jem Member

    Messages:
    349
    Thanks Received:
    55
    If you partitioned your disk before setting up the mirror, then GPT will have placed its secondary header in the last sector of the disk.

    When you then set up a gmirror, it will have stored its metadata also in the last sector of the disk, clobbering the secondary GPT header.

    In 8.1, GPT header checksumming wasn't fully implemented, but it is now in 8.2 which is probably why you're now seeing complaints about corruption.

    The solution is to create your mirror from the raw disks, BEFORE partitioning the resulting gmirror device. You'll notice that the gmirror device is one sector smaller than the two disks it's made from because it hides it's own metadata sector. When GPT puts its secondary header at the end of the mirror device, it'll actually end up on the second from last sector of the physical disks.

    Then, gmirror metadata and GPT secondary header won't trample all over eachother.
     
  3. CumpsD

    CumpsD New Member

    Messages:
    7
    Thanks Received:
    0
    Yes, I partitioned it before the mirror :/

    Is it possible to repartition my current disks *without* losing any data on it?

    My guess I came up with while thinking about it today, after reading your answer:

    - I need to boot using fixit mode
    - Somehow remove the current partitioning scheme from the mirror
    - Break up the mirror
    - Check if the harddisks itself don't have any partitioning left
    - Restore the mirror
    - Repartition the mirror, using the same sectors as before
    - Data magically was saved? :)

    What's the safe way to go about this making sure I can keep everything I got on the disks right now?
     
  4. jem

    jem Member

    Messages:
    349
    Thanks Received:
    55
    In the same situation, I'd try the following:

    1. Detach ad6 from your existing gm0 mirror, leaving it running on just ad4.
    2. Create a new, second gmirror device with just ad6 initially
    3. GPT partition your new mirror, install bootblocks etc (might be a good time to revise your partition layout)
    4. Move data from your old gmirror to your new gmirror, with dump() and restore()
    5. Attempt to boot the system from the new mirror
    6. If successful, destroy the old mirror and attach ad4 to the new one

    Note that this is just off the top of my head. There may well be problems I haven't thought of with this process.
     
  5. CumpsD

    CumpsD New Member

    Messages:
    7
    Thanks Received:
    0
    I'm halfway there :) Just got one last question I'm unsure about (found different sources using different statements)

    When you say "install bootblocks", would this do?
    gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 mirror/gm0

    Or do I need different switches?
     
  6. DutchDaemon

    DutchDaemon Administrator Staff Member Administrator Moderator

    Messages:
    10,765
    Thanks Received:
    1,870
  7. jem

    jem Member

    Messages:
    349
    Thanks Received:
    55
    Spot on, although gm0 is your old mirror isn't it? You'll need the bootblocks on the new mirror.

    When you attempt to test booting from your new mirror, you'll probably need to use your BIOS's boot menu to boot from your second disk (ad6).

    I'd strongly advise having the memstick image written to a USB flash drive to aid in recovery if something goes wrong.
     
  8. CumpsD

    CumpsD New Member

    Messages:
    7
    Thanks Received:
    0
    Yes, I did it with the new mirror, but due to some mistakes on my side, gm0 was my new (and only) mirror ;)

    I accidentally (don't ask how), managed to not be able to boot into my old mirror anymore, so I went in with a livecd and fixit, broke up the mirror, repartitioned ad6 with the above partition table, and booted into it (after changing the bios to boot from ad6). By miracle it worked, I learned a lot by doing this :) (But was scared everything was lost sometimes :p)

    Then I recreated the mirror with ad4 in it, and partitioned the mirror device (gm0), I installed the bootcode using the above command, and restored ad6 to gm0 and rebooted. Which worked! :)

    Right now I'm adding ad4 to the mirror as well, which should have it back into the same condition.

    However, and this is something that I'm worried about, when I booted, just before I got the options menu (normal boot, safe mode, single user, etc..) I noticed a warning about an invalid backup GPT header. So I'm afraid I'll have the same problems all over again

    I'll be able to tell in a few, when the mirror is restored and I'll reboot (using the old 8.1 kernel first to see if I get any GPT complaints)

    *fingers crossed*
     
  9. jem

    jem Member

    Messages:
    349
    Thanks Received:
    55
    No, don't worry about that initial message about an invalid backup header.

    When FreeBSD first starts booting it is accessing a single disk directly, not a mirror device, as it hasn't loaded the geom_mirror module yet. Because you created your GPT partitioning on the mirror device, the secondary GPT header is on the second from last sector of the physical disk. GPT considers that a problem and complains about it.

    However, as soon as the gmirror module loads and the last sector of the disks (containing the gmirror metadata) are hidden from view, the secondary GPT header appears to be on the last sector of the gmirror device, which is OK.
     
  10. CumpsD

    CumpsD New Member

    Messages:
    7
    Thanks Received:
    0
    Indeed, the only GPT warning was during boot, but nothing in dmesg anymore, sounds good. Now to try the 8.2 GENERIC :)

    (Sorry DutchDaemon, I kept an eye out but missed the formatting of devices)
     
  11. phoenix

    phoenix Moderator Staff Member Moderator

    Messages:
    3,446
    Thanks Received:
    769
    On the -stable mailing list right now, there's a discussion about this very issue. There appears to be a bug in the way GEOM tastes different providers, with the GPART class overriding the GMIRROR class. Or, at least with the way the gptboot code does the GEOM tasting.

    The best way to describe the issue is like so:

    Here's the entire disk before you do anything to it:
    Code:
    -----------------------------
    |          ad0              |
    -----------------------------
    


    Here's the disk after you use gmirror; the gmirror metadata is put into the last sector of ad0, and creates a new provider gm0 that's 1 sector smaller than ad0:
    Code:
    -----------------------------
    |          ad0           |MD|
    -----------------------------
    |          gm0           |
    --------------------------
    


    Then you partition gm0 using gpart, and it stores it's metadata in the first and last sectors of gm0:
    Code:
    -----------------------------
    |          ad0           |MD|
    -----------------------------
    |GPT|      gm0       |GPT|
    --------------------------
    |          gpt/p1    |
    ----------------------
    

    At this point, everything is configured and stored correctly.

    The problem comes when you boot, and the GPTBOOT codes tastes the ad0 GEOM provider. It sees the GPT in the first sector of the disk (which is also the first sector of gm0, and the first sector of gpt/p1), and then automatically looks in the last sector of the disk (ad0) for the secondary GPT ... and finds the gmirror meta-data instead. Thus, it complains about a corrupt secondary GPT.

    There's no fix as yet.
     
    jb_fvwm2 thanks for this.
  12. CumpsD

    CumpsD New Member

    Messages:
    7
    Thanks Received:
    0
    Ah, thanks for the explanation.

    I'm happy it works now, overjoyed ;) Now rebuilding my ports, kernel and I'm up and running again, ready for the next challenge, trying to freebsd-update my jails :)

    Thanks for the great help all!
     
  13. CumpsD

    CumpsD New Member

    Messages:
    7
    Thanks Received:
    0
    (Can't edit my last reply, nor the first, not long enough here yet, but if someone could mark it as solved? Thanks)
     
  14. jem

    jem Member

    Messages:
    349
    Thanks Received:
    55
    A picture tells a thousand words! Nice illustration of what I was trying to communicate! :)
     
  15. gour

    gour New Member

    Messages:
    29
    Thanks Received:
    1
    from one to mirror?

    Hello,

    I am new Free/PC-BSD user (just migrated from Linux after using it since '99) and your post saved me some more hours of troubleshooting while I was fighting with 9.0 PC-BSD snapshot install and decided to use just one HD.

    Now, I'm curious what would be the recommended way to go from one-disk setup to mirror?

    I've two identical hard drives and the first one has small UFS /boot, swap and the rest is ZFS tank with {/,/usr,/var}. How to proceed?