9.0_RELEASE upgrade with RAID1

I have been prepping my servers for a FreeBSD 8.2R -> 9.0R upgrade. So far the upgrade process has been very easy and straight-forward.

This thread caused me some pause on my gmirror server. It has two mirrors using MBR.

Has this problem been confirmed from someone else regarding an issue with gmirror and the upgrade? Can we also confirm he was using MBR and not GPT?

If my memory serves me right, GPT may not be used for entire-disk mirroring but only individually on each partition, correct (end of disk metadata storage)? Whereas MBR you could do entire-disk or partition mirroring.

Code: 
[root@postfix /usr/ports/security/maia]# gpart status
         Name  Status  Components
 mirror/gm0s1      OK  mirror/gm0
mirror/gm0s1a      OK  mirror/gm0s1
  mirror/gm1a      OK  mirror/gm1

Code: 
[root@postfix /usr/ports/security/maia]# gpart show
=>       63  625142322  mirror/gm0  MBR  (298G)
         63  625137282           1  freebsd  [active]  (298G)
  625137345       5040              - free -  (2.5M)

=>        0  625137282  mirror/gm0s1  BSD  (298G)
          0    8388608             1  freebsd-ufs  (4.0G)
    8388608    8388608             2  freebsd-swap  (4.0G)
   16777216   16777216             4  freebsd-ufs  (8.0G)
   33554432    4194304             5  freebsd-ufs  (2.0G)
   37748736   41943040             6  freebsd-ufs  (20G)
   79691776  545445506             7  freebsd-ufs  (260G)

=>        0  976773167  mirror/gm1  BSD  (466G)
          0         16              - free -  (8.0K)
         16  976773151           1  freebsd-ufs  (466G)

If someone has done a successful upgrade with [gmirror] across the OS drives, I would appreciate some assurance. Thanks!
 
robertclemens said:
If my memory serves me right, GPT may not be used for entire-disk mirroring but only individually on each partition, correct (end of disk metadata storage)? Whereas MBR you could do entire-disk or partition mirroring.
Click to expand...

Yes.
 
I posted a Problem Report yesterday.

I don't think that the upgrade process corrupted the MBR partition scheme which I had in R8.3. Since FreeBSD R9.0 defaults to GPT, it's more likely that the problem lies in backwards compatibility.
 
ericmacmini said:
I posted a Problem Report yesterday.

I don't think that the upgrade process corrupted the MBR partition scheme which I had in R8.3. Since FreeBSD R9.0 defaults to GPT, it's more likely that the problem lies in backwards compatibility.
Click to expand...

Can you post a link to the PR you filed so people can see what needs to be done or do you think a solution will come through freebsd-update?
 
Anyone have any idea what to do here? I'm a longtime newbie and was hoping to have this installation last a while. Would it be best to just reinstall or is this something that will get resolved? If I do reinstall, can I set up a raid 1 or will I end up with the same issue? Thanks.
 
If you like the simplicity of gmirror the way it used to be done, I think the right thing to do would be stick with the 8.x series for now. It's still considered current and will be maintained for a while.
 
Would it be possible/simple to rollback the freebsd-update -upgrade -9.0_RELEASE I just did?
 
archen said:
If you like the simplicity of gmirror the way it used to be done, I think the right thing to do would be stick with the 8.x series for now. It's still considered current and will be maintained for a while.
Click to expand...

gmirror(8) didn't change, it'll still work the same. Something changed elsewhere. Backing up the data and recreating the mirror might (should) solve the problem.
 
Hi Lido,
I've experienced exactly the same problem after an upgrade. I followed this old thread and it solved my problem.
To summarize, when you are on the boot menu, you need to choose to boot in single user and after run an FSCK on your filesystem.
Hope it helps
 
Well, I also went through this non-booting integrity check fail. The kern.geom.part.check_integrity=0 saved my life either.
The story is similar to others - I upgraded my source tree to FreeBSD 9.0-RELEAEE and I found the unpleasant surprise.
I have whole-device mirror with MBR partition (this is an old install). I noticed following messages during boot:

Code: 
Jan 21 14:24:11 leo kernel: GEOM_MIRROR: Device mirror/gm0 launched (2/2).
Jan 21 14:24:11 leo kernel: GEOM_PART: integrity check failed (mirror/gm0, MBR)
Jan 21 14:24:11 leo kernel: GEOM: mirror/gm0s1: geometry does not match label (16h,63s != 255h,63s).

The partition was created a long time ago with sysinstall and there are NO other partitions - this is a dedicated FreeBSD server.
I remember I saw this warning for long time but since it worked OK I never doubted about it.

I hope this post will help to find an universal, long-term solution.
 
I see that a lot of people here are facing some problems with this issue.

Filling a PR is a good idea because that way it will get the attention of the developers.
 
This morning I backed up all relevant data, since I am considering to gmirror individual partitions instead of the whole disk.
I have following steps in mind, at step 3 I have a question. Of course, other tips/remarks are very welcome also.

1. create a new GPT scheme on my second disk (ada1):

Code: 
# gpart create -s gpt ada1
# gpart add -t freebsd-boot -l boot-secondary -s 64 ada1
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ada1
# gpart add {my other necessary partitions}

2. dump / restore all data from ada0 to the newly GPT schemed disk ada1.

3. Reboot the system on disk ada1.
[Question:] How can force the /boot/loader.conf to boot from disk ada1 ?

4. Create an identical layout on the primary ada0 disk.

5. dump / restore all data from ada1 back to the newly GPT schemed primary disk ada0 .

6. Reboot the system on disk ada0 .

7. Setting up the gmirror for individual partitions.
 
ericmacmini said:
This morning I backed up all relevant data, since I am considering to gmirror individual partitions instead of the whole disk.
I have following steps in mind, at step 3 I have a question. Of course, other tips/remarks are very welcome also.

1. create a new GPT scheme on my second disk (ada1):
Click to expand...

Give the new partitions GPT labels that are different from ada0.

3. Reboot the system on disk ada1.
[Question:] How can force the /boot/loader.conf to boot from disk ada1 ?
Click to expand...

Probably easiest to use the boot menu to select that drive. Or just swap cables. (Mount and edit the /etc/fstab on the new drive to refer to the new GPT labels before booting. Or do what I usually do, forget it and have to manually enter the path to the boot device.)

5. dump / restore all data from ada1 back to the newly GPT schemed primary disk ada0 .
Click to expand...

In theory, it's fine. In practice, make a full backup first. If you back up ada0 to files on some other media, they can be restored to ada1, so the backup doesn't have to run twice.
 
I agree that there might be a problem with GPT partition scheme and GMIRROR on whole drive. I use GPT and per-partition mirror and journal on my newer installs. In my case and as I can see in some other mentioned here we don't use GPT scheme, but MBR.
Is there any way to see what is the fail in the integrity check? This might explain the problem.
 
Unfortunately I am not a C-language expert. I already looked up the source code for the command gpart show.

The CORRUPT error message is generated by the following C-function in the source code:

Code: 
gpart_show_geom(struct ggeom *gp, const char *element, int show_providers)
.....
	printf("=>%*jd  %*jd  %*s  %s  (%s)%s\n",
	    wblocks, (intmax_t)first, wblocks, (intmax_t)(last - first + 1),
	    wname, gp->lg_name,
	    scheme, fmtsize(pp->lg_mediasize),
	    s ? " [CORRUPT]": "");
......

I suppose that the CORRUPT message is generated when variable s equals FALSE in the statement above. The variable s is last set with:

Code: 
	s = find_geomcfg(gp, "state");
	if (s != NULL && *s != 'C')
		s = NULL;

Maybe, somebody with much more C-Language skills can help us futher from here.

[QUESTION] Is there an easy way to only recompile gpart from the /src/ directory, or do you need to make buildworld completly?
 
If you do not want to completely break your system, don't try hack the code :)
First of you should understand where is the problem.

The problem is in your partition table. As you see it marked corrupted.
When partition table marked as corrupted you can not modify it. It is protection mechanism.
You can enable verbose boot mode and you will see why your partition table marked as corrupted.
There will be some lines that begins from "GEOM_PART: ".
 
So here is my verbose boot mode output:
Code: 
Jan 23 22:09:38 leo kernel: GEOM_MIRROR: Device mirror/gm0 launched (2/2).
Jan 23 22:09:38 leo kernel: GEOM_PART: partition 1 has end offset beyond last LBA: 1250263727 > 1250263726
Jan 23 22:09:38 leo kernel: GEOM_PART: integrity check failed (mirror/gm0, MBR)
Jan 23 22:09:38 leo kernel: GEOM: mirror/gm0s1: geometry does not match label (16h,63s != 255h,63s).

Since my drives were partitioned with sysinstall (maybe in FreeBSD 6), then re-partitioned for FreeBSD 8 (there was again problem with partitions and the system couldn't boot) may I suggest there was an error in Sysinstall or fdisk?
 
I have a sinking feeling that this is due to the instructions for gmirror(8) in the Handbook. It has the user set up a drive, then overrides the safety and writes the gmirror meta-data into the last block of that partition. When gmirror is asked for the partition size, it will come back as one less block (so the meta-data isn't overwritten). But the partition was created first, so the partition is one block too large. The cure to that would be to do it the right way, by creating the mirror first and then filesystems inside the mirror.
 
I must admit this mirror was "born" by following the exact instructions in the Handbook. The Handbook itself was one of the major reasons for me to choose FreeBSD for my production servers back in 2002.
My newer installations are based on gpt partition scheme and per-partition mirorring - if we have 2 HDDs (ad4, ad6) I use the following:

Code: 
gpart create -s gpt ad4 (or ad6)
gpart add -s 64K -t freebsd-boot ad4 (or ad6)
gpart add -s 2G -t freebsd-ufs -l root1 ad4 (or root2 ad6)
gmirror label -vb load root ad4p2 ad6p2
newfs -L root /dev/mirror/root

Do you think that this implementation will be affected by the same issue? I have not enough time to try it myself but I'll do my best in next few days.

Warning for all newer users: This post is not a manual and the procedure described above misses some important details!
 
von_Gaden said:
Do you think that this implementation will be affected by the same issue? I have not enough time to try it myself but I'll do my best in next few days.
Click to expand...

That should be fine. The partition provides n blocks, the mirror reserves the last one for itself and provides n-1 blocks to the filesystem.

But please post back after trying it. An "I actually did it" is worth a thousand oughtas.

There are other ways to do it, of course. Use MBR partitioning, create the mirror out of those partitions. And there can be more than one mirror. My gmirror article shows how to create a mirror for each filesystem with GPT partitions.
 
wblock@ said:
But the partition was created first, so the partition is one block too large. The cure to that would be to do it the right way, by creating the mirror first and then filesystems inside the mirror.
Click to expand...

Yes. But not filesystem. The mirror should be created first, then partition table on top of mirror. E.g.
Code: 
# gmirror label -v gm0 ad4 ad6
# gpart create -s mbr mirror/gm0
# gpart add -t freebsd mirror/gm0
....
 
You must log in or register to reply here.
Back
Top