Full disk gmirror, gptboot warning

Hi all!

I have been experimenting with 3TB drives, so as far as I know, I have to use GPT.

Now, I usually use gmirror on the complete, raw disk. This makes replacing a failed disk an extremely fast operation (only one command to insert it, instead of partitioning it)

Only, with GPT I have noticed a little problem. At boot, the boot loader gives me the warning: gptboot: invalid backup GPT header.

The system seems to be fine. But before I go any further, could this mean trouble?

I *think* it's not a big deal for me. I really installed the GPT *within* the gmirror volume. So it's actually a bit smaller than the whole disk, since some gmirror metadata is in the last sectors. Something like this:

Code:

+-------------------------------------------------------+
| ad11                                                  |
|                                                       |
|  +--------------------------------------------+----+  |
|  | mirror/home                             gpt|meta|  |
|  |                                         bak|data|  |
|  +--------------------------------------------+----+  |
|                                                       |
+-------------------------------------------------------+

So, it seems to me, since the boot loader is not gmirror aware, it just looks for the GPT backup in the last sector of the physical disk.

In which case there should be no problem, because as the OS takes over, the GPT backup is where the tools will expect to find it, and if needed I can do a restore.

Are my assumptions correct? Or is there possibility for trouble later?

Thanks!

Sebulon

Oct 24, 2012

#2

Hi,

reading this thread may help you better understand the internals of GPT and gmirror:
http://lists.freebsd.org/pipermail/freebsd-current/2012-June/034805.html

/Sebulon

J

jem

Oct 24, 2012

#3

semafoor said:
So, it seems to me, since the boot loader is not gmirror aware, it just looks for the GPT backup in the last sector of the physical disk.

In which case there should be no problem, because as the OS takes over, the GPT backup is where the tools will expect to find it, and if needed I can do a restore.

Are my assumptions correct? Or is there possibility for trouble later?

Thanks!

You've got it almost exactly right. It's not just the bootloader that isn't gmirror aware, but the kernel itself isn't until the gmirror module loads.

Before the gmirror module loads, the kernel just sees two physical disks with a secondary GPT header incorrectly located in the second from last sector.

Once the module loads, the GPT header is seen in the correct location on the resulting mirror device.

wblock@

Developer

Oct 24, 2012

#4

semafoor said:
Hi all!

I have been experimenting with 3TB drives, so as far as I know, I have to use GPT.

Did you try MBR?

So, it seems to me, since the boot loader is not gmirror aware, it just looks for the GPT backup in the last sector of the physical disk.

The end of the disk is the defined location for the backup GPT. The concern with putting it anywhere else is that booting systems won't find it. Which is actually the problem you experienced.

If you do actually have to use GPT, it's possible to mirror GPT partitions rather than the whole drive. But try MBR first.

OP

semafoor

Oct 24, 2012

Thread Starter
#5

Thanks for the replies!!

I haven't tried MBR, since (1) I was assuming that MBR implied a 32bit LBA which would not support such large drives, and (2) because of 4k sector size I need to align the partitions, for which I only found the 'gpart -a' switch quickly...

I've actually considered mirroring the partitions inside the GPT, I have that setup at home right now, but I found it really cumbersome to add a new gmirror provider! First I had to get the partitions right (copying it verbatim gave me a problem because I needed to change the labels). Also I'd hate to have two separate procedures for swapping a disk! But, it's true that I didn't have this warning then, because gmirror only worked inside the partitions.

At the same time I really love FreeBSD's storage architecture for just giving me all these raw interfaces to tie together using gmirror and geli. It's just that GPT is a little new and scary for me

wblock@

Developer

Oct 24, 2012

#6

semafoor said:
Thanks for the replies!!

I haven't tried MBR, since (1) I was assuming that MBR implied a 32bit LBA which would not support such large drives,

Could be, but worth trying to make sure.

and (2) because of 4k sector size I need to align the partitions, for which I only found the 'gpart -a' switch quickly...

gpart(8) also does MBR. In fact, it does a lot of different partitioning schemes. Here's a message I posted yesterday on the freebsd-questions mailing list that shows setting up a 4K-aligned MBR and bsdlabel with gpart(8): http://lists.freebsd.org/pipermail/freebsd-questions/2012-October/245751.html

I've actually considered mirroring the partitions inside the GPT, I have that setup at home right now, but I found it really cumbersome to add a new gmirror provider! First I had to get the partitions right (copying it verbatim gave me a problem because I needed to change the labels). Also I'd hate to have two separate procedures for swapping a disk! But, it's true that I didn't have this warning then, because gmirror only worked inside the partitions.

There are some concerns with mirroring partitions. The one that most concerns me is that replacing a whole disk causes a bunch of rebuilds to take place at the same time. Head contention can be terrible for that.