PDA

View Full Version : GEOM: adX: the secondary GPT table is corrupt or invalid


FestusHagen
January 17th, 2009, 19:24
Hi all,

7.1-R, RAID1 on 4 ST31500341AS via a Promise TX4-300.

The RAID array is purly userland data ... Nothing for the OS.
The OS is on it's own disks ad0 & ad1

I'm not exactly sure when it happened, I started out with 6.2-R, Then fresh Installed 6.4-R, Then fresh Installed 7.0-R, Then fresh Installed 7.1-R.

Yes, I've done alot of different configs - I'm Educating!

Sometime during all this I started getting the following error on the RAID disks:
GEOM: adX: the secondary GPT table is corrupt or invalid.
GEOM: adX: using the primary only -- recovery suggested.
Across all 4 disks of the RAID (ad4,ad6,ad8,ad10).
The OS disks are fine.

After some code research I find an undocumented "recover" command that recovers GPT tables.
gpt recover adX
Clears the error though damages the mirror.
gmirror label Data adX
Fixes the mirror but brings back the original error.

Is there a way out of this without losing the data, Yea, I have a BUP but it is many many hours to restore. (over 2TB)

Thanks

-Enjoy
fh : )_~

FestusHagen
January 21st, 2009, 16:03
A) Did I

say something wrong?
say something stupid?

B) Am I

missing the obvious?
not providing enough info?


Or is it just that nobody has an answer?

Thanks

-Enjoy
fh : )_~

r_t_f_m
January 22nd, 2009, 01:42
Hi !

I had a similar problem as described in bin/115406 ( http://www.freebsd.org/cgi/query-pr.cgi?pr=115406 )

After booting without hard disk attached, hot-pluging it produced the same error message.

Once the patch 115406 applied and (re)partitioned , everything works flawlessly.

Hope it helps.

Dutchman01
February 11th, 2009, 14:53
I have tha same problem on freebds 7.1 stable

Feb 11 07:29:12 kernel: GEOM_LABEL: Label ufs/data removed.
Feb 11 07:29:12 kernel: GEOM_LABEL: Label for provider ad8s2 is ufs/data.
Feb 11 07:29:12 kernel: GEOM_LABEL: Label ufs/data removed.
Feb 11 07:29:12 kernel: GEOM_MIRROR: Device mirror/RAID1 launched (2/2).
Feb 11 07:29:12 kernel: Trying to mount root from ufs:/dev/md0
Feb 11 07:29:12 kernel: GEOM_LABEL: Label for provider ad8s2 is ufs/data.
Feb 11 07:29:12 kernel: GEOM: ad4: using the primary only -- recovery suggested.
Feb 11 07:29:12 kernel: GEOM: ad4: the secondary GPT table is corrupt or invalid.
Feb 11 07:29:12 kernel: ad8: 76319MB <WDC WD800JD-00LSA0 06.01D06> at ata4-master SATA300
Feb 11 07:29:12 kernel: ad4: 305245MB <Seagate ST3320620A 3.AAF> at ata2-master UDMA100
Feb 11 07:29:12 kernel: GEOM: ad0: using the primary only -- recovery suggested.
Feb 11 07:29:12 kernel: GEOM: ad0: the secondary GPT table is corrupt or invalid.
Feb 11 07:29:12 kernel: ad0: 305245MB <Seagate ST3320620A 3.AAF> at ata0-master UDMA100
Feb 11 07:29:12 kernel: md0: Preloaded image </mfsroot> 83886080 bytes at 0xc0e6c620

kdbaumann
May 8th, 2009, 01:17
I have the same issues with what appears to be nearly the same setup.

3Ware raid card, dual processor, 7.1R FreeBSd.

I get the same messages. From what I see here there isn't much that can be done and it doesn't seem to impact the drives?

I would ask if the others used gpart or gpt to setup their raids. I used gpart, if everyone else did as well then perhaps the issue is in gpart...

Thanks. Any info would be helpful.

jef
June 14th, 2009, 06:17
gjournal over gmirror over GPT using gpart here

7.2-RELEASE DVD AMD64

ad0: 152627MB <Hitachi HTS541616J9AT00 SB4OA70H> at ata0-master UDMA100
GEOM: ad0: the secondary GPT table is corrupt or invalid.
GEOM: ad0: using the primary only -- recovery suggested.
GEOM_MIRROR: Device mirror/gm0 launched (1/1).
GEOM_LABEL: Label for provider mirror/gm0p2 is ufsid/4a33bf8b5ac9f68d.
GEOM_LABEL: Label for provider mirror/gm0p2 is ufs/root.
GEOM_JOURNAL: Journal 1515293619: mirror/gm0p4 contains data.
GEOM_JOURNAL: Journal 1515293619: mirror/gm0p4 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p4 clean.
GEOM_JOURNAL: Journal 3992286814: mirror/gm0p5 contains data.
GEOM_JOURNAL: Journal 3992286814: mirror/gm0p5 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p5 clean.
GEOM_JOURNAL: Journal 2315895659: mirror/gm0p6 contains data.
GEOM_JOURNAL: Journal 2315895659: mirror/gm0p6 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p6 clean.
GEOM_JOURNAL: Journal 2119265588: mirror/gm0p7 contains data.
GEOM_JOURNAL: Journal 2119265588: mirror/gm0p7 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p7 clean.
GEOM_LABEL: Label for provider mirror/gm0p8 is ufsid/4a3455146bc1ba31.
GEOM_LABEL: Label for provider mirror/gm0p8 is ufs/portbuilder.
GEOM_LABEL: Label for provider mirror/gm0p4.journal is ufsid/4a33bf8c03e8ba1f.
GEOM_LABEL: Label for provider mirror/gm0p4.journal is ufs/var.
GEOM_LABEL: Label for provider mirror/gm0p5.journal is ufsid/4a33bf8cdbd79c2c.
GEOM_LABEL: Label for provider mirror/gm0p5.journal is ufs/tmp.
GEOM_LABEL: Label for provider mirror/gm0p6.journal is ufsid/4a33bf8cf91b4524.
GEOM_LABEL: Label for provider mirror/gm0p6.journal is ufs/usr.
GEOM_LABEL: Label for provider mirror/gm0p7.journal is ufsid/4a33bf8c45bfed60.
GEOM_LABEL: Label for provider mirror/gm0p7.journal is ufs/vartmp.


[jeff@port10 ~]$ gpart show
=> 34 312581740 mirror/gm0 GPT (149G)
34 128 1 freebsd-boot (64K)
162 2097152 2 freebsd-ufs (1.0G)
2097314 8388608 3 freebsd-swap (4.0G)
10485922 23068672 4 freebsd-ufs (11G)
33554594 23068672 5 freebsd-ufs (11G)
56623266 23068672 6 freebsd-ufs (11G)
79691938 44040192 7 freebsd-ufs (21G)
123732130 4194304 8 freebsd-ufs (2.0G)
127926434 184655340 - free - (88G)

[jeff@port10 ~]$ gmirror list
Geom name: gm0
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 2953586606
Providers:
1. Name: mirror/gm0
Mediasize: 160041885184 (149G)
Sectorsize: 512
Mode: r6w6e11
Consumers:
1. Name: ad0
Mediasize: 160041885696 (149G)
Sectorsize: 512
Mode: r1w1e1
State: ACTIVE
Priority: 0
Flags: NONE
GenID: 0
SyncID: 1
ID: 2910921724

[jeff@port10 ~]$ gjournal list
Geom name: gjournal 1515293619
ID: 1515293619
Providers:
1. Name: mirror/gm0p4.journal
Mediasize: 10737417728 (10G)
Sectorsize: 512
Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p4
Mediasize: 11811160064 (11G)
Sectorsize: 512
Mode: r1w1e1
Jend: 11811159552
Jstart: 10737417728
Role: Data,Journal

Geom name: gjournal 3992286814
ID: 3992286814
Providers:
1. Name: mirror/gm0p5.journal
Mediasize: 10737417728 (10G)
Sectorsize: 512
Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p5
Mediasize: 11811160064 (11G)
Sectorsize: 512
Mode: r1w1e1
Jend: 11811159552
Jstart: 10737417728
Role: Data,Journal

Geom name: gjournal 2315895659
ID: 2315895659
Providers:
1. Name: mirror/gm0p6.journal
Mediasize: 10737417728 (10G)
Sectorsize: 512
Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p6
Mediasize: 11811160064 (11G)
Sectorsize: 512
Mode: r1w1e1
Jend: 11811159552
Jstart: 10737417728
Role: Data,Journal

Geom name: gjournal 2119265588
ID: 2119265588
Providers:
1. Name: mirror/gm0p7.journal
Mediasize: 21474835968 (20G)
Sectorsize: 512
Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p7
Mediasize: 22548578304 (21G)
Sectorsize: 512
Mode: r1w1e1
Jend: 22548577792
Jstart: 21474835968
Role: Data,Journal

jef
June 14th, 2009, 06:24
Script used to generate (been fighting a recalcitrant BIOS, so have done this a few times, unfortunately)

#!/bin/sh -x

#
# Builds minimal FreeBSD on gjournal on GPT (via gpart) on gmirror
#

# Jeff Kletsky -- 2009

#
# Thanks to:
#
# http://m8d.de/news/freebsd-on-gpt.php
# http://www.freebsd.org/doc/en/books/handbook/geom-mirror.html
# http://www.freebsd.org/doc/en/books/handbook/geom-gjournal.html
# http://www.freebsd.org/doc/en_US.ISO8859-1/articles/gjournal-desktop/
# http://lists.freebsd.org/pipermail/freebsd-stable/2008-May/042415.html
# http://lists.freebsd.org/pipermail/freebsd-hackers/2007-January/019276.html
#

#
# once mirror is up and running with the second drive, remember to
#
# # gmirror configure -F <mirror_name>
#
# c.f.,
# http://lists.freebsd.org/pipermail/freebsd-hackers/2007-January/019276.html
#

#
# Decision was to mirror entire drive to make it easier when
# adding additional partitions. Some suggestions were read
# that this configuration also makes it if/when one of the
# underlying drives need to be replaced.
#
# Downside is that swap space is mirrored, possibly slowing performance
# As these machines have 2GB and aren't generally doing anything memory
# intensive, it is unlikely that swap will be heavily used
#

#
# Configured for buiding my "jailserver" box, others may be ok as well
#

DEVICE=${DEVICE:-'ad0'}
MIRROR=${MIRROR:-'gm0'}

VERSION=${releaseName:-'7.2-RELEASE'}


#
# Begin from Fixit prompt off DVD
# Expects distribution to be mounted at /dist (default)
#

#
# To do it...
#
# Fixit# mkdir /usbstick
# Fixit# mount_msdosfs /dev/da0s1 /usbstick
# Fixit# sh -x /usbstick/install_script.sh
#
# 'mount -t msdosfs' will not work, apparently due to paths under Fixit
#
# redirection or script command can be used to capture output
#

export DESTDIR='/install_mnt'

#
# Because gpart is rather rough from a user standpoint:
#

M=$((2*1024))
G=$((2*1024*1024))


# Function to return the first free starting sector in the last free space
# (handles free space in among partitions in a "safe" way)

next_free ( ) {
gpart show $1 | fgrep -e '- free -' | tail -n 1 | awk '{ print $1; }'
}


# 'gpart destroy' will not remove a populated GPT table
# Note also that 'gpart -f <args>' is *not* force

remove_partitions ( ) {
for index in `gpart show $1 | fgrep -v '=>' | fgrep -ve '- free -' | \
awk '{ if ( $3 ~ /[0-9]+/ ) print $3 ; }' | sort -n | uniq | \
awk '{ PARTS=PARTS" "$1 ; } END { print PARTS ; }'`
do gpart delete -i $index $1
done
}


# Enable "shoot yourself in the foot mode" for geom
# Probably only needed if you need to get rid of existing labels
# or have mounted one of the partitions this boot

sysctl kern.geom.debugflags=16

# Load the needed kernel modules

kldload /dist/boot/kernel/geom_mirror.ko
kldload /dist/boot/kernel/geom_journal.ko


# Make sure things are unlabeled and "fresh"

gmirror remove ${MIRROR} ${DEVICE}
gmirror clear ${DEVICE}
gmirror forget ${MIRROR} ${DEVICE}
gmirror clear ${MIRROR}


remove_partitions ${DEVICE}
gpart destroy ${DEVICE}

# Just to be safe, though doesn't handle end-of-disk GPT sectors

dd if=/dev/zero of=/dev/${DEVICE} bs=512 count=1k


#
# Start creating the new providers and filesystems
#

# Create the mirror at the device level

gmirror label -vb round-robin ${MIRROR} ${DEVICE}


# Create a new partition table

gpart create -s GPT mirror/${MIRROR}

# Now install the bootcode 1st stage

gpart bootcode -b /dist/boot/pmbr mirror/${MIRROR}


# Boot and root partitions I have read must be first two
# Worth checking on later, but let's get this working

# boot - 1
gpart add -b `next_free mirror/${MIRROR}` -s 128 -t freebsd-boot \
mirror/${MIRROR}

# with the 2nd-stage boot code

gpart bootcode -p /dist/boot/gptboot -i 1 mirror/${MIRROR}

# root - 2
gpart add -b `next_free mirror/${MIRROR}` -s $((1*$G)) -t freebsd-ufs \
mirror/${MIRROR}





# Swap next, so root can expand later if it needs to

# swap - 3
gpart add -b `next_free mirror/${MIRROR}` -s $((4*$G)) -t freebsd-swap \
mirror/${MIRROR}


# Now the rest of the partitions

# var - 4
gpart add -b `next_free mirror/${MIRROR}` -s $((11*$G)) -t freebsd-ufs \
mirror/${MIRROR}

# tmp - 5
gpart add -b `next_free mirror/${MIRROR}` -s $((11*$G)) -t freebsd-ufs \
mirror/${MIRROR}

#usr - 6
gpart add -b `next_free mirror/${MIRROR}` -s $((11*$G)) -t freebsd-ufs \
mirror/${MIRROR}

#var/tmp - 7
gpart add -b `next_free mirror/${MIRROR}` -s $((21*$G)) -t freebsd-ufs \
mirror/${MIRROR}

gpart show mirror/${MIRROR}


# Set up journals
# '-f' needed here in case previously used disk

gjournal label -f mirror/${MIRROR}p4
gjournal label -f mirror/${MIRROR}p5
gjournal label -f mirror/${MIRROR}p6
gjournal label -f mirror/${MIRROR}p7

gjournal show


# newfs and labels

newfs -L root mirror/${MIRROR}p2
glabel create swap mirror/${MIRROR}p3
newfs -JL var mirror/${MIRROR}p4.journal
newfs -JL tmp mirror/${MIRROR}p5.journal
newfs -JL usr mirror/${MIRROR}p6.journal
newfs -JL vartmp mirror/${MIRROR}p7.journal

glabel show


# mount the new filesystem

mkdir ${DESTDIR}
mount /dev/mirror/${MIRROR}p2 ${DESTDIR}

# Could use '-o async' for these

cd ${DESTDIR}
mkdir var tmp usr
mount /dev/mirror/${MIRROR}p4.journal var
mount /dev/mirror/${MIRROR}p5.journal tmp
mount /dev/mirror/${MIRROR}p6.journal usr

mkdir var/tmp
mount /dev/mirror/${MIRROR}p7.journal var/tmp

df -h


# Prep and install a minimal system


# base
cd /dist/${VERSION}/base
./install.sh

# GENERIC
cd ../kernels
./install.sh GENERIC
cd ${DESTDIR}/boot

# also needs to be in /boot/kernel
cp -rp GENERIC/ kernel
diff -r GENERIC kernel

df -h


# Ready system for reboot

# loader.conf

cat >> ${DESTDIR}/boot/loader.conf <<EOF
geom_mirror_load="YES"
geom_journal_load="YES"
EOF

# fstab

cat > ${DESTDIR}/etc/fstab.dev <<EOF
# Device Mountpoint FStype Options Dump Pass#
/dev/mirror/${MIRROR}p3 none swap sw 0 0
/dev/mirror/${MIRROR}p2 / ufs rw 1 1
/dev/mirror/${MIRROR}p5.journal /tmp ufs rw,async 2 2
/dev/mirror/${MIRROR}p6.journal /usr ufs rw,async 2 2
/dev/mirror/${MIRROR}p4.journal /var ufs rw,async 2 2
/dev/mirror/${MIRROR}p7.journal /var/tmp ufs rw,async 2 2
/dev/acd0 /cdrom cd9660 ro,noauto 0 0
EOF

cat > ${DESTDIR}/etc/fstab.label <<EOF
# Device Mountpoint FStype Options Dump Pass#
/dev/label/swap none swap sw 0 0
/dev/label/root / ufs rw 1 1
/dev/label/tmp /tmp ufs rw,async 2 2
/dev/label/usr /usr ufs rw,async 2 2
/dev/label/var /var ufs rw,async 2 2
/dev/label/vartmp /var/tmp ufs rw,async 2 2
/dev/acd0 /cdrom cd9660 ro,noauto 0 0
EOF

# Could probably do this with a symlink,
# but this preserves the original in the case of an edit

cp ${DESTDIR}/etc/fstab.dev ${DESTDIR}/etc/fstab

#
# This would be a good place to chroot and do things like
# setting the root password
#
# c.f., http://m8d.de/news/freebsd-on-gpt.php
#

# THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE FREEBSD PROJECT OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
# BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.