GEOM: adX: the secondary GPT table is corrupt or invalid

Hi all,

7.1-R, RAID1 on 4 ST31500341AS via a Promise TX4-300.

The RAID array is purly userland data ... Nothing for the OS.
The OS is on it's own disks ad0 & ad1

I'm not exactly sure when it happened, I started out with 6.2-R, Then fresh Installed 6.4-R, Then fresh Installed 7.0-R, Then fresh Installed 7.1-R.

Yes, I've done alot of different configs - I'm Educating!

Sometime during all this I started getting the following error on the RAID disks:

GEOM: adX: the secondary GPT table is corrupt or invalid.
GEOM: adX: using the primary only -- recovery suggested.
Across all 4 disks of the RAID (ad4,ad6,ad8,ad10).
The OS disks are fine.

After some code research I find an undocumented "recover" command that recovers GPT tables.
gpt recover adX​
Clears the error though damages the mirror.
gmirror label Data adX​
Fixes the mirror but brings back the original error.

Is there a way out of this without losing the data, Yea, I have a BUP but it is many many hours to restore. (over 2TB)

Thanks

-Enjoy
fh : )_~
 
A) Did I
  1. say something wrong?
  2. say something stupid?
B) Am I
  1. missing the obvious?
  2. not providing enough info?

Or is it just that nobody has an answer?

Thanks

-Enjoy
fh : )_~
 
I have tha same problem on freebds 7.1 stable

Feb 11 07:29:12 kernel: GEOM_LABEL: Label ufs/data removed.
Feb 11 07:29:12 kernel: GEOM_LABEL: Label for provider ad8s2 is ufs/data.
Feb 11 07:29:12 kernel: GEOM_LABEL: Label ufs/data removed.
Feb 11 07:29:12 kernel: GEOM_MIRROR: Device mirror/RAID1 launched (2/2).
Feb 11 07:29:12 kernel: Trying to mount root from ufs:/dev/md0
Feb 11 07:29:12 kernel: GEOM_LABEL: Label for provider ad8s2 is ufs/data.
Feb 11 07:29:12 kernel: GEOM: ad4: using the primary only -- recovery suggested.
Feb 11 07:29:12 kernel: GEOM: ad4: the secondary GPT table is corrupt or invalid.
Feb 11 07:29:12 kernel: ad8: 76319MB <WDC WD800JD-00LSA0 06.01D06> at ata4-master SATA300
Feb 11 07:29:12 kernel: ad4: 305245MB <Seagate ST3320620A 3.AAF> at ata2-master UDMA100
Feb 11 07:29:12 kernel: GEOM: ad0: using the primary only -- recovery suggested.
Feb 11 07:29:12 kernel: GEOM: ad0: the secondary GPT table is corrupt or invalid.
Feb 11 07:29:12 kernel: ad0: 305245MB <Seagate ST3320620A 3.AAF> at ata0-master UDMA100
Feb 11 07:29:12 kernel: md0: Preloaded image </mfsroot> 83886080 bytes at 0xc0e6c620
 
Same problem

I have the same issues with what appears to be nearly the same setup.

3Ware raid card, dual processor, 7.1R FreeBSd.

I get the same messages. From what I see here there isn't much that can be done and it doesn't seem to impact the drives?

I would ask if the others used gpart or gpt to setup their raids. I used gpart, if everyone else did as well then perhaps the issue is in gpart...

Thanks. Any info would be helpful.
 
gjournal over gmirror over GPT using gpart here

7.2-RELEASE DVD AMD64

Code:
ad0: 152627MB <Hitachi HTS541616J9AT00 SB4OA70H> at ata0-master UDMA100
GEOM: ad0: the secondary GPT table is corrupt or invalid.
GEOM: ad0: using the primary only -- recovery suggested.
GEOM_MIRROR: Device mirror/gm0 launched (1/1).
GEOM_LABEL: Label for provider mirror/gm0p2 is ufsid/4a33bf8b5ac9f68d.
GEOM_LABEL: Label for provider mirror/gm0p2 is ufs/root.
GEOM_JOURNAL: Journal 1515293619: mirror/gm0p4 contains data.
GEOM_JOURNAL: Journal 1515293619: mirror/gm0p4 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p4 clean.
GEOM_JOURNAL: Journal 3992286814: mirror/gm0p5 contains data.
GEOM_JOURNAL: Journal 3992286814: mirror/gm0p5 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p5 clean.
GEOM_JOURNAL: Journal 2315895659: mirror/gm0p6 contains data.
GEOM_JOURNAL: Journal 2315895659: mirror/gm0p6 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p6 clean.
GEOM_JOURNAL: Journal 2119265588: mirror/gm0p7 contains data.
GEOM_JOURNAL: Journal 2119265588: mirror/gm0p7 contains journal.
GEOM_JOURNAL: Journal mirror/gm0p7 clean.
GEOM_LABEL: Label for provider mirror/gm0p8 is ufsid/4a3455146bc1ba31.
GEOM_LABEL: Label for provider mirror/gm0p8 is ufs/portbuilder.
GEOM_LABEL: Label for provider mirror/gm0p4.journal is ufsid/4a33bf8c03e8ba1f.
GEOM_LABEL: Label for provider mirror/gm0p4.journal is ufs/var.
GEOM_LABEL: Label for provider mirror/gm0p5.journal is ufsid/4a33bf8cdbd79c2c.
GEOM_LABEL: Label for provider mirror/gm0p5.journal is ufs/tmp.
GEOM_LABEL: Label for provider mirror/gm0p6.journal is ufsid/4a33bf8cf91b4524.
GEOM_LABEL: Label for provider mirror/gm0p6.journal is ufs/usr.
GEOM_LABEL: Label for provider mirror/gm0p7.journal is ufsid/4a33bf8c45bfed60.
GEOM_LABEL: Label for provider mirror/gm0p7.journal is ufs/vartmp.

Code:
[jeff@port10 ~]$ gpart show
=>       34  312581740  mirror/gm0  GPT  (149G)
         34        128           1  freebsd-boot  (64K)
        162    2097152           2  freebsd-ufs  (1.0G)
    2097314    8388608           3  freebsd-swap  (4.0G)
   10485922   23068672           4  freebsd-ufs  (11G)
   33554594   23068672           5  freebsd-ufs  (11G)
   56623266   23068672           6  freebsd-ufs  (11G)
   79691938   44040192           7  freebsd-ufs  (21G)
  123732130    4194304           8  freebsd-ufs  (2.0G)
  127926434  184655340              - free -  (88G)

[jeff@port10 ~]$ gmirror list
Geom name: gm0
State: COMPLETE
Components: 1
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 2953586606
Providers:
1. Name: mirror/gm0
   Mediasize: 160041885184 (149G)
   Sectorsize: 512
   Mode: r6w6e11
Consumers:
1. Name: ad0
   Mediasize: 160041885696 (149G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 2910921724

[jeff@port10 ~]$ gjournal list
Geom name: gjournal 1515293619
ID: 1515293619
Providers:
1. Name: mirror/gm0p4.journal
   Mediasize: 10737417728 (10G)
   Sectorsize: 512
   Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p4
   Mediasize: 11811160064 (11G)
   Sectorsize: 512
   Mode: r1w1e1
   Jend: 11811159552
   Jstart: 10737417728
   Role: Data,Journal

Geom name: gjournal 3992286814
ID: 3992286814
Providers:
1. Name: mirror/gm0p5.journal
   Mediasize: 10737417728 (10G)
   Sectorsize: 512
   Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p5
   Mediasize: 11811160064 (11G)
   Sectorsize: 512
   Mode: r1w1e1
   Jend: 11811159552
   Jstart: 10737417728
   Role: Data,Journal

Geom name: gjournal 2315895659
ID: 2315895659
Providers:
1. Name: mirror/gm0p6.journal
   Mediasize: 10737417728 (10G)
   Sectorsize: 512
   Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p6
   Mediasize: 11811160064 (11G)
   Sectorsize: 512
   Mode: r1w1e1
   Jend: 11811159552
   Jstart: 10737417728
   Role: Data,Journal

Geom name: gjournal 2119265588
ID: 2119265588
Providers:
1. Name: mirror/gm0p7.journal
   Mediasize: 21474835968 (20G)
   Sectorsize: 512
   Mode: r1w1e1
Consumers:
1. Name: mirror/gm0p7
   Mediasize: 22548578304 (21G)
   Sectorsize: 512
   Mode: r1w1e1
   Jend: 22548577792
   Jstart: 21474835968
   Role: Data,Journal
 
Script used to generate (been fighting a recalcitrant BIOS, so have done this a few times, unfortunately)

Code:
#!/bin/sh -x

#
# Builds minimal FreeBSD on gjournal on GPT (via gpart) on gmirror
#

# Jeff Kletsky -- 2009

#
# Thanks to:
#
# http://m8d.de/news/freebsd-on-gpt.php
# http://www.freebsd.org/doc/en/books/handbook/geom-mirror.html
# http://www.freebsd.org/doc/en/books/handbook/geom-gjournal.html
# http://www.freebsd.org/doc/en_US.ISO8859-1/articles/gjournal-desktop/
# http://lists.freebsd.org/pipermail/freebsd-stable/2008-May/042415.html
# http://lists.freebsd.org/pipermail/freebsd-hackers/2007-January/019276.html
# 

#
# once mirror is up and running with the second drive, remember to
#
#       # gmirror configure -F <mirror_name>
#
# c.f., 
# http://lists.freebsd.org/pipermail/freebsd-hackers/2007-January/019276.html
#

#
# Decision was to mirror entire drive to make it easier when
# adding additional partitions.  Some suggestions were read 
# that this configuration also makes it if/when one of the 
# underlying drives need to be replaced.
#
# Downside is that swap space is mirrored, possibly slowing performance
# As these machines have 2GB and aren't generally doing anything memory
# intensive, it is unlikely that swap will be heavily used
#

#
# Configured for buiding my "jailserver" box, others may be ok as well
#

DEVICE=${DEVICE:-'ad0'}
MIRROR=${MIRROR:-'gm0'}

VERSION=${releaseName:-'7.2-RELEASE'}


#
# Begin from Fixit prompt off DVD
# Expects distribution to be mounted at /dist (default)
#

#
# To do it...
#
#       Fixit# mkdir /usbstick
#       Fixit# mount_msdosfs /dev/da0s1 /usbstick
#       Fixit# sh -x /usbstick/install_script.sh
#
# 'mount -t msdosfs' will not work, apparently due to paths under Fixit
#
# redirection or script command can be used to capture output
# 

export DESTDIR='/install_mnt'

#
# Because gpart is rather rough from a user standpoint:
#

M=$((2*1024))
G=$((2*1024*1024))


# Function to return the first free starting sector in the last free space
# (handles free space in among partitions in a "safe" way)

next_free ( ) {
    gpart show $1 | fgrep -e '- free -' | tail -n 1 | awk '{ print $1; }'
}


# 'gpart destroy' will not remove a populated GPT table
# Note also that 'gpart -f <args>' is *not* force

remove_partitions ( ) {
    for index in `gpart show $1 | fgrep -v '=>' | fgrep -ve '- free -' | \
	awk '{ if ( $3 ~ /[0-9]+/ ) print $3 ; }' | sort -n | uniq | \
	awk '{ PARTS=PARTS" "$1 ; } END { print PARTS ; }'`
    do gpart delete -i $index $1
    done
}


# Enable "shoot yourself in the foot mode" for geom
# Probably only needed if  you need to get rid of existing labels
# or have mounted one of the partitions this boot

sysctl kern.geom.debugflags=16

# Load the needed kernel modules

kldload /dist/boot/kernel/geom_mirror.ko
kldload /dist/boot/kernel/geom_journal.ko


# Make sure things are unlabeled and "fresh"

gmirror remove ${MIRROR} ${DEVICE}
gmirror clear ${DEVICE}
gmirror forget ${MIRROR} ${DEVICE}
gmirror clear ${MIRROR}


remove_partitions ${DEVICE}
gpart destroy ${DEVICE}

# Just to be safe, though doesn't handle end-of-disk GPT sectors

dd if=/dev/zero of=/dev/${DEVICE} bs=512 count=1k


#
# Start creating the new providers and filesystems
#

# Create the mirror at the device level

gmirror label -vb round-robin ${MIRROR} ${DEVICE}


# Create a new partition table

gpart create -s GPT mirror/${MIRROR}

# Now install the bootcode 1st stage

gpart bootcode -b /dist/boot/pmbr mirror/${MIRROR}


# Boot and root partitions I have read must be first two 
# Worth checking on later, but let's get this working

# boot - 1
gpart add -b `next_free mirror/${MIRROR}` -s 128 -t freebsd-boot \
    mirror/${MIRROR}

# with the 2nd-stage boot code

gpart bootcode -p /dist/boot/gptboot -i 1 mirror/${MIRROR}

# root - 2
gpart add -b `next_free mirror/${MIRROR}` -s $((1*$G)) -t freebsd-ufs \
    mirror/${MIRROR}





# Swap next, so root can expand later if it needs to

# swap - 3
gpart add -b `next_free mirror/${MIRROR}` -s $((4*$G)) -t freebsd-swap \
    mirror/${MIRROR}


# Now the rest of the partitions

# var - 4
gpart add -b `next_free mirror/${MIRROR}` -s $((11*$G)) -t freebsd-ufs \
    mirror/${MIRROR}

# tmp - 5
gpart add -b `next_free mirror/${MIRROR}` -s $((11*$G)) -t freebsd-ufs \
    mirror/${MIRROR}

#usr - 6
gpart add -b `next_free mirror/${MIRROR}` -s $((11*$G)) -t freebsd-ufs \
    mirror/${MIRROR}

#var/tmp - 7
gpart add -b `next_free mirror/${MIRROR}` -s $((21*$G)) -t freebsd-ufs \
    mirror/${MIRROR}

gpart show mirror/${MIRROR}


# Set up journals
# '-f' needed here in case previously used disk

gjournal label -f mirror/${MIRROR}p4
gjournal label -f mirror/${MIRROR}p5
gjournal label -f mirror/${MIRROR}p6
gjournal label -f mirror/${MIRROR}p7

gjournal show


# newfs and labels

newfs -L root mirror/${MIRROR}p2
glabel create swap mirror/${MIRROR}p3
newfs -JL var mirror/${MIRROR}p4.journal
newfs -JL tmp mirror/${MIRROR}p5.journal
newfs -JL usr mirror/${MIRROR}p6.journal
newfs -JL vartmp mirror/${MIRROR}p7.journal

glabel show


# mount the new filesystem

mkdir ${DESTDIR}
mount /dev/mirror/${MIRROR}p2 ${DESTDIR}

# Could use '-o async' for these

cd ${DESTDIR}
mkdir var tmp usr
mount /dev/mirror/${MIRROR}p4.journal var
mount /dev/mirror/${MIRROR}p5.journal tmp
mount /dev/mirror/${MIRROR}p6.journal usr

mkdir var/tmp
mount /dev/mirror/${MIRROR}p7.journal var/tmp

df -h


# Prep and install a minimal system


# base
cd /dist/${VERSION}/base
./install.sh

# GENERIC
cd ../kernels
./install.sh GENERIC
cd ${DESTDIR}/boot

# also needs to be in /boot/kernel
cp -rp GENERIC/ kernel
diff -r GENERIC kernel

df -h


# Ready system for reboot

# loader.conf

cat >> ${DESTDIR}/boot/loader.conf <<EOF
geom_mirror_load="YES"
geom_journal_load="YES"
EOF

# fstab

cat > ${DESTDIR}/etc/fstab.dev <<EOF
# Device		Mountpoint	FStype	Options		Dump	Pass#
/dev/mirror/${MIRROR}p3			none		swap	sw		0	0
/dev/mirror/${MIRROR}p2			/		ufs	rw		1	1
/dev/mirror/${MIRROR}p5.journal		/tmp		ufs	rw,async	2	2
/dev/mirror/${MIRROR}p6.journal		/usr		ufs	rw,async	2	2
/dev/mirror/${MIRROR}p4.journal		/var		ufs	rw,async	2	2
/dev/mirror/${MIRROR}p7.journal		/var/tmp	ufs	rw,async	2	2
/dev/acd0				/cdrom		cd9660	ro,noauto	0	0
EOF

cat > ${DESTDIR}/etc/fstab.label <<EOF
# Device		Mountpoint	FStype	Options		Dump	Pass#
/dev/label/swap		none		swap	sw		0	0
/dev/label/root		/		ufs	rw		1	1
/dev/label/tmp		/tmp		ufs	rw,async	2	2
/dev/label/usr		/usr		ufs	rw,async	2	2
/dev/label/var		/var		ufs	rw,async	2	2
/dev/label/vartmp	/var/tmp	ufs	rw,async	2	2
/dev/acd0		/cdrom		cd9660	ro,noauto	0	0
EOF

# Could probably do this with a symlink, 
# but this preserves the original in the case of an edit

cp ${DESTDIR}/etc/fstab.dev ${DESTDIR}/etc/fstab

#
# This would be a good place to chroot and do things like
# setting the root password
#
# c.f., http://m8d.de/news/freebsd-on-gpt.php
#

# THIS SOFTWARE IS PROVIDED BY ``AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL THE FREEBSD PROJECT OR CONTRIBUTORS BE
# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
# BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
# LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
# NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
Back
Top