Solved Invalid partition table error

The size of the /boot/encryption.key is 4096 bytes. I've backed it up.

Output of zpool status:

20211226_135950_HDR.jpg


Regarding the order in which I could see with the partitions with gpart it was:

1. bootpool (2G)
2. main zpool (zroot)
3. swap partition (2G)
 
covacat said:
afaik the geli keys or on last sectors on the encrypted block device (disk,partition) and they are decrypted at run time with the passphprase
so if you dump the metadata, you can recover the size of the geli provider and by simple arithmetic deduce its start
then create a label and attach it
look in dmesg for the number of sectors of ada0
subtract 32 of that number and let the result be N
dd if=/dev/ada0 iseek=N of=somefile.out bs=4k
hexdump -Cv somefile.out and look for the geli magic string GEOM::ELI
at offset 30 you have the size of the provider (8 bytes little endian) in bytes
convert it to sectors
subtract it from ada0s1 size and you get its original offset, so you can recreate the label
Click to expand...

  1. Look in dmesg for the number of sectors of ada0:
    Code: 
    ada0: 476490MB (976773168 512 byte sectors)
  2. Subtract 32 of that number and let the result be N
    Code: 
     976773136
  3. dd if=/dev/ada0 iseek=976773136 of=somefile.out bs=4k
    Code: 
    dd: /dev/ada0: Input/output error
 
That does sound reasonable, you installed the FreeBSD 9 in 2015 most likely too. This disk has 512 sector size (you are legacy booting MBR layout). Also dmesg from the above says so. Use bs=512 (which is default) when using dd on your disk.

I'm not available for few hours, I'll check back later. You can use dd if=/dev/diskid/DISK-J3310081H173EBs1 bs=512 count=2|hd to get that start of the partition. Also you can use zdb -l /dev/diskid/DISK-J3310081H173EBs1 to get the exact size of the partition. From top of my head I don't know if you can find the LBA of the partition from zdb or something similar ; if not you can search for it as mentioned above. I also wonder if /boot/zfs/zpool.cache can provide some information.
 
_martin output of zdb -l /dev/diskid/DISK-J3310081H173EBs1 is:

Code: 
------------------------------------
LABEL 0
------------------------------------
    version: 5000
    name: 'bootpool'
    state: 0
    txg: 28240385
    pool_guid: 17749244303695315483
    errata: 0
    hostname: ''
    top_guid: 5973502406743286860
    guid: 5973502406743286860
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 5973502406743286860
        path: '/dev/diskid/DISK-J3310081H173EBs1'
        phys_path: '/dev/ada0s1a'
        whole_disk: 1
        metaslab_array: 34
        metaslab_shift: 24
        ashift: 12
        asize: 2142765056
        is_log: 0
        DTL: 162
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
    labels = 0 1 2 3

Going further I arrived at # dd if=/dev/diskid/DISK-J3310081H173EBs1 bs=512 skip=976773080 | hd | less and this is the result:

dd_dump.jpg


Which, is what I think covacat suggested to look for in one of previous messages. Now trying to understand how and what to calculate.

Additionally, here's /boot/zfs/zpool.cache
 

Attachments

potzilov said:
Regarding the order in which I could see with the partitions with gpart it was:

1. bootpool (2G)
2. main zpool (zroot)
3. swap partition (2G)
Click to expand...
Then ignore my previous post regarding manual creation of bsdlabel as it was based on your first post (2G Bootpool, 2G swap, rest)
 
According to your zpool.cache file your zroot is on /dev/ada0s1d which means that the correct order of your partitions is

ada0s1a:bootpool
ada0s1b:swap
ada0s1d:zroot

so you can try to recreate the bsdlabel.
 
Code: 
bc
>>> ibase=16
>>> 72F0BFC000/200
964190176
>>>
from data in GEOM::ELI meta data block size of the provider is 964190176 sectors
 
I spoke with potzilov on chat and did determine few things. With the commands I pasted above we confirmed the start of the bootpool partition. We assumed rpool starts at LBA 12582976 which is 6G offset from the start of the disk. So either swap was bigger than 2G or there was some free space there. Or I did some mistake. But judging from the found offset to geom::eli and the size it does fit.

I suggested to do a full backup of the disk by dd just in case. It's a pitty qemu-nbd is not fully supported on FreeBSD (we could have created dsf /dev/ndb0 with the proper offset to a disk and geli attach on it). I'm testing if gnop can be used but it seems it has problems finding proper metadata with that.
Once backup is done bsdlabel can be recreated.
 
gnop -o offset should work
the offset should be in bytes, not sectors,
also my calculation gives me
12582912 which is exactly 6G , 0xC00000
 
I tried that, even with specifying size with -s. But if I specify correct size metadata is not there (in gnop provider). If I do +1(sector) to size I see metadata there but geli complains about the size mismatch.
 
bootpool starts at sector 64, verified with the dump (start from ada0 and first sector of the device zfs sees). Above post from potzilov shows the geom:eli at offset 0xe00 when skip=976773080 from bootpool is used. 0xe00/0x200 is 7. To find the end of the rpool from start: 64+7+976773080 = 976773151. 976773152 is the first sector outside of the partition. Size from that metadata is 0x72F0BFC000 which is 964190176 sectors as you verified above. 976773152-964190176 = 12582976 which is how I came to the number.
I did one check in my lab on test disk and the location was the same (i.e. no additional metadata above for partition). But still, 12582976 is ~6GB from start, so 2+2 for bootpool+swap doesn't fit.

But this is where I want to do the qemu-nbd/gnop approach. We have a key from bootpool and OP knows the password - he could easily verify the location of the partition this way without modifying the actual disk.
 
ok, i calculated from the start of ada0s1 which is 64 after the start of the disk, so it is basically the same result
didn't pay attention to his dd command and assumed its the whole disk but it was the partition
 
Ok, it's late where I am but I don't get it. Even history of my commands show I did the same thing ; but now it works. gnop create -o <bytes> -s <partition_size> /dev/provider and then attaching with geli attach /dev/provider.nop works. So this is the way how I'd test it.
 
It seems FreeBSD 13 has problem with this gnop approach. Device was created with gnop create -o 6442483712 -s 493665370112 ada0. Last sector was checked with dd if=/dev/ada0.nop bs=512 count=1 skip=964190175 |hd to match the GEOM::ELI header. All OK. geli was failing though.

Error is pretty vague and unhelpful: Cannot access ada0.nop (error=1). truss output:
Code: 
openat(AT_FDCWD,"/dev/geom.ctl",O_RDONLY,00)     = 3 (0x3)
ioctl(3,GEOM_CTL,0x800a0f0c0)             ERR#22 'Invalid argument'
close(3)                     = 0 (0x0)

We toggled the debug mode sysctl kern.geom.nop.debug=2 but nothing obvious showed up.
I had a suggestion to boot to 12.3 live cd. There everything works as expected. geli attach -k /path/to/key/from/bootpool /dev/ada0.nop did the trick. Apparently pools were updated to newer versions so only ro mode was possible; enough for a full data recovery. Pool was imported with zpool import -o readonly=on -R /tmp/oldsys zroot (sidenote: I just hate that FreeBSD calls rpool zroot)

Once data is synced and safe I suggested potzilov we recreate the bsdlabel. If anything for others who might google this thread.
 
I forgot to mention: since now you have access to the rpool you can check the contents of /var/backups, there might be a backup of the metadata for this disk.

Not needed now but zdb -U /path/to/zpool.cache can be used to check the contents of the cache file. geli made this a bit easier as it saves its size in the metadata (size of the geli provider is bigger than zroot's asize).
 
You must log in or register to reply here.
Back
Top