ZFS ZFS lost pool

[AndrewIOM]

Hi,

I had a disk in a zfs pool fail. I ordered a new disk and after I shutdown to replace the disk I have now lost the pool.

# zpool import
   pool: data
     id: 10364801433109347175
  state: UNAVAIL
status: One or more devices contains corrupted data.
 action: The pool cannot be imported due to damaged devices or data.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-5E
 config:

        data                      UNAVAIL  insufficient replicas
          mirror-0                ONLINE
            ada0                  ONLINE
            ada1                  ONLINE
          mirror-1                UNAVAIL  insufficient replicas
            ada2                  UNAVAIL  invalid label
            13378492932401563744  UNAVAIL  invalid label
          mirror-2                ONLINE
            ada3                  ONLINE
            ada4                  ONLINE
          mirror-3                ONLINE
            ada5                  ONLINE

13378492932401563744 is the new disk but with ada2 unavailable, I can't rebuild the pool

Can I recreate the label on ada2?

# zdb -lu /dev/ada2
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3

I can get data from ada0

 Uberblock[31]
        magic = 0000000000bab10c
        version = 5000
        txg = 15153279
        guid_sum = 5751658451015623055
        timestamp = 1676310864 UTC = Mon Feb 13 17:54:24 2023
        mmp_magic = 00000000a11cea11
        mmp_delay = 0
        mmp_valid = 0
        checkpoint_txg = 0
        labels = 0 1 2 3

Is recovery possible or am I SOL?


Thanks
Andrew

 

[AndrewIOM]

Hi,

I plugged the failed disk into a linux system and dmesg shows

[332102.960727] usb 2-4.4: new SuperSpeed USB device number 9 using xhci_hcd
[332102.980276] usb 2-4.4: New USB device found, idVendor=152d, idProduct=0583, bcdDevice=91.28
[332102.980284] usb 2-4.4: New USB device strings: Mfr=1, Product=2, SerialNumber=5
[332102.980286] usb 2-4.4: Product: JMS56x Series
[332102.980288] usb 2-4.4: Manufacturer: JMicron
[332102.980290] usb 2-4.4: SerialNumber: DD56419883897
[332103.034842] scsi host15: uas
[332103.035452] scsi 15:0:0:0: Direct-Access     WDC WD60 EFRX-68L0BN1     9128 PQ: 0 ANSI: 6
[332103.038099] sd 15:0:0:0: Attached scsi generic sg4 type 0
[332103.038438] sd 15:0:0:0: [sdd] 11721045168 512-byte logical blocks: (6.00 TB/5.46 TiB)
[332103.038632] sd 15:0:0:0: [sdd] Write Protect is off
[332103.038634] sd 15:0:0:0: [sdd] Mode Sense: 67 00 10 08
[332103.039014] sd 15:0:0:0: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[332103.039503] sd 15:0:0:0: [sdd] Preferred minimum I/O size 4096 bytes
[332103.039506] sd 15:0:0:0: [sdd] Optimal transfer size 33553920 bytes not a multiple of preferred minimum block size (4096 bytes)
[332108.454267] sd 15:0:0:0: [sdd] Attached SCSI disk
[332118.811576] sd 15:0:0:0: [sdd] tag#8 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=7s
[332118.811598] sd 15:0:0:0: [sdd] tag#8 Sense Key : Medium Error [current]
[332118.811602] sd 15:0:0:0: [sdd] tag#8 Add. Sense: Unrecovered read error
[332118.811605] sd 15:0:0:0: [sdd] tag#8 CDB: Read(16) 88 00 00 00 00 02 ba a0 f4 00 00 00 00 08 00 00
[332118.811608] critical medium error, dev sdd, sector 11721044992 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 2
[332123.044022] sd 15:0:0:0: [sdd] tag#9 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=4s
[332123.044040] sd 15:0:0:0: [sdd] tag#9 Sense Key : Medium Error [current]
[332123.044044] sd 15:0:0:0: [sdd] tag#9 Add. Sense: Unrecovered read error
[332123.044048] sd 15:0:0:0: [sdd] tag#9 CDB: Read(16) 88 00 00 00 00 02 ba a0 f4 00 00 00 00 08 00 00
[332123.044050] critical medium error, dev sdd, sector 11721044992 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
[332123.044057] Buffer I/O error on dev sdd, logical block 1465130624, async page read

So I don't think I can copy anything from this disk

Thanks
Andrew