ZFS zpool destroy failing

D

dove

I have an old pool that I created for testing. The underlying devices are long gone or have been overwritten. It was created a long time in the past - I'm not even sure if they were files, mfs or real disks.

When I do ' zpool import, I see:

Code: 
% sudo zpool import
pool: zmirrortmp
     id: 16645399895977040150
  state: UNAVAIL
 status: One or more devices are missing from the system.
 action: The pool cannot be imported. Attach the missing
        devices and try again.
   see: http://illumos.org/msg/ZFS-8000-3C
 config:

        zmirrortmp               UNAVAIL  insufficient replicas
          mirror-0               UNAVAIL  insufficient replicas
            7933931077065391454  UNAVAIL  cannot open
            3870017713620253843  UNAVAIL  cannot open

Trying to destroy this old pool fails:

Code: 
% sudo zpool destroy -f zmirrortmp
cannot open 'zmirrortmp': no such pool

How can I destory this pool? Note that I have other active pools.
 
Zero out the first and last couple of sectors of the disk. The pool information is stored there and that's what you're seeing with an import.
 
Of which disk?

The real question I'd like to ask is: Where does ZFS store the metadata that says that a pool exists? I thought it was stored on all the disks in that pool, so if at least one of those disks is found, ZFS knows what other disks it needs to find. But in this case, it seems that ZFS knows that a pool exists, even though none of the disks in that pool seem to be accessible. How did it figure that out?

Maybe a kind soul could point at a "theory of ops" for ZFS pool discovery and metadata, if that's written down anywhere. And if it isn't, maybe typing in a paragraph or two would be nice.
 
SirDice said:
Zero out the first and last couple of sectors of the disk. The pool information is stored there and that's what you're seeing with an import.
Click to expand...

Echoing ralphbsz... which disk? I would like to know for certain what metadata to clear and where it exists rather than guess.

As I said, I have no idea regarding the composition of the underlying devices. Could have been a file. Could have been a full disk. Could have been a partition of a disk. Or maybe an MFS and a disk or anything that I was using to test at the time. I assume it must be in the metadata on one of the existing disks, but they are all in use in a different pool now. They may have been partitioned differently. My current pool uses gpt partitions that are almost the full disk (minus about 100M on some of the disks). So I suspect that the bogus metadata is on unused space on an existing disk, but I need to figure out where zpool import' is finding it. And I need to be careful to avoid clearing out any metadata in use for the active pool.

Maybe there's a zdb incantation that will point me to the metadata for the old (now long gone) pool that 'zpool import' is finding.
 
For what it's worth...
Code: 
% sudo zdb zmirrortmp
zdb: can't open 'zmirrortmp': No such file or directory
 
Is there anything listed in the cache file? Maybe rename the cache file and check the import output?

Edit: Cache file is located at /boot/zfs/zpool.cache It's not a text file, so you can't edit it directly, but you can rename it, and it (should|will?) be created the next time you import a pool. You can also try setting -o cachefile=none as part of the import command.
 
Last edited:
If you want the nitty-gritty details, there's a draft of the ZFS On-Disk Specification available online. But, basically, the zpool metadata is stored in the first and last 512KiB of the vdev. There's really only one 256KiB label, but you get four copies of it for redundancy. So, as you suspect, there might be one lurking around somewhere on the disk if phoenix isn't correct about all of this actually coming from a stale zpool.cache.

The good news is that finding where an old label might be on your disk is actually pretty simple, since the name-value pair metadata--which you can view on a healthy pool using zdb(8)--is stored as ASCII strings and ints (probably unsigned ints?). So all you need to look for is the ghost zpool's GUID in the contents of your disk; it should follow a "guid" string. The tricky part is calculating what byte offset to delete from once you've found your label, but I believe you only need to wipe the human-readable part of it for it to no longer register.

Anway, it's been a long time since I messed with this stuff, so take everything I just wrote with a grain of salt (i.e., all the usual disclaimers apply).
 
You can't 'just' import a broken pool, be sure to use the right parameters. For example:

# zpool import -Df. Optionally (not sure from mind) you may need -m too. And well, I'd also recommend using -N because I doubt you'll be interested in the data.

This should be enough to access & destroy the pool. When worse comes to worst you can always look into the labelclear option.
 
You must log in or register to reply here.
Back
Top