At some point in time, my pool named
Now, is there any way for the pool to begin using the label without loss of content of L2ARC?
I assume that either of these approaches will result in loss:
I wondered whether stopping zfsd(8) would make a difference, it did not:
Near the end of terminal session below, I discovered then destroyed a label:
– after which, the expected label was reported (by
I have no memory of applying
forums.freebsd.org
From postscripts to <https://forums.freebsd.org/posts/547868>:
– apparently harmless, because a subsequent run of
Transcend began using a device reference instead of the previously specified label – gpt/cache-transcend – for its one cache device.Question
Now, is there any way for the pool to begin using the label without loss of content of L2ARC?
I assume that either of these approaches will result in loss:
- zpool-replace(8)
- zpool-remove(8) followed by zpool-add(8).
Notes
Code:
root@mowa219-gjp4-8570p-freebsd:~ # zpool online Transcend /dev/da3p1
root@mowa219-gjp4-8570p-freebsd:~ # glabel list | grep da3
root@mowa219-gjp4-8570p-freebsd:~ # lsblk da3
DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT
da3 2:77 14G GPT - -
<FREE> -:- 1.0M - - -
da3p1 2:78 14G freebsd-zfs gpt/cache-transcend <ZFS>
root@mowa219-gjp4-8570p-freebsd:~ # zpool history Transcend | grep cache-transcend | grep -v -e offline -e online
2021-07-25.13:03:00 zpool add Transcend cache gpt/cache-transcend
2021-08-29.05:27:16 zpool add Transcend cache gpt/cache-transcend
root@mowa219-gjp4-8570p-freebsd:~ # gpart show -l da3
=> 34 30310333 da3 GPT (14G)
34 2014 - free - (1.0M)
2048 30308319 1 cache-transcend (14G)
root@mowa219-gjp4-8570p-freebsd:~ # zdb -l /dev/da3p1
------------------------------------
LABEL 0
------------------------------------
version: 5000
state: 4
guid: 1869112229494633056
labels = 0 1 2 3
------------------------------------
L2ARC device header
------------------------------------
magic: 6504978260106102853
version: 1
pool_guid: 8076233369858608335
flags: 0
start_lbps[0]: 5259558912
start_lbps[1]: 5246668800
log_blk_ent: 924
start: 4198400
end: 15517089792
evict: 5280088064
lb_asize_refcount: 4235264
lb_count_refcount: 241
trim_action_time: 0
trim_state: 0
------------------------------------
L2ARC device log blocks
------------------------------------
log_blk_count: 241 with valid cksum
0 with invalid cksum
log_blk_asize: 4235264
root@mowa219-gjp4-8570p-freebsd:~ # zpool history Transcend | grep -e add -e remove
2021-04-19.06:15:22 zpool add Transcend cache da2
2021-07-24.18:11:47 zpool remove Transcend da0
2021-07-25.13:03:00 zpool add Transcend cache gpt/cache-transcend
2021-08-29.05:27:16 zpool add Transcend cache gpt/cache-transcend
root@mowa219-gjp4-8570p-freebsd:~ # zpool offline Transcend /dev/da3p1
root@mowa219-gjp4-8570p-freebsd:~ # tail -f -n 0 /var/log/messages
Apr 17 08:09:11 mowa219-gjp4-8570p-freebsd kernel: ugen0.7: <vendor 0x0951 product 0x1666> at usbus0 (disconnected)
Apr 17 08:09:11 mowa219-gjp4-8570p-freebsd kernel: umass3: at uhub1, port 7, addr 7 (disconnected)
Apr 17 08:09:11 mowa219-gjp4-8570p-freebsd kernel: da3 at umass-sim3 bus 3 scbus6 target 0 lun 0
Apr 17 08:09:11 mowa219-gjp4-8570p-freebsd kernel: da3: <Kingston DataTraveler 3.0 1.00> s/n 60A44C3FACC93110B9970045 detached
Apr 17 08:09:14 mowa219-gjp4-8570p-freebsd kernel: (da3:umass-sim3:3:0:0): Periph destroyed
Apr 17 08:09:14 mowa219-gjp4-8570p-freebsd kernel: umass3: detached
Apr 17 08:09:14 mowa219-gjp4-8570p-freebsd ZFS[32006]: vdev state changed, pool_guid=8076233369858608335 vdev_guid=1869112229494633056
Apr 17 08:09:14 mowa219-gjp4-8570p-freebsd ZFS[32010]: vdev is removed, pool_guid=8076233369858608335 vdev_guid=1869112229494633056
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: ugen0.7: <vendor 0x0951 product 0x1666> at usbus0
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: umass3 on uhub1
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: umass3: <vendor 0x0951 product 0x1666, class 0/0, rev 3.10/11.76, addr 9> on usbus0
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: da3 at umass-sim3 bus 3 scbus6 target 0 lun 0
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: da3: <Kingston DataTraveler 3.0 1.00> Removable Direct Access SPC-4 SCSI device
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: da3: Serial Number 60A44C3FACC93110B9970045
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: da3: 400.000MB/s transfers
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: da3: 14800MB (30310400 512 byte sectors)
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd kernel: da3: quirks=0x2<NO_6_BYTE>
Apr 17 08:09:16 mowa219-gjp4-8570p-freebsd ZFS[32017]: vdev state changed, pool_guid=8076233369858608335 vdev_guid=1869112229494633056
^C
root@mowa219-gjp4-8570p-freebsd:~ # gpart show -l da3
=> 34 30310333 da3 GPT (14G)
34 2014 - free - (1.0M)
2048 30308319 1 cache-transcend (14G)
root@mowa219-gjp4-8570p-freebsd:~ # zpool status -v Transcend
pool: Transcend
state: ONLINE
status: Some supported and requested features are not enabled on the pool.
The pool can still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: scrub repaired 0B in 01:32:30 with 0 errors on Sun Dec 12 16:25:35 2021
config:
NAME STATE READ WRITE CKSUM
Transcend ONLINE 0 0 0
gpt/Transcend ONLINE 0 0 0
cache
da3p1 ONLINE 0 0 0
errors: No known data errors
root@mowa219-gjp4-8570p-freebsd:~ #I wondered whether stopping zfsd(8) would make a difference, it did not:
Code:
root@mowa219-gjp4-8570p-freebsd:~ # service zfsd stop
Stopping zfsd.
Waiting for PIDS: 8037.
root@mowa219-gjp4-8570p-freebsd:~ # zpool offline Transcend /dev/da3p1
root@mowa219-gjp4-8570p-freebsd:~ # tail -f -n 0 /var/log/messages
Apr 17 08:10:52 mowa219-gjp4-8570p-freebsd kernel: ugen0.7: <vendor 0x0951 product 0x1666> at usbus0 (disconnected)
Apr 17 08:10:52 mowa219-gjp4-8570p-freebsd kernel: umass3: at uhub1, port 7, addr 9 (disconnected)
Apr 17 08:10:52 mowa219-gjp4-8570p-freebsd kernel: da3 at umass-sim3 bus 3 scbus6 target 0 lun 0
Apr 17 08:10:52 mowa219-gjp4-8570p-freebsd kernel: da3: <Kingston DataTraveler 3.0 1.00> s/n 60A44C3FACC93110B9970045 detached
Apr 17 08:10:52 mowa219-gjp4-8570p-freebsd kernel: (da3:umass-sim3:3:0:0): Periph destroyed
Apr 17 08:10:52 mowa219-gjp4-8570p-freebsd kernel: umass3: detached
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: ugen0.7: <Kingston DataTraveler 3.0> at usbus0
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: umass3 on uhub1
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: umass3: <Kingston DataTraveler 3.0, class 0/0, rev 3.10/11.76, addr 10> on usbus0
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: da3 at umass-sim3 bus 3 scbus6 target 0 lun 0
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: da3: <Kingston DataTraveler 3.0 1.00> Removable Direct Access SPC-4 SCSI device
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: da3: Serial Number 60A44C3FACC93110B9970045
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: da3: 400.000MB/s transfers
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: da3: 14800MB (30310400 512 byte sectors)
Apr 17 08:10:57 mowa219-gjp4-8570p-freebsd kernel: da3: quirks=0x2<NO_6_BYTE>
^C
root@mowa219-gjp4-8570p-freebsd:~ # zpool online Transcend gpt/cache-transcend
cannot online gpt/cache-transcend: no such device in pool
root@mowa219-gjp4-8570p-freebsd:~ # zpool export Transcend
root@mowa219-gjp4-8570p-freebsd:~ # zpool import Transcend ; zpool status Transcend && zfs load-key Transcend/VirtualBox && zfs mount Transcend/VirtualBox ; mount | grep Transcend ; sleep 5 ; zpool iostat -v
pool: Transcend
state: ONLINE
status: One or more devices has been taken offline by the administrator.
Sufficient replicas exist for the pool to continue functioning in a
degraded state.
action: Online the device using 'zpool online' or replace the device with
'zpool replace'.
scan: scrub repaired 0B in 01:32:30 with 0 errors on Sun Dec 12 16:25:35 2021
config:
NAME STATE READ WRITE CKSUM
Transcend ONLINE 0 0 0
gpt/Transcend ONLINE 0 0 0
cache
da3p1 OFFLINE 0 0 0
errors: No known data errors
Transcend on /media/t500 (zfs, local, nfsv4acls)
Transcend/VirtualBox on /media/t500/VirtualBox (zfs, local, nfsv4acls)
capacity operations bandwidth
pool alloc free read write read write
------------------ ----- ----- ----- ----- ----- -----
Transcend 214G 250G 22 25 794K 311K
gpt/Transcend 214G 250G 22 25 794K 311K
cache - - - - - -
da3p1 - - 0 0 0 0
------------------ ----- ----- ----- ----- ----- -----
august 703G 209G 5 20 204K 926K
ada0p3.eli 703G 209G 5 20 204K 926K
cache - - - - - -
gpt/duracell 14.3G 1.12G 3 0 107K 154K
gpt/cache-august 21.9G 6.95G 7 0 173K 165K
------------------ ----- ----- ----- ----- ----- -----
root@mowa219-gjp4-8570p-freebsd:~ # service zfsd start
Starting zfsd.
root@mowa219-gjp4-8570p-freebsd:~ #Additional background
Near the end of terminal session below, I discovered then destroyed a label:
label/demo– after which, the expected label was reported (by
lsblk):gpt/cache-transcend
Code:
root@mowa219-gjp4-8570p-freebsd:~ # glabel list
Geom name: ada0p1
Providers:
1. Name: gpt/efiboot0
Mediasize: 272629760 (260M)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r0w0e0
secoffset: 0
offset: 0
seclength: 532480
length: 272629760
index: 0
Consumers:
1. Name: ada0p1
Mediasize: 272629760 (260M)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r0w0e0
Geom name: da0p1
Providers:
1. Name: gpt/duracell
Mediasize: 16566435328 (15G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1048576
Mode: r1w1e1
secoffset: 0
offset: 0
seclength: 32356319
length: 16566435328
index: 0
Consumers:
1. Name: da0p1
Mediasize: 16566435328 (15G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1048576
Mode: r1w1e2
Geom name: da1p1
Providers:
1. Name: gpt/Transcend
Mediasize: 500106796544 (466G)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e1
secoffset: 0
offset: 0
seclength: 976771087
length: 500106796544
index: 0
Consumers:
1. Name: da1p1
Mediasize: 500106796544 (466G)
Sectorsize: 512
Stripesize: 4096
Stripeoffset: 0
Mode: r1w1e2
Geom name: da2p1
Providers:
1. Name: gpt/cache-august
Mediasize: 30942930432 (29G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1048576
Mode: r1w1e1
secoffset: 0
offset: 0
seclength: 60435411
length: 30942930432
index: 0
Consumers:
1. Name: da2p1
Mediasize: 30942930432 (29G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1048576
Mode: r1w1e2
Geom name: da3p1
Providers:
1. Name: label/demo
Mediasize: 15517858816 (14G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1048576
Mode: r0w0e0
secoffset: 0
offset: 0
seclength: 30308318
length: 15517858816
index: 0
Consumers:
1. Name: da3p1
Mediasize: 15517859328 (14G)
Sectorsize: 512
Stripesize: 0
Stripeoffset: 1048576
Mode: r0w0e0
root@mowa219-gjp4-8570p-freebsd:~ # lsblk
DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT
ada0 0:119 932G GPT - -
ada0p1 0:121 260M efi gpt/efiboot0 -
<FREE> -:- 1.0M - - -
ada0p2 0:123 16G freebsd-swap gpt/swap0 SWAP
ada0p2.eli 2:62 16G freebsd-swap - SWAP
ada0p3 0:125 915G freebsd-zfs gpt/zfs0 <ZFS>
ada0p3.eli 0:131 915G zfs - -
<FREE> -:- 708K - - -
da0 0:147 15G GPT - -
<FREE> -:- 1.0M - - -
da0p1 0:148 15G freebsd-zfs gpt/duracell <ZFS>
da1 0:177 466G GPT - -
<FREE> -:- 1.0M - - -
da1p1 0:178 466G freebsd-zfs gpt/Transcend <ZFS>
da2 0:185 29G GPT - -
<FREE> -:- 1.0M - - -
da2p1 0:186 29G freebsd-zfs gpt/cache-august <ZFS>
da3 2:77 14G GPT - -
<FREE> -:- 1.0M - - -
da3p1 2:78 14G freebsd-zfs label/demo <ZFS>
root@mowa219-gjp4-8570p-freebsd:~ # glabel destroy label/demo
root@mowa219-gjp4-8570p-freebsd:~ # lsblk da3
DEVICE MAJ:MIN SIZE TYPE LABEL MOUNT
da3 2:77 14G GPT - -
<FREE> -:- 1.0M - - -
da3p1 2:78 14G freebsd-zfs gpt/cache-transcend <ZFS>
root@mowa219-gjp4-8570p-freebsd:~ # zpool offline Transcend /dev/da3p1
root@mowa219-gjp4-8570p-freebsd:~ # zpool online Transcend gpt/cache-transcend
cannot online gpt/cache-transcend: no such device in pool
root@mowa219-gjp4-8570p-freebsd:~ #I have no memory of applying
label/demo, I see that it was unexpectedly found for the same cache device in early August 2021:
Other - Two labels for a single partition
Below, how can a single partition have two labels? The unwanted LABEL label/demo appears for da1p1 only after the device (a ZFS cache device) is physically: disconnected reconnected without being taken online. root@mowa219-gjp4-zbook-freebsd:~ # geom -t Geom Class...
From postscripts to <https://forums.freebsd.org/posts/547868>:
– apparently harmless, because a subsequent run of
zpool iostat -v found the pool still using the expected label:gpt/cache-transcend