ZFS Problem when moving a ZFS pool disk from an ada to a da device

D

Dre

I have a problem when physically changing a ZFS pool disk from an ada to a da device.
ZFS doesn't detect the disk any more.


The pool was created like this

$ gpart create -s GPT ada1
ada1 created
$ gpart show ada1
=> 34 7814037101 ada1 GPT (3.7T)
34 7814037101 - free - (3.7T)
$ gpart add -b 2048 -s 7813627501 -t freebsd-zfs -l disk10 ada1
ada1p1 added
$ gpart show ada1
=> 34 7814037101 ada1 GPT (3.7T)
34 2014 - free - (1M)
2048 7813627501 1 freebsd-zfs (3.7T)
7813629549 407586 - free - (199M)

disk11 and disk12 created the same way

$ zpool create tank2 raidz gpt/disk10 gpt/disk11 gpt/disk12

If I then export the pool and check zpool import everything looks fine
$ zpool import
pool: tank2
id: 5667179852108998157
state: ONLINE
status: One or more devices were configured to use a non-native block size.
Expect reduced performance.
action: The pool can be imported using its name or numeric identifier.
config:

tank2 ONLINE
raidz1-0 ONLINE
gpt/disk10 ONLINE
gpt/disk11 ONLINE
gpt/disk12 ONLINE



$ gpart show -l
=> 34 7814037101 ada1 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk10 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 ada2 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk11 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 ada3 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk12 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 diskid/DISK-WD-WMC1F0784422 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk10 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 diskid/DISK-WD-WMC1F0712526 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk11 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 diskid/DISK-WD-WMC1F0634396 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk12 (3.6T)
7813629549 407586 - free - (199M)




If I then move disk11 to a da device instead

$ zpool import
pool: tank2
id: 5667179852108998157
state: DEGRADED
status: One or more devices are missing from the system.
action: The pool can be imported despite missing or damaged devices. The
fault tolerance of the pool may be compromised if imported.
see: http://illumos.org/msg/ZFS-8000-2Q
config:

tank2 DEGRADED
raidz1-0 DEGRADED
gpt/disk10 ONLINE
16291655659778260829 UNAVAIL cannot open
gpt/disk12 ONLINE

the disk does not show up when checking for GPTs

$ gpart show -l
=> 34 7814037101 ada1 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk10 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 ada3 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk12 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 diskid/DISK-WD-WMC1F0784422 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk10 (3.6T)
7813629549 407586 - free - (199M)

=> 34 7814037101 diskid/DISK-WD-WMC1F0634396 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 disk12 (3.6T)
7813629549 407586 - free - (199M)


Did I screw something up when creating this?

I have some disks that are not created with gpart but just have a glable. These are not a problem to move around between ada and da devices.
 
I should have changed the title of the thread to something better but it is what it is. Don't think I can change that now.

The lable no longer shows up in /dev/gpt/ when moving the disk from an ada device to an da device. Question is why...
 
Hello,

Could you please provide the output of:

1. camcontrol devlist
2. diskinfo -v /dev/daX
3. gpart show /dev/daX

Where /dev/daX is the device you have moved your disk to.

Thanks
 
Ahh.. diskinfo. Forgot about that one.

$ camcontrol devlist
<ATA WDC WD4001FAEX-0 1L01> at scbus0 target 9 lun 0 (pass7,da7)
<WDC WD4001FAEX-00MJRA0 01.01L01> at scbus5 target 0 lun 0 (ada1,pass9)

$ diskinfo -v /dev/da7
/dev/da7
512 # sectorsize
2199023254528 # mediasize in bytes (2.0T)
4294967294 # mediasize in sectors
0 # stripesize
0 # stripeoffset
267349 # Cylinders according to firmware.
255 # Heads according to firmware.
63 # Sectors according to firmware.
WD-WMC1F0712526 # Disk ident.
Not_Zoned # Zone Mode

ada1 is the same typ of disk as da7.

$ diskinfo -v /dev/ada1
/dev/ada1
512 # sectorsize
4000787030016 # mediasize in bytes (3.6T)
7814037168 # mediasize in sectors
4096 # stripesize
0 # stripeoffset
7752021 # Cylinders according to firmware.
16 # Heads according to firmware.
63 # Sectors according to firmware.
WD-WMC1F0784422 # Disk ident.
Not_Zoned # Zone Mode

$ gpart show /dev/da7
gpart: No such geom: /dev/da7

$ gpart show /dev/ada1
=> 34 7814037101 ada1 GPT (3.6T)
34 2014 - free - (1.0M)
2048 7813627501 1 freebsd-zfs (3.6T)
7813629549 407586 - free - (199M)

The disk is apparently not detected/reported correctly.
Can this be some kind of limitation with my expander board?
I'm using a LSI SAS3081E-R PCI.
Never had disks larger than 2TB attached to the PCI card before.
 
Looks like it is an old gen SAS-1 controller on the LSI SAS3081E-R board that doesn't support larger disks than 2 TB.
 
You must log in or register to reply here.
Back
Top