Solved Provider detached from system: status REMOVED

Got a alert from my system that one of my multipath devices was detached from the system. I am having trouble getting the provider to come back online. Any help with this would be much appreciated.

When I run camcontrol devlist the provider does not show up in the list. It also does not show up in geom disk list

zpool status
Code:
 raidz2-16                    DEGRADED     0     0     0
            5105234823914261161        REMOVED      0     0     0  was /dev/multipath/S1-D39-ZHZ4H1G5
            multipath/S1-D40-ZHZ48VGQ  ONLINE       0     0     0
            multipath/S1-D41-ZHZ4DV99  ONLINE       0     0     0
            multipath/S1-D42-ZHZ4LV64  ONLINE       0     0     0
            multipath/S1-D43-ZHZ1XQWZ  ONLINE       0     0     0
            multipath/S1-D44-ZHZ45JLQ  ONLINE       0     0     0
            multipath/S1-D45-ZHZ13LGS  ONLINE       0     0     0
            multipath/S1-D46-ZHZ4AMD5  ONLINE       0     0     0

Code:
Apr  7 18:46:23 R59DC-NAS-CREATIVE      (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 88 00 00 00 28 00 00 length 20480 SMID 299 terminated ioc 804b loginfo 31170000 scsi 0 state c xfer
0
Apr  7 18:46:23 R59DC-NAS-CREATIVE      (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 80 68 00 00 00 30 00 00 length 24576 SMID 928 terminated ioc 804(da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 88 00 00 00 28 00 00
Apr  7 18:46:23 R59DC-NAS-CREATIVE      (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 00 00 00 00 30 00 00 length 24576 SMID 404 terminated ioc 804b loginfo 31170000 scsi 0 state c xfer
0
Apr  7 18:46:23 R59DC-NAS-CREATIVE      (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7e a8 00 00 00 28 00 00 length 20480 SMID 493 terminated ioc 804b loginfo 31170000 scsi 0 state c xfer
0
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request completed with an error
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 80 68
00 00 00 30 00 00
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request completed with an error
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 00
00 00 00 30 00 00
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request completed with an error
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7e a8
00 00 00 28 00 00
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request completed with an error
Apr  7 18:46:23 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:25 R59DC-NAS-CREATIVE da288 at mpr0 bus 0 scbus13 target 544 lun 0
Apr  7 18:46:25 R59DC-NAS-CREATIVE da288: mpr0: (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 88 00 00 00 28 00 00
GEOM_MULTIPATH: da288 in S1-D39-ZHZ4H1G5 was disconnected
(da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
(da218:mpr0:0:391:0): Retrying command
(da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 80 68 00 00 00 30 00 00
(da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
(da218:mpr0:0:391:0): Retrying command
(da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 00 00 00 00 30 00 00
(da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
(da218:mpr0:0:391:0): Retrying command
(da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7e a8 00 00 00 28 00 00
(da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
(da218:mpr0:0:391:0): Retrying command
(da218:mpr0:0:391:0): CAM status: CCB request completed with an error
(da218:mpr0:0:391:0): Retrying command
(da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7e a8 00 00 00 28 00 00
(da218:mpr0:0:391:0): CAM status: CCB request completed with an error
(da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da288 in S1-D39-ZHZ4H1G5 was disconnected
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 80 68
00 00 00 30 00 00
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 00
00 00 00 30 00 00
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7e a8
00 00 00 28 00 00
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:25 R59DC-NAS-CREATIVE da218 at mpr0 bus 0 scbus13 target 391 lun 0
Apr  7 18:46:25 R59DC-NAS-CREATIVE da218:
GEOM_MULTIPATH: da288 removed from S1-D39-ZHZ4H1G5
GEOM_MULTIPATH: da218 in S1-D39-ZHZ4H1G5 was disconnected
GEOM_MULTIPATH: Error 6, da218 in S1-D39-ZHZ4H1G5 marked FAIL
GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
GEOM_MULTIPATH: da218 removed from S1-D39-ZHZ4H1G5
(da288:mpr0:0:544:0): Periph destroyed
(da218:mpr0:0:391:0): Periph destroyed
da358 at mpr1 bus 0 scbus14 target 85 lun 0
da358: <SEAGATE ST12000NM0038 E002> s/n ZHZ4H1G50000C0119AANmpr1:  detached
(da358:mpr1:0:85:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 88 00 00 00 28 00 00
(da358:mpr1:0:85:0): CAM status: CCB request aborted by the host
(da358:mpr1:0:85:0): Error 5, Periph was invalidated
GEOM_MULTIPATH: Error 5, da358 in S1-D39-ZHZ4H1G5 marked FAIL
(da358:mpr1:0:85:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 80 68 00 00 00 30 00 00
GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
<118>(da218:mpr0:0:391:0): Retrying command
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da288 removed from S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da218 in S1-D39-ZHZ4H1G5 was disconnected
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da218 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da218 removed from S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da288:mpr0:0:544:0): Periph destroyed
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da218:mpr0:0:391:0): Periph destroyed
Apr  7 18:46:25 R59DC-NAS-CREATIVE da358 at mpr1 bus 0 scbus14 target 85 lun 0
Apr  7 18:46:25 R59DC-NAS-CREATIVE da358: <SEAGATE ST12000NM0038 E002> s/n ZHZ4H1G50000C0119AANmpr1:
detached
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 88 00 00 00 28 00 00
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): Error 5, Periph was invalidated
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 5, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 80 68 00 00 00 30 00 00
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): Error 5, Periph was invalidated
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7f 00 00 00 00 30 00 00
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): Error 5, Periph was invalidated
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): READ(16). CDB: 88 00 00 00 00 01 0a 1a 7e a8 00 00 00 28 00 00
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): CAM status: CCB request aborted by the host
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): Error 5, Periph was invalidated
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 in S1-D39-ZHZ4H1G5 was disconnected
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 5, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 5, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 5, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: Error 6, da358 in S1-D39-ZHZ4H1G5 marked FAIL
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: all paths in S1-D39-ZHZ4H1G5 were marked FAIL, restore da358
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 is now active path in S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE GEOM_MULTIPATH: da358 removed from S1-D39-ZHZ4H1G5
Apr  7 18:46:25 R59DC-NAS-CREATIVE (da358:mpr1:0:85:0): Periph destroyed
 
Hi,

From my experience when one of my disk is noticed as removed with multipath that's means disk is KO.
Just need to replace it and recreate multipath.
 
Hi,

From my experience when one of my disk is noticed as removed with multipath that's means disk is KO.
Just need to replace it and recreate multipath.
It's looking that that's the case, crazy thing is this system is only 2 months old if that.
 
Harddisk failure rates are given in MTBF, that's a mean time, in other words an average. For an average lifespan of 10 years, if there's someone with a disk that's been performing perfectly well for 20 years it means there must also be some poor schmuck with a disk that broke within a couple of days ;)
 
Harddisk failure rates are given in MTBF, that's a mean time, in other words an average. For an average lifespan of 10 years, if there's someone with a disk that's been performing perfectly well for 20 years it means there must also be some poor schmuck with a disk that broke within a couple of days ;)
Good info to know haha, never heard of MTBF.
 
If you're around electronics long enough, you realize that sometimes things are just not sealed up good enough, the magical blue smoke leaks out and it stops working... ;)

Maybe at least it's under warranty???
Definitely going to so see if I can RMA it!
 
Back
Top