Hello! I'm trying to get multipath working without any success.
My hardware is:
Server: HPE Proliant DL360p Gen9
SAS HBA: LSI 9300-8e (latest firmware)
External enclosure: HP D2600 SAS2 JBOD Dual Controller
HDDs: NL-SAS Dual-port SEAGATE ST6000NM0034 x10
SSD: Stec ZeusRAM 8GB
OS: FreeBSD 12.1, fresh install +
Each port of HBA is connected to different JBOD controller
The problem is that the OS can see only one path no matter what I do.
If I disconnect one cable from one JBOD controller OS can't see disks and enclosure at all. If I reboot system after that OS can see disks again on second cable.
HBA is OK, I double checked. I can see all 22 paths for disks on BIOS boot screen and LSI configuration utility.
In FreeBSD:
All log files are attached to this post.
I tried to boot to Centos7-Live and it can see all disks at all paths (log files also attached):
What am I doing wrong?
My hardware is:
Server: HPE Proliant DL360p Gen9
SAS HBA: LSI 9300-8e (latest firmware)
External enclosure: HP D2600 SAS2 JBOD Dual Controller
HDDs: NL-SAS Dual-port SEAGATE ST6000NM0034 x10
SSD: Stec ZeusRAM 8GB
OS: FreeBSD 12.1, fresh install +
Each port of HBA is connected to different JBOD controller
The problem is that the OS can see only one path no matter what I do.
If I disconnect one cable from one JBOD controller OS can't see disks and enclosure at all. If I reboot system after that OS can see disks again on second cable.
HBA is OK, I double checked. I can see all 22 paths for disks on BIOS boot screen and LSI configuration utility.
In FreeBSD:
camcontrol devlist
- 11 disks.
Code:
<HP MB6000JEFND HPD5> at scbus0 target 0 lun 0 (pass0,da0)
<HP MB6000JEFND HPD3> at scbus0 target 1 lun 0 (pass1,da1)
<SEAGATE ST6000NM0034 MS85> at scbus0 target 2 lun 0 (pass2,da2)
<SEAGATE ST6000NM0034 MS84> at scbus0 target 3 lun 0 (pass3,da3)
<SEAGATE ST6000NM0034 MS2E> at scbus0 target 4 lun 0 (pass4,da4)
<SEAGATE ST6000NM0034 MS85> at scbus0 target 5 lun 0 (pass5,da5)
<SEAGATE ST6000NM0034 MS85> at scbus0 target 6 lun 0 (pass6,da6)
<SEAGATE ST6000NM0034 MS84> at scbus0 target 7 lun 0 (pass7,da7)
<SEAGATE ST6000NM0034 MS2E> at scbus0 target 8 lun 0 (pass8,da8)
<SEAGATE ST6000NM0034 MS85> at scbus0 target 10 lun 0 (pass9,da9)
<STEC ZeusRAM C025> at scbus0 target 11 lun 0 (pass10,da10)
<HP D2600 SAS AJ940A 0150> at scbus0 target 24 lun 0 (ses0,pass11)
<HP RAID 0 OK> at scbus1 target 0 lun 0 (pass12,da11)
<HP Virtual DVD-ROM \0000.0> at scbus6 target 0 lun 0 (pass13,cd0)
<kdata KDATA 1.00> at scbus7 target 0 lun 0 (da12,pass14)
sas3ircu 0 display
- 11 disks. dmesg | grep mpr
- 11 disks
Code:
mpr0: <Avago Technologies (LSI) SAS3008> port 0x6000-0x60ff mem 0xf7ef0000-0xf7efffff,0xf7e80000-0xf7ebffff irq 26 at device 0.0 numa-domain 0 on pci1
mpr0: Firmware: 16.00.10.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 7a85c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRetry,EventReplay,MSIXIndex,HostDisc,FastPath,RDPQArray>
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x000a> enclosureHandle<0x0002> slot 13
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x000b> enclosureHandle<0x0002> slot 14
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x000c> enclosureHandle<0x0002> slot 15
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x000d> enclosureHandle<0x0002> slot 16
mpr0: At enclosure level 0 and connector name ( )
mpr0: nvd0: 1831420MB (468843606 4096 byte sectors)
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x000f> enclosureHandle<0x0002> slot 18
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x0010> enclosureHandle<0x0002> slot 19
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x0011> enclosureHandle<0x0002> slot 20
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x0012> enclosureHandle<0x0002> slot 21
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x0013> enclosureHandle<0x0002> slot 23
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <401<SspTarg>,End Device> <6.0Gbps> handle<0x0014> enclosureHandle<0x0002> slot 24
mpr0: At enclosure level 0 and connector name ( )
mpr0: Found device <4441<SspInit,SspTarg,SepDev>,End Device> <6.0Gbps> handle<0x0015> enclosureHandle<0x0002> slot 0
mpr0: At enclosure level 0 and connector name ( )
da1 at mpr0 bus 0 scbus0 target 1 lun 0
da0 at mpr0 bus 0 scbus0 target 0 lun 0
da10 at mpr0 bus 0 scbus0 target 11 lun 0
da3 at mpr0 bus 0 scbus0 target 3 lun 0
da2 at mpr0 bus 0 scbus0 target 2 lun 0
da5 at mpr0 bus 0 scbus0 target 5 lun 0
da6 at mpr0 bus 0 scbus0 target 6 lun 0
da8 at mpr0 bus 0 scbus0 target 8 lun 0
da9 at mpr0 bus 0 scbus0 target 10 lun 0
da4 at mpr0 bus 0 scbus0 target 4 lun 0
da7 at mpr0 bus 0 scbus0 target 7 lun 0
ses0 at mpr0 bus 0 scbus0 target 24 lun 0
I tried to boot to Centos7-Live and it can see all disks at all paths (log files also attached):
lsscsi
Code:
[0:0:0:0] cd/dvd HP Virtual DVD-ROM /dev/sr0
[3:0:0:0] disk kdata KDATA 1.00 /dev/sda
[4:0:0:0] storage HP P420i 8.32 -
[4:1:0:0] disk HP LOGICAL VOLUME 8.32 /dev/sdb
[5:0:0:0] disk HP MB6000JEFND HPD5 /dev/sdc
[5:0:1:0] disk HP MB6000JEFND HPD3 /dev/sdd
[5:0:2:0] disk SEAGATE ST6000NM0034 MS85 /dev/sde
[5:0:3:0] disk SEAGATE ST6000NM0034 MS84 /dev/sdf
[5:0:4:0] disk SEAGATE ST6000NM0034 MS2E /dev/sdg
[5:0:5:0] disk SEAGATE ST6000NM0034 MS85 /dev/sdh
[5:0:6:0] disk SEAGATE ST6000NM0034 MS85 /dev/sdi
[5:0:7:0] disk SEAGATE ST6000NM0034 MS84 /dev/sdj
[5:0:8:0] disk SEAGATE ST6000NM0034 MS2E /dev/sdk
[5:0:9:0] disk SEAGATE ST6000NM0034 MS85 /dev/sdl
[5:0:10:0] disk STEC ZeusRAM C025 /dev/sdm
[5:0:11:0] enclosu HP D2600 SAS AJ940A 0150 -
[5:0:12:0] disk HP MB6000JEFND HPD5 /dev/sdn
[5:0:13:0] disk HP MB6000JEFND HPD3 /dev/sdo
[5:0:14:0] disk SEAGATE ST6000NM0034 MS85 /dev/sdp
[5:0:15:0] disk SEAGATE ST6000NM0034 MS84 /dev/sdq
[5:0:16:0] disk SEAGATE ST6000NM0034 MS2E /dev/sdr
[5:0:17:0] disk SEAGATE ST6000NM0034 MS85 /dev/sds
[5:0:18:0] disk SEAGATE ST6000NM0034 MS85 /dev/sdt
[5:0:19:0] disk SEAGATE ST6000NM0034 MS84 /dev/sdu
[5:0:20:0] disk SEAGATE ST6000NM0034 MS2E /dev/sdv
[5:0:21:0] disk SEAGATE ST6000NM0034 MS85 /dev/sdw
[5:0:22:0] disk STEC ZeusRAM C025 /dev/sdx
[5:0:23:0] enclosu HP D2600 SAS AJ940A 0150 -
What am I doing wrong?