HighPoint Rocket 620 CAM Command timeout

Hi,

I'm trying to get a HighPoint Rocket 620 card working to use as a port-multiplier card for a drive array over eSata. HighPoint shows compatibility with Freebsd for this model, however any time I try to access one or more disks in the array, the controller and drives will disappear/detach, and if I let it attempt to retry, eventually the system will panic. It's also worth mentioning that the same drive array is accessible and reads/writes without issue via my motherboard's onboard SATA controller (albeit, only one drive in the array shows up when connected this way, hence the need for a port multiplier).

I get the following errors within 10-15 seconds of attempting to create a zpool via:
Code:
# zpool create -m /mnt/storage storage ada1 ada2 ada3 ada4
Code:
ahcich1: Timeout on slot 19 port 0
ahcich1: is 00000000 cs 00080000 ss 00000000 rs 00080000 tfd 50 serr 00000000 cmd 00739317
(ada1:ahcich1:0:0:0): WRITE_DMA. ACB: ca 00 60 00 00 40 00 00 00 00 01 00
(ada1:ahcich1:0:0:0): CAM status: Command timeout
(ada1:ahcich1:0:0:0): Retrying command
ahcich1: Poll timeout on slot 19 port 15
ahcich1: is 00000000 cs 00080000 ss 00000000 rs 00080000 tfd 150 serr 00000000 cmd 00738017
(aprobe4:ahcich1:0:15:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe4:ahcich1:0:15:0): CAM status: Command timeout
(aprobe4:ahcich1:0:15:0): Error 5, Retries exhausted
pmp0 at ahcich1 bus 0 scbus3 target 15 lun 0
pmp0: <Port Multiplier 0325197b 000e> detached
ada1 at ahcich1 bus 0 scbus3 target 0 lun 0
ada1: <ST31000524AS 0958> s/n 6VP4K6VN detached
ada2 at ahcich1 bus 0 scbus3 target 1 lun 0
ada2: <SAMSUNG HD103UJ 1AA01118> s/n S13PJ90SC13402 detached
ada3 at ahcich1 bus 0 scbus3 target 2 lun 0
ada3: <SAMSUNG HD103UJ 1AA01118> s/n S13PJ90SC13405 detached
ada4 at ahcich1 bus 0 scbus3 target 3 lun 0
ada4: <ST31000528AS CC37> s/n 9VP2GEQ6 detached
ahcich1: Timeout on slot 19 port 0
ahcich1: is 00000000 cs 00f80000 ss 00000000 rs 00f80000 tfd 150 serr 00000000 cmd 00738017
ahcich1:  ... waiting for slots 00f00000
ahcich1: Timeout on slot 20 port 1
ahcich1: is 00000000 cs 00f80000 ss 00000000 rs 00f80000 tfd 150 serr 00000000 cmd 00738017
ahcich1:  ... waiting for slots 00e00000
ahcich1: Timeout on slot 21 port 2
ahcich1: is 00000000 cs 00f80000 ss 00000000 rs 00f80000 tfd 150 serr 00000000 cmd 00738017
ahcich1:  ... waiting for slots 00c00000
ahcich1: Timeout on slot 22 port 3
ahcich1: is 00000000 cs 00f80000 ss 00000000 rs 00f80000 tfd 150 serr 00000000 cmd 00738017
ahcich1:  ... waiting for slots 00800000
ahcich1: Timeout on slot 23 port 15
ahcich1: is 00000000 cs 00f80000 ss 00000000 rs 00f80000 tfd 150 serr 00000000 cmd 00738017
(aprobe0:ahcich1:0:0:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe0:ahcich1:0:0:0): CAM status: Command timeout
(aprobe0:ahcich1:0:0:0): Error 5, Retry was blocked
(aprobe1:ahcich1:0:1:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe1:ahcich1:0:1:0): CAM status: Command timeout
(aprobe1:ahcich1:0:1:0): Error 5, Retry was blocked
(aprobe2:ahcich1:0:2:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe2:ahcich1:0:2:0): CAM status: Command timeout
(aprobe2:ahcich1:0:2:0): Error 5, Retry was blocked
(aprobe3:ahcich1:0:3:0): ATA_IDENTIFY. ACB: ec 00 00 00 00 40 00 00 00 00 00 00
(aprobe3:ahcich1:0:3:0): CAM status: Command timeout
(aprobe3:ahcich1:0:3:0): Error 5, Retry was blocked
(pmp0:ahcich1:0:15:0): WRITE_PM. ACB: e8 60 00 00 00 0f 00 00 00 00 00 00
(pmp0:ahcich1:0:15:0): CAM status: Command timeout
(pmp0:ahcich1:0:15:0): Error 5, Periph was invalidated
(pmp0:ahcich1:0:15:0): Periph destroyed
ahcich1: Poll timeout on slot 23 port 0
ahcich1: is 00000000 cs 00800000 ss 00000000 rs 00800000 tfd 150 serr 00000000 cmd 00738017
ahcich1: Poll timeout on slot 23 port 1
ahcich1: is 00000000 cs 00800000 ss 00000000 rs 00800000 tfd 150 serr 00000000 cmd 00738017
(aprobe0:ahcich1:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe0:ahcich1:0:0:0): CAM status: Command timeout
(aprobe0:ahcich1:0:0:0): Error 5, Retries exhausted
ahcich1: Poll timeout on slot 23 port 2
ahcich1: is 00000000 cs 00800000 ss 00000000 rs 00800000 tfd 150 serr 00000000 cmd 00738017
(aprobe1:ahcich1:0:1:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe1:ahcich1:0:1:0): CAM status: Command timeout
(aprobe1:ahcich1:0:1:0): Error 5, Retries exhausted
ahcich1: Poll timeout on slot 23 port 3
ahcich1: is 00000000 cs 00800000 ss 00000000 rs 00800000 tfd 150 serr 00000000 cmd 00738017
(aprobe2:ahcich1:0:2:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe2:ahcich1:0:2:0): CAM status: Command timeout
(aprobe2:ahcich1:0:2:0): Error 5, Retries exhausted
ahcich1: Poll timeout on slot 23 port 15
ahcich1: is 00000000 cs 00800000 ss 00000000 rs 00800000 tfd 150 serr 00000000 cmd 00738017
(aprobe3:ahcich1:0:3:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe3:ahcich1:0:3:0): CAM status: Command timeout
(aprobe3:ahcich1:0:3:0): Error 5, Retries exhausted
(aprobe4:ahcich1:0:15:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe4:ahcich1:0:15:0): CAM status: Command timeout
(aprobe4:ahcich1:0:15:0): Error 5, Retries exhausted
ahcich1: Timeout on slot 23 port 0
ahcich1: is 00000000 cs 07800000 ss 00000000 rs 07800000 tfd 150 serr 00000000 cmd 00738017
ahcich1:  ... waiting for slots 07000000
ahcich1: Timeout on slot 24 port 1
ahcich1: is 00000000 cs 07800000 ss 00000000 rs 07800000 tfd 150 serr 00000000 cmd 00738017
ahcich1:  ... waiting for slots 06000000
ahcich1: Timeout on slot 25 port 2
ahcich1: is 00000000 cs 07800000 ss 00000000 rs 07800000 tfd 150 serr 00000000 cmd 00738017
ahcich1:  ... waiting for slots 04000000
ahcich1: Timeout on slot 26 port 3
ahcich1: is 00000000 cs 07800000 ss 00000000 rs 07800000 tfd 150 serr 00000000 cmd 00738017
(ada1:ahcich1:0:0:0): WRITE_DMA. ACB: ca 00 60 00 00 40 00 00 00 00 01 00
(ada1:ahcich1:0:0:0): CAM status: Command timeout
(ada1:ahcich1:0:0:0): Error 5, Periph was invalidated
(ada2:ahcich1:0:1:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00
(ada2:ahcich1:0:1:0): CAM status: Command timeout
(ada2:ahcich1:0:1:0): Error 5, Periph was invalidated
(ada2:ahcich1:0:1:0): Periph destroyed
(ada3:ahcich1:0:2:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00
(ada3:ahcich1:0:2:0): CAM status: Command timeout
(ada3:ahcich1:0:2:0): Error 5, Periph was invalidated
(ada3:ahcich1:0:2:0): Periph destroyed
(ada4:ahcich1:0:3:0): SETFEATURES ENABLE RCACHE. ACB: ef aa 00 00 00 40 00 00 00 00 00 00
(ada4:ahcich1:0:3:0): CAM status: Command timeout
(ada4:ahcich1:0:3:0): Error 5, Periph was invalidated
(ada1:ahcich1:0:0:0): Periph destroyed
(ada4:ahcich1:0:3:0): Periph destroyed
ahcich1: Poll timeout on slot 26 port 0
ahcich1: is 00000000 cs 04000000 ss 00000000 rs 04000000 tfd 150 serr 00000000 cmd 00738017
ahcich1: Poll timeout on slot 26 port 1
ahcich1: is 00000000 cs 04000000 ss 00000000 rs 04000000 tfd 150 serr 00000000 cmd 00738017
(aprobe0:ahcich1:0:0:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe0:ahcich1:0:0:0): CAM status: Command timeout
(aprobe0:ahcich1:0:0:0): Error 5, Retries exhausted
ahcich1: Poll timeout on slot 26 port 2
ahcich1: is 00000000 cs 04000000 ss 00000000 rs 04000000 tfd 150 serr 00000000 cmd 00738017
(aprobe1:ahcich1:0:1:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe1:ahcich1:0:1:0): CAM status: Command timeout
(aprobe1:ahcich1:0:1:0): Error 5, Retries exhausted
ahcich1: Poll timeout on slot 26 port 3
ahcich1: is 00000000 cs 04000000 ss 00000000 rs 04000000 tfd 150 serr 00000000 cmd 00738017
(aprobe2:ahcich1:0:2:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe2:ahcich1:0:2:0): CAM status: Command timeout
(aprobe2:ahcich1:0:2:0): Error 5, Retries exhausted
(aprobe3:ahcich1:0:3:0): SOFT_RESET. ACB: 00 00 00 00 00 00 00 00 00 00 00 00
(aprobe3:ahcich1:0:3:0): CAM status: Command timeout
(aprobe3:ahcich1:0:3:0): Error 5, Retries exhausted

The controller appears to be recognized properly until I attempt to use it:
Code:
# pciconf -lvcb
Code:
ahci0@pci0:2:0:0:	class=0x010601 card=0x91201b4b chip=0x91201b4b rev=0x12 hdr=0x00
    vendor     = 'Marvell Technology Group Ltd.'
    device     = '88SE9120 SATA 6Gb/s Controller'
    class      = mass storage
    subclass   = SATA
    bar   [10] = type I/O Port, range 32, base 0xe090, size 8, enabled
    bar   [14] = type I/O Port, range 32, base 0xe080, size 4, enabled
    bar   [18] = type I/O Port, range 32, base 0xe070, size 8, enabled
    bar   [1c] = type I/O Port, range 32, base 0xe060, size 4, enabled
    bar   [20] = type I/O Port, range 32, base 0xe050, size 16, enabled
    bar   [24] = type Memory, range 32, base 0xe0700000, size 2048, enabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message 
    cap 10[70] = PCI-Express 2 legacy endpoint max data 128(512) link x1(x1)
                 speed 2.5(5.0) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 5 corrected
atapci1@pci0:2:0:1:	class=0x01018f card=0x91a41b4b chip=0x91a41b4b rev=0x12 hdr=0x00
    vendor     = 'Marvell Technology Group Ltd.'
    device     = '88SE9128 IDE Controller'
    class      = mass storage
    subclass   = ATA
    bar   [10] = type I/O Port, range 32, base 0xe040, size 8, enabled
    bar   [14] = type I/O Port, range 32, base 0xe030, size 4, enabled
    bar   [18] = type I/O Port, range 32, base 0xe020, size 8, enabled
    bar   [1c] = type I/O Port, range 32, base 0xe010, size 4, enabled
    bar   [20] = type I/O Port, range 32, base 0xe000, size 16, enabled
    bar   [24] = type Memory, range 32, base 0xe0520000, size 16, disabled
    cap 01[40] = powerspec 3  supports D0 D3  current D0
    cap 05[50] = MSI supports 1 message 
    cap 10[70] = PCI-Express 2 legacy endpoint max data 128(512) link x1(x1)
                 speed 2.5(5.0) ASPM disabled(L0s/L1)
    ecap 0001[100] = AER 1 0 fatal 0 non-fatal 5 corrected

The controller/drives as reported by dmesg:
Code:
# dmesg
Code:
pmp0 at ahcich1 bus 0 scbus3 target 15 lun 0
pmp0: <Port Multiplier 0325197b 000e> ATA-0 device
pmp0: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes)
pmp0: 5 fan-out ports
ada1 at ahcich1 bus 0 scbus3 target 0 lun 0
ada1: <ST31000524AS 0958> ATA-7 SATA 2.x device
ada1: Serial Number 6VP4K6VN
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 512bytes)
ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad10
ada2 at ahcich1 bus 0 scbus3 target 1 lun 0
ada2: <SAMSUNG HD103UJ 1AA01118> ATA-7 SATA 2.x device
ada2: Serial Number S13PJ90SC13402
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada2: Previously was known as ad11
ada3 at ahcich1 bus 0 scbus3 target 2 lun 0
ada3: <SAMSUNG HD103UJ 1AA01118> ATA-7 SATA 2.x device
ada3: Serial Number S13PJ90SC13405
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)
ada4 at ahcich1 bus 0 scbus3 target 3 lun 0
ada4: <ST31000528AS CC37> ATA-8 SATA 2.x device
ada4: Serial Number 9VP2GEQ6
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C)

As others have had success with some of the following options in /boot/loader.conf, I tried them, but with no success:
Code:
hw.pci.enable_msix="0"
hw.pci.enable_msi="0"
hint.ahci.0.msi="0"
hw.hptrr.attach_generic=0

Does this seem like a driver/configuration issue, or am I looking at a DOA controller card? Any insight much appreciated.
 
I haven't had much luck with eSATA and port-multipliers either. I had tested a RocketRaid card (forgot which model) but got the same time-outs as you. Unfortunately the official RocketRaid drivers only seem to support old and unsupported FreeBSD versions. I returned the card and swapped it with a siis(4) based chip-set. The same kind of time-outs. It's not the enclosure (I tried a different one too).

In the end I just gave up, bought an Icydock ( 4 swappable disks in 3 5.25" bays) and hooked everything up internally using a cheap Promise TX4 card. It's been working just fine ever since.
 
I can't help but notice that all your drives are ATA-7. But one, which is ATA-8. Given that you're dealing with timing issues. This may contribute to the problem. It might be worth attempting to tweak the setting in the BIOS, if possible. I also remember the ability to change the "timeouts" (length of time to wait for drive to come ready) for any/all attached drives. This is tunable (at least) within the kernel (sysctl(8). There's a section on Disk Tuning in the handbook, and other articles related to this. But I can't recall exactly where they are. Maybe someone else can chime in with them. If I do recall. I'll post links here.

--Chris
 
Thanks for the replies. Looks like I may need to find a different solution. Was hoping to get a zpool set up here with the 4 drives connected individually, but I may just fall back and use the drive enclosure's built-in RAID 10.

I can't help but notice that all your drives are ATA-7. But one, which is ATA-8. Given that you're dealing with timing issues. This may contribute to the problem.
Thanks for calling this out; I hadn't noticed. However, the issue persists with one drive in the enclosure, and I got to the point of attempting to use my boot SSD (connected through an SATA port on the motherboard) and it hung at loading the kernel, so I figured it's nothing to do with the enclosure, drives, or esata cable.

It might be worth attempting to tweak the setting in the BIOS, if possible. I also remember the ability to change the "timeouts" (length of time to wait for drive to come ready) for any/all attached drives. This is tunable (at least) within the kernel (sysctl(8). There's a section on Disk Tuning in the handbook, and other articles related to this. But I can't recall exactly where they are. Maybe someone else can chime in with them. If I do recall. I'll post links here.
I've set the hard disk delay to 5s, 10s in BIOS with no change. The GENERIC kernel has
Code:
options 	SCSI_DELAY=5000
by default and I usually remove this with SSDs, however compiling it back in shows no change. I've also played with the PCI Latency timer with no success.

Unfortunately the official RocketRaid drivers only seem to support old and unsupported FreeBSD versions.
Unfortunately indeed. I'll be checking with HighPoint to see if they've got anything to add. Will report back if I find anything/manage to get it working. It's lovely that their AHCI compatibility, well, isn't.

Cheers
 
Back
Top