Root mount waiting for: CAM

W

wowie

On a new install of FreeBSD 13.0:
I'm getting this error repeatedly until a timeout is reached. At that point the computer begins booting normally.
I'm a bit confused by the dmesg output. It looks as though the CAM error is migrating from USB to USB rather than persisting within a single port.

I would greatly appreciate any of y'alls assistance in understanding what is causing this issue

(redundant/irrelevant dmesg lines have been removed for your sanity)

dmesg

Code: 
pcm5:         PCM cap: 0x000e05e0 16 20 24 bits, 44 48 88 96 192 KHz
pcm5:             ADC: 8
pcm5:             ADC: 9
pcm5:
pcm5:     nid=8 [audio input]
pcm5:       + <- nid=35 [audio mixer] [src: speaker, monitor]
pcm5:              + <- nid=25 [pin: Mic (Pink Jack)] [src: monitor]
pcm5:              + <- nid=29 [beep widget] [src: speaker]
pcm5:
pcm5:     nid=9 [audio input]
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
pcm5:       + <- nid=34 [audio mixer] [src: speaker, monitor]
pcm5:              + <- nid=25 [pin: Mic (Pink Jack)] [src: monitor]
pcm5:              + <- nid=29 [beep widget] [src: speaker]
pcm5:
pcm5: Master Volume (OSS: vol): -62/2dB
pcm5:    +- ctl 36 (nid  27 in ):    mute
pcm5:    +- ctl 60 (nid  38 out):    -62/2dB (65 steps)
pcm5:    +- ctl 61 (nid  38 in   0): mute
pcm5:    +- ctl 62 (nid  38 in   1): mute
pcm5:
pcm5: PCM Volume (OSS: pcm): 0/0dB
pcm5:    +- ctl 61 (nid  38 in   0): mute
pcm5:
pcm5: Microphone2 Volume (OSS: monitor): 0/30dB
pcm5:    +- ctl 33 (nid  25 out):    0/30dB (4 steps)
pcm5:    +- ctl 39 (nid  34 in   1): mute
pcm5:    +- ctl 50 (nid  35 in   1): mute
pcm5:
pcm5: Speaker/Beep Volume (OSS: speaker)
pcm5:    +- ctl 43 (nid  34 in   5): mute
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
pcm5:    +- ctl 54 (nid  35 in   5): mute
pcm5:
pcm5: Recording Level (OSS: rec): -16/30dB
pcm5:    +- ctl  2 (nid   8 in   0): -16/30dB (47 steps) + mute
pcm5:    +- ctl  3 (nid   9 in   0): -16/30dB (47 steps) + mute
pcm5:    +- ctl 39 (nid  34 in   1): mute
pcm5:    +- ctl 43 (nid  34 in   5): mute
pcm5:    +- ctl 50 (nid  35 in   1): mute
pcm5:    +- ctl 54 (nid  35 in   5): mute
pcm5:
pcm5: Input Mix Level (OSS: mix)
pcm5:    +- ctl 62 (nid  38 in   1): mute
pcm5:
pcm5: Input Monitoring Level (OSS: igain): 0/0dB
pcm5:    +- ctl 62 (nid  38 in   1): mute
pcm5:
pcm5: Mixer "vol":
pcm5: Mixer "pcm":
pcm5: Mixer "rec":
pcm5: Mixer "igain":
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
pcm5: Mixer "monitor":
pcm5: Soft PCM mixer ENABLED
pcm5: Playback channel set is: Front Left, Front Right,
pcm5: Playback channel matrix is: 2.0 (connected)
pcm5: Recording channel set is: Front Left, Front Right,
pcm5: Recording channel matrix is: 2.0 (disconnected)
pcm6: <Realtek ALC889 (Onboard Digital)> at nid 30 on hdaa1
pcm6: Playback:
pcm6:      Stream cap: 0x00000005 AC3 PCM
pcm6:         PCM cap: 0x000e05f0 16 20 24 bits, 32 44 48 88 96 192 KHz
pcm6:             DAC: 6
pcm6:
pcm6:     nid=30 [pin: SPDIF-out (Fixed)]
pcm6:       + <- nid=6 [audio output] [src: pcm]
pcm6:
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
pcm6: Mixer "vol" -> "none": child=0x00000010
pcm6: Mixer "pcm": parent="vol"
pcm6: Soft PCM mixer ENABLED
pcm6: Playback channel set is: Front Left, Front Right,
pcm6: Playback channel matrix is: 2.0 (unknown)
pcm7: <Realtek ALC889 (Rear Digital)> at nid 17 on hdaa1
pcm7: Playback:
pcm7:      Stream cap: 0x00000005 AC3 PCM
pcm7:         PCM cap: 0x000e05f0 16 20 24 bits, 32 44 48 88 96 192 KHz
pcm7:             DAC: 16
pcm7:
pcm7:     nid=17 [pin: SPDIF-out (Orange Jack)]
pcm7:       + <- nid=16 [audio output] [src: pcm]
pcm7:
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
pcm7: Mixer "vol" -> "none": child=0x00000010
pcm7: Mixer "pcm": parent="vol"
pcm7: Soft PCM mixer ENABLED
pcm7: Playback channel set is: Front Left, Front Right,
pcm7: Playback channel matrix is: 2.0 (unknown)
pci0: driver added
found-> vendor=0x8086, dev=0x342e, revid=0x13
        domain=0, bus=0, slot=20, func=0
        class=08-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0000, statreg=0x0010, cachelnsz=64 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
pci0:0:20:0: reprobing on driver added
found-> vendor=0x8086, dev=0x3422, revid=0x13
        domain=0, bus=0, slot=20, func=1
        class=08-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0000, statreg=0x0010, cachelnsz=64 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
pci0:0:20:1: reprobing on driver added
found-> vendor=0x8086, dev=0x3423, revid=0x13
        domain=0, bus=0, slot=20, func=2
        class=08-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0000, statreg=0x0010, cachelnsz=64 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
pci0:0:20:2: reprobing on driver added
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
found-> vendor=0x8086, dev=0x3438, revid=0x13
        domain=0, bus=0, slot=20, func=3
        class=08-00-00, hdrtype=0x00, mfdev=1
        cmdreg=0x0000, statreg=0x0000, cachelnsz=64 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
pci0:0:20:3: reprobing on driver added
found-> vendor=0x8086, dev=0x3a30, revid=0x00
        domain=0, bus=0, slot=31, func=3
        class=0c-05-00, hdrtype=0x00, mfdev=0
        cmdreg=0x0103, statreg=0x0280, cachelnsz=0 (dwords)
        lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
        intpin=c, irq=18
pci0:0:31:3: reprobing on driver added
pci1: driver added
pci2: driver added
pci3: driver added
pci4: driver added
pci5: driver added
pci6: driver added
pci7: driver added
found-> vendor=0x1106, dev=0x3044, revid=0xc0
        domain=0, bus=8, slot=2, func=0
        class=0c-00-10, hdrtype=0x00, mfdev=0
        cmdreg=0x0117, statreg=0x0210, cachelnsz=64 (dwords)
        lattimer=0x40 (1920 ns), mingnt=0x00 (0 ns), maxlat=0x20 (8000 ns)
        intpin=a, irq=18
        powerspec 2  supports D0 D2 D3  current D0
pci0:8:2:0: reprobing on driver added
Trying to mount root from zfs:zroot/ROOT/default []...
Root mount waiting for: CAM usbus0 usbus1 usbus2 usbus3 usbus4 usbus5 usbus6 usbus7 usbus8
ugen8.1: <Intel EHCI root HUB> at usbus8
ugen1.1: <Intel UHCI root HUB> at usbus1
uhub0 on usbus8
ugen3.1: <Intel UHCI root HUB> at usbus3
uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus8
uhub1 on usbus3
uhub1: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus3
ugen5.1: <Intel UHCI root HUB> at usbus5
ugen4.1: <Intel EHCI root HUB> at usbus4
ugen2.1: <Intel UHCI root HUB> at usbus2
uhub2 on usbus0
uhub2: <0x1033 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub3 on usbus2
ugen6.1: <Intel UHCI root HUB> at usbus6
uhub4 on usbus5
uhub4: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus5
uhub3: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus2
uhub5 on usbus4
uhub5: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus4
ugen7.1: <Intel UHCI root HUB> at usbus7
uhub6 on usbus6
uhub6: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus6
uhub7 on usbus1
uhub7: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus1
uhub8 on usbus7
uhub8: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus7
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
uhub2: 4 ports with 4 removable, self powered
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
uhub4: 2 ports with 2 removable, self powered
uhub8: 2 ports with 2 removable, self powered
uhub3: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
uhub7: 2 ports with 2 removable, self powered
uhub6: 2 ports with 2 removable, self powered
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM usbus4 usbus8
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM usbus4 usbus8
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
uhub5: 6 ports with 6 removable, self powered
uhub0: 6 ports with 6 removable, self powered
ugen4.2: <vendor 0x05e3 USB2.0 Hub> at usbus4
uhub9 on uhub5
uhub9: <vendor 0x05e3 USB2.0 Hub, class 9/0, rev 2.00/60.70, addr 2> on usbus4
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM usbus4
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
uhub9: 4 ports with 4 removable, self powered
Root mount waiting for: CAM usbus4
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
ugen4.3: <Logitech USB Receiver> at usbus4
ugen4.4: <vendor 0x0461 USB Keyboard> at usbus4
ukbd0 on uhub9
ukbd0: <vendor 0x0461 USB Keyboard, class 0/0, rev 2.00/1.10, addr 4> on usbus4
kbd2 at ukbd0
kbd2: ukbd0, generic (0), config:0x0, flags:0x3d0000
Root mount waiting for: CAM usbus4
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
ugen4.5: <vendor 0x046d product 0x081b> at usbus4
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
ata2: reset tp2 stat0=a1 stat1=80 devices=0x0
ata3: reset tp1 mask=03 ostat0=21 ostat1=21
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
Root mount waiting for: CAM
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
ata3: reset tp2 stat0=a1 stat1=80 devices=0x0
ata4: SATA reset: ports status=0x01
ata4: p0: SATA connect time=0ms status=00000123
ata4: p1: SATA connect timeout status=00000000
ata4: reset tp1 mask=03 ostat0=50 ostat1=00
ata4: stat0=0x50 err=0x01 lsb=0x00 msb=0x00
ata4: stat1=0x00 err=0x01 lsb=0x00 msb=0x00
ata4: reset tp2 stat0=50 stat1=00 devices=0x1
Root mount waiting for: CAM
ata5: SATA reset: ports status=0x02
ata5: p0: SATA connect time=0ms status=00000113
ata5: p1: SATA connect timeout status=00000000
ata5: reset tp1 mask=03 ostat0=00 ostat1=00
ata5: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb
ata5: stat1=0x00 err=0x01 lsb=0x14 msb=0xeb
ata5: reset tp2 stat0=00 stat1=00 devices=0x30000
Root mount waiting for: CAM
ata6: SATA reset: ports status=0x00
ata6: p0: SATA connect timeout status=00000000
Root mount waiting for: CAM
ata7: SATA reset: ports status=0x00
ata7: p0: SATA connect timeout status=00000000
Root mount waiting for:pass0 at ata4 bus 0 scbus2 target 0 lun 0
CAM
...
timeotu reached
 
pciconf -lv

Code: 
hostb0@pci0:0:0:0:      class=0x060000 rev=0x13 hdr=0x00 vendor=0x8086 device=0x3405 subvendor=0x1043 subdevice=0x836b
    vendor     = 'Intel Corporation'
    device     = '5520/5500/X58 I/O Hub to ESI Port'
    class      = bridge
    subclass   = HOST-PCI
pcib1@pci0:0:1:0:       class=0x060400 rev=0x13 hdr=0x01 vendor=0x8086 device=0x3408 subvendor=0x1043 subdevice=0x836b
    vendor     = 'Intel Corporation'
    device     = '5520/5500/X58 I/O Hub PCI Express Root Port 1'
    class      = bridge
    subclass   = PCI-PCI
pcib2@pci0:0:2:0:       class=0x060400 rev=0x13 hdr=0x01 vendor=0x8086 device=0x3409 subvendor=0x1043 subdevice=0x836b
    vendor     = 'Intel Corporation'
    device     = '5520/5500/X58 I/O Hub PCI Express Root Port 2'
    class      = bridge
    subclass   = PCI-PCI
pcib3@pci0:0:3:0:       class=0x060400 rev=0x13 hdr=0x01 vendor=0x8086 device=0x340a subvendor=0x1043 subdevice=0x836b
    vendor     = 'Intel Corporation'
    device     = '5520/5500/X58 I/O Hub PCI Express Root Port 3'
    class      = bridge
    subclass   = PCI-PCI
pcib4@pci0:0:7:0:       class=0x060400 rev=0x13 hdr=0x01 vendor=0x8086 device=0x340e subvendor=0x1043 subdevice=0x836b
    vendor     = 'Intel Corporation'
    device     = '5520/5500/X58 I/O Hub PCI Express Root Port 7'
    class      = bridge
    subclass   = PCI-PCI
none0@pci0:0:20:0:      class=0x080000 rev=0x13 hdr=0x00 vendor=0x8086 device=0x342e subvendor=0x0000 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = '7500/5520/5500/X58 I/O Hub System Management Registers'
    class      = base peripheral
    subclass   = interrupt controller
none1@pci0:0:20:1:      class=0x080000 rev=0x13 hdr=0x00 vendor=0x8086 device=0x3422 subvendor=0x0000 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = '7500/5520/5500/X58 I/O Hub GPIO and Scratch Pad Registers'
    class      = base peripheral
    subclass   = interrupt controller
none2@pci0:0:20:2:      class=0x080000 rev=0x13 hdr=0x00 vendor=0x8086 device=0x3423 subvendor=0x0000 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = '7500/5520/5500/X58 I/O Hub Control Status and RAS Registers'
    class      = base peripheral
    subclass   = interrupt controller
none3@pci0:0:20:3:      class=0x080000 rev=0x13 hdr=0x00 vendor=0x8086 device=0x3438 subvendor=0x0000 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = '7500/5520/5500/X58 I/O Hub Throttle Registers'
    class      = base peripheral
    subclass   = interrupt controller
uhci0@pci0:0:26:0:      class=0x0c0300 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a37 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
uhci1@pci0:0:26:1:      class=0x0c0300 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a38 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
uhci2@pci0:0:26:2:      class=0x0c0300 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a39 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
ehci0@pci0:0:26:7:      class=0x0c0320 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a3c subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB2 EHCI Controller'
    class      = serial bus
    subclass   = USB
hdac1@pci0:0:27:0:      class=0x040300 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a3e subvendor=0x1043 subdevice=0x8418
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) HD Audio Controller'
    class      = multimedia
    subclass   = HDA
pcib5@pci0:0:28:0:      class=0x060400 rev=0x00 hdr=0x01 vendor=0x8086 device=0x3a40 subvendor=0x1043 subdevice=0x8418
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) PCI Express Root Port 1'
    class      = bridge
    subclass   = PCI-PCI
pcib6@pci0:0:28:2:      class=0x060400 rev=0x00 hdr=0x01 vendor=0x8086 device=0x3a44 subvendor=0x1043 subdevice=0x8418
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) PCI Express Root Port 3'
    class      = bridge
    subclass   = PCI-PCI
pcib7@pci0:0:28:4:      class=0x060400 rev=0x00 hdr=0x01 vendor=0x8086 device=0x3a48 subvendor=0x1043 subdevice=0x8418
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) PCI Express Root Port 5'
    class      = bridge
    subclass   = PCI-PCI
uhci3@pci0:0:29:0:      class=0x0c0300 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a34 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
uhci4@pci0:0:29:1:      class=0x0c0300 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a35 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
uhci5@pci0:0:29:2:      class=0x0c0300 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a36 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB UHCI Controller'
    class      = serial bus
    subclass   = USB
ehci1@pci0:0:29:7:      class=0x0c0320 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a3a subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) USB2 EHCI Controller'
    class      = serial bus
    subclass   = USB
pcib8@pci0:0:30:0:      class=0x060401 rev=0x90 hdr=0x01 vendor=0x8086 device=0x244e subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801 PCI Bridge'
    class      = bridge
    subclass   = PCI-PCI
isab0@pci0:0:31:0:      class=0x060100 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a16 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JIR (ICH10R) LPC Interface Controller'
    class      = bridge
    subclass   = PCI-ISA
atapci1@pci0:0:31:2:    class=0x01018f rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a20 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) 4 port SATA IDE Controller'
    class      = mass storage
    subclass   = ATA
ichsmb0@pci0:0:31:3:    class=0x0c0500 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a30 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) SMBus Controller'
    class      = serial bus
    subclass   = SMBus
atapci2@pci0:0:31:5:    class=0x010185 rev=0x00 hdr=0x00 vendor=0x8086 device=0x3a26 subvendor=0x1043 subdevice=0x82d4
    vendor     = 'Intel Corporation'
    device     = '82801JI (ICH10 Family) 2 port SATA IDE Controller'
    class      = mass storage
    subclass   = ATA
atapci0@pci0:1:0:0:     class=0x01018f rev=0x11 hdr=0x00 vendor=0x1b4b device=0x91a3 subvendor=0x1043 subdevice=0x8400
    vendor     = 'Marvell Technology Group Ltd.'
    class      = mass storage
    subclass   = ATA
xhci0@pci0:2:0:0:       class=0x0c0330 rev=0x03 hdr=0x00 vendor=0x1033 device=0x0194 subvendor=0x1043 subdevice=0x8413
    vendor     = 'NEC Corporation'
    device     = 'uPD720200 USB 3.0 Host Controller'
    class      = serial bus
    subclass   = USB
vgapci0@pci0:3:0:0:     class=0x030000 rev=0xa1 hdr=0x00 vendor=0x10de device=0x1187 subvendor=0x196e subdevice=0x0984
    vendor     = 'NVIDIA Corporation'
    device     = 'GK104 [GeForce GTX 760]'
    class      = display
    subclass   = VGA
hdac0@pci0:3:0:1:       class=0x040300 rev=0xa1 hdr=0x00 vendor=0x10de device=0x0e0a subvendor=0x196e subdevice=0x0984
    vendor     = 'NVIDIA Corporation'
    device     = 'GK104 HDMI Audio Controller'
    class      = multimedia
    subclass   = HDA
mskc0@pci0:6:0:0:       class=0x020000 rev=0x12 hdr=0x00 vendor=0x11ab device=0x4364 subvendor=0x1043 subdevice=0x81f8
    vendor     = 'Marvell Technology Group Ltd.'
    device     = '88E8056 PCI-E Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet
ath0@pci0:5:0:0:        class=0x028000 rev=0x01 hdr=0x00 vendor=0x168c device=0x0024 subvendor=0x1186 subdevice=0x3a70
    vendor     = 'Qualcomm Atheros'
    device     = 'AR5418 Wireless Network Adapter [AR5008E 802.11(a)bgn] (PCI-Express)'
    class      = network
none4@pci0:8:2:0:       class=0x0c0010 rev=0xc0 hdr=0x00 vendor=0x1106 device=0x3044 subvendor=0x1043 subdevice=0x81fe
    vendor     = 'VIA Technologies, Inc.'
    device     = 'VT6306/7/8 [Fire II(M)] IEEE 1394 OHCI Controller'
    class      = serial bus
    subclass   = FireWire
 
wowie said:
It looks as though the CAM error is migrating from USB to USB rather than persisting within a single port.
Click to expand...

wowie said:
ata2: stat0=0x21 err=0xff lsb=0xff msb=0xff
[...]
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
[...]
ata3: stat0=0x21 err=0xff lsb=0xff msb=0xff
[...]
ata5: stat0=0x00 err=0x01 lsb=0x14 msb=0xeb
Click to expand...

Those errors are all ata disk related, not usb. So unless you are using some USB disks (which IIRC should appear under the da driver anyways), you have either a faulty controller, cables or disk enclosure (I doubt all 3 disks are dying at the same time...) or there is a driver problem.
Are you running 13.0-RELEASE? The ICH10 chipset is fairly well supported by now, so I doubt this is a driver issue unless maybe you're running a development version and there was a regression in the ata driver lately...

How many/what disks do you actually have connected and on what disks you've set up your ZFS pool? Can you show the output of # camcontrol devlist and zpool status -v?
As the system is booting fine, those ata disks (or ports) doesn't seem to bee needed by freebsd. If you don't need the ata driver a quick&dirty workaround would be to try to blacklist that driver via /boot/loader.conf
 
sko said:
Those errors are all ata disk related, not usb. So unless you are using some USB disks (which IIRC should appear under the da driver anyways), you have either a faulty controller, cables or disk enclosure (I doubt all 3 disks are dying at the same time...) or there is a driver problem.
Are you running 13.0-RELEASE? The ICH10 chipset is fairly well supported by now, so I doubt this is a driver issue unless maybe you're running a development version and there was a regression in the ata driver lately...

How many/what disks do you actually have connected and on what disks you've set up your ZFS pool? Can you show the output of # camcontrol devlist and zpool status -v?
As the system is booting fine, those ata disks (or ports) doesn't seem to bee needed by freebsd. If you don't need the ata driver a quick&dirty workaround would be to try to blacklist that driver via /boot/loader.conf
Click to expand...
Thanks for lending a hand sko!

Nothing special going on here. Just a single disk w/ some partitions added by the autoinstaller for swap/boot/etc

Def running the current version. Just performed the install a week ago or so, and now am chasing down the odd tidbits and hardening.
freebsd-version
yields
Code: 
13.0-RELEASE

zpool status -v
yields
Code: 
  pool: zroot
state: ONLINE
config:

        NAME          STATE     READ WRITE CKSUM
        zroot         ONLINE       0     0     0
          ada0p3.eli  ONLINE       0     0     0

errors: No known data errors

camcontrol devlist
yeilds
Code: 
<WDC WD20EZAZ-00L9GB0 80.00A80>    at scbus2 target 0 lun 0 (ada0,pass0)
<TSSTcorp CDDVDW TS-H663C UO00>    at scbus3 target 0 lun 0 (cd0,pass1)

Also, in case they are helpful:

egrep 'ad[0-9]|cd[0-9]' /var/run/dmesg.boot
yields
Code: 
cd0 at ata5 bus 0 scbus3 target 0 lun 0
cd0: <TSSTcorp CDDVDW TS-H663C UO00> Removable CD-ROM SCSI device
cd0: Serial Number R7396GAZA01562
cd0: 150.000MB/s transfers (SATA 1.x, UDMA5, ATAPI 12bytes, PIO 8192bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present - tray closed

geom disk list
yields
Code: 
Geom name: cd0
Providers:
1. Name: cd0
   Mediasize: 0 (0B)
   Sectorsize: 2048
   Mode: r0w0e0
   descr: TSSTcorp CDDVDW TS-H663C
   ident: (null)
   rotationrate: unknown
   fwsectors: 0
   fwheads: 0

Geom name: ada0
Providers:
1. Name: ada0
   Mediasize: 2000398934016 (1.8T)
   Sectorsize: 512
   Stripesize: 4096
   Stripeoffset: 0
   Mode: r2w2e4
   descr: WDC WD20EZAZ-00L9GB0
   lunid: 50014ee2be57aa5a
   ident: WD-WX82AB03LH6A
   rotationrate: 5400
   fwsectors: 63
   fwheads: 16

gpart show
yields
Code: 
=>        40  3907029088  ada0  GPT  (1.8T)
          40        1024     1  freebsd-boot  (512K)
        1064         984        - free -  (492K)
        2048    25165824     2  freebsd-swap  (12G)
    25167872  3881861120     3  freebsd-zfs  (1.8T)
  3907028992         136        - free -  (68K)

Thanks again for any insight you can offer here. Much appreciated!! :)
 
I encountered this myself today - rebooted after patching 13.2-RELEASE-p4 -> 13.2-RELEASE-p8
usually, these FreeBSD systems reboot with amazing alacrity, but not today.
When the host didn't come back right away, I checked via IPMI and all I saw was a continuous repeat on the console:
waiting_for_CAM.png

This is a server class host (HPE ProLiant DL360 Gen10) with a RAID HBA.
the hardware examined via IPMI showed both disks and the RAID controller were fine.
I waited half an hour+ When it didn't come back, I performed a "Cold Boot" via IPMI.
The server came right up.
devices look fine to me
Code: 
grant@ef-ns01:~[20231213-9:02][#236687]> sudo camcontrol devlist
Password:
<HPE RAID 1(1+0) OK>               at scbus0 target 0 lun 1 (pass0,da0)
<HPE Smart Adapter 1.04>           at scbus0 target 66 lun 0 (pass1,ses0)
<HPE P408i-a SR Gen10 1.04>        at scbus0 target 1088 lun 1 (pass2)
<Generic- SD/MMC CRW 1.00>         at scbus1 target 0 lun 0 (da1,pass3)
grant@ef-ns01:~[20231213-9:02][#238686]> uname -r
13.2-RELEASE-p8
grant@ef-ns01:~[20231213-9:06][#240685]>

???
 
an old thread, I know, but just wanted to add that I have encountered this as well on a Dell XR7620 with HBA355i, running 14.1.

On this system, about 25% of the time when rebooting from within the os, it will get into this waiting for CAM state and fail to boot until it is power cycled.

Worked with Dell to do HW diagnostics, nothing stood out.

When it fails to boot, this is the text leading up to it:

Code: 
mpr0: Out of chain frames, consider increasing hw.mpr.max_chains.

[ USB is initialized here, then a 60 second delay ]

mpr0: Calling Reinit from mpr_wait_command, timeout=60, elapsed=60
mpr0: Reinitializing controller
mpr0: Firmware: 24.15.03.00, Driver: 23.00.00.00-fbsd
mpr0: IOCCapabilities: 2fa84c<ScsiTaskFull,DiagTrace, EEDP, TransRetry, EventReplay, MSI›
Index,HostDisc,FastPath,RDPQArray, AtomicReqDesc>
mpr0: Reinit success
mpr_config_get_dpm_pg0: request for page completed with error 60
ipmi0: IPMI device rev. 1, firmware rev. 6.10, version 2.0, device support mask 0xdf ipmi0: Number of channels 2 ipmi0: Attached watchdog
ipmi0: Establishing power cycle handler
Trying to mount root from zfs:zroot/ROOT/default Il...
Root mount waiting for: CAM
Root mount waiting for: CAM
mpr0: Found device <5c01<SspTarg, Direct,LsiDev,SepDev>, End Device > <3.06bps> handle<×0012> enclosureHandle<0x0002> slot 4
mpr0: At enclosure level 1 and connector name (CO  )
mpr0: Found device <881<SataDev,Direct>,End Device > <6.0Gbps> handle<0x0017> enclosureHandle<0x0002> slot 1
mpr0: At enclosure level 1 and connector name (CO  )
mpr0: Found device <881<SataDev, Direct>,End Device> <6.0Gbps> handle<0x0018> enclosureHandle<0x0002> slot 0
mpr0: At enclosure level 1 and connector name (CO  )
Root mount waiting for: CAM
mpr0: mpras_prepare_remove: Sending reset for target ID 1
mpr0: No pending commands: starting remove_device for target 1 handle 0x0017 mpr0: clearing target 1 handle 0x0017
mpr0: At enclosure level 1, slot 1, connector name (CO mpr0: mprsas_prepare_remove: Sending reset for target ID O
mpr0: No pending commands: starting remove_device for target 0 handle 0x0018 mpr0: clearing target @ handle 0x0018
mpr0: At enclosure level 1, slot 0, connector name (CO mpr0: mprsas_prepare_remove: Sending reset for target ID 4
mpr0: No pending commands: starting remove_device for target 4 handle 0x0012 mpr0: clearing target 4 handle 0x0012
mpr0: At enclosure level 1, slot 4, connector name (CO)
Root mount waiting for: CAM
Root mount waiting for: CAM
Root mount waiting for: CAM

HBA card details:
Code: 
# mprutil show all
Adapter:
mpr0 Adapter:
       Board Name: Dell HBA355i Fnt
   Board Assembly: 
        Chip Name: SAS3816
    Chip Revision: ALL
    BIOS Revision: 0.00.00.00
Firmware Revision: 24.15.05.00
  Integrated RAID: no
         SATA NCQ: ENABLED
 PCIe Width/Speed: x8 (16.0 GB/sec)
        IOC Speed: Full
      Temperature: 45 C

PhyNum  CtlrHandle  DevHandle  Disabled  Speed   Min    Max    Device
0                              N                 3.0    12     SAS Initiator 
1                              N                 3.0    12     SAS Initiator 
2                              N                 3.0    12     SAS Initiator 
3                              N                 3.0    12     SAS Initiator 
4                              N                 3.0    12     SAS Initiator 
5                              N                 3.0    12     SAS Initiator 
6                              N                 3.0    12     SAS Initiator 
7                              N                 3.0    12     SAS Initiator 
8                              N                 3.0    12     SAS Initiator 
9                              N                 3.0    12     SAS Initiator 
10                             N                 3.0    12     SAS Initiator 
11                             N                 3.0    12     SAS Initiator 
12                             N                 3.0    12     SAS Initiator 
13                             N                 3.0    12     SAS Initiator 
14      0009        0017       N         6.0     3.0    12     SAS Initiator 
15      000a        0018       N         6.0     3.0    12     SAS Initiator 
16                             N                 ???    ???    No Device     
17      0001        0012       N         3.0     ???    ???    SEP Target    
18                             N                 ???    ???    No Device     
19                             N                 ???    ???    No Device     
20                             N                 ???    ???    No Device     

Devices:
B____T    SAS Address      Handle  Parent    Device        Speed Enc  Slot  Wdt
00   04   3f4fe0805878d50c 0012    0001      SEP Target    3.0   0002 04    0
00   01   3f4ee0805878d50e 0017    0009      SATA Target   6.0   0002 01    1
00   00   3f4ee0805878d50f 0018    000a      SATA Target   6.0   0002 00    1

Enclosures:
Slots      Logical ID     SEPHandle  EncHandle    Type
  17    3f4ee0805811d500    0000       0001     Direct Attached SES-2
  05    3f4ee0805805a30c    0012       0002     Direct Attached SES-2

Expanders:
NumPhys   SAS Address     DevHandle   Parent  EncHandle  SAS Level

When it boots normally, I see the
Code: 
mpr0: Found device <881<SataDev,Direct>,End Device> <6.0Gbps> handle<0x0017> enclosureHandle<0x0002> slot 1
message but it proceeds quickly to loading the da0 and da1 drives, none of the timeout messages this time.
Code: 
da0 at mpr0 bus 0 scbus10 target 0 lun 0

I'm going to investigate the hw.mpr.max_chains setting to see if that has an impact here. Anyone have any other thoughts on what might be causing this in a low percentage of boot attempts?

Thanks!


EDIT:
setting hw.mpr.max_chains=32768 in my loader.conf didn't seem to have an effect. If anything it make the chance of hitting this error (and still getting the message about running out of chain frames) much higher, like 80% failure rate.
As noted here trying to set hw.mpr.max_chains using sysctl during runtime leads to unknown oid, but setting it at the loader prompt appears to accept it, and spits it back out on show hw.mpr.max_chains. But still get the out of chains error when booting.
 
Do you have the latest firmware for your HBA card? Those "??? No Device" lines in `mprutil` smell funny.
 
Yes, we've updated the firmware to 24.15.14.00 at Dell's direction, and it does show that version in iDRAC, but curiously it still reports "Firmware Revision: 24.15.05.00" when running mprutil and during bootup when we encounter the problem:
1741240301418.png

1741240337041.png


I'm wondering if the issue could be the mpr driver and firmware version being out of sync with each other? strange to me that idrac would report a different firmware than the system sees.
 
You must log in or register to reply here.
Back
Top