Solved Can anyone help me understand error messages - CAM status ATA status error...

exeter

exeter

I've just had this stream of errors[1]. Can anyone indicate what is at fault. I ran a ZFS scrub without any errors.

[1]
Code: 
Nov 30 11:07:59 pr5 kernel: (ada0:ata2:0:0:0): READ_DMA. ACB: c8 00 90 91 cf 47 00 00 00 00 00 00
Nov 30 11:07:59 pr5 kernel: (ada0:ata2:0:0:0): CAM status: ATA Status Error
Nov 30 11:07:59 pr5 kernel: (ada0:ata2:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 84 (ICRC ABRT )
Nov 30 11:07:59 pr5 kernel: (ada0:ata2:0:0:0): RES: 51 84 90 91 cf 07 07 00 00 00 00
Nov 30 11:07:59 pr5 kernel: (ada0:ata2:0:0:0): Retrying command, 3 more tries remain
Nov 30 11:21:25 pr5 kernel: (ada0:ata2:0:0:0): READ_DMA. ACB: c8 00 28 79 14 41 00 00 00 00 00 00
Nov 30 11:21:25 pr5 kernel: (ada0:ata2:0:0:0): CAM status: ATA Status Error
Nov 30 11:21:25 pr5 kernel: (ada0:ata2:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 84 (ICRC ABRT )
Nov 30 11:21:25 pr5 kernel: (ada0:ata2:0:0:0): RES: 51 84 28 79 14 01 01 00 00 00 00
Nov 30 11:21:25 pr5 kernel: (ada0:ata2:0:0:0): Retrying command, 3 more tries remain
Nov 30 11:56:42 pr5 kernel: arp: 192.168.0.156 moved from 28:f1:0e:27:20:7c to b8:8a:60:69:c6:48 on re0
Nov 30 12:02:39 pr5 kernel: arp: 192.168.0.190 moved from fc:08:4a:6b:df:82 to 34:f3:9a:61:0c:6a on re0
Nov 30 12:04:18 pr5 su[1365]: user1 to root on /dev/pts/2
Nov 30 12:04:19 pr5 kernel: arp: 192.168.0.190 moved from 34:f3:9a:61:0c:6a to fc:08:4a:6b:df:82 on re0
Nov 30 12:06:38 pr5 kernel: (ada0:ata2:0:0:0): READ_DMA. ACB: c8 00 38 da 54 46 00 00 00 00 00 00
Nov 30 12:06:38 pr5 kernel: (ada0:ata2:0:0:0): CAM status: ATA Status Error
Nov 30 12:06:38 pr5 kernel: (ada0:ata2:0:0:0): ATA status: 51 (DRDY SERV ERR), error: 84 (ICRC ABRT )
Nov 30 12:06:38 pr5 kernel: (ada0:ata2:0:0:0): RES: 51 84 38 da 54 06 06 00 00 00 00
Nov 30 12:06:38 pr5 kernel: (ada0:ata2:0:0:0): Retrying command, 3 more tries remain
Nov 30 13:07:13 pr5 kernel: arp: 192.168.0.156 moved from 28:f1:0e:27:20:7c to b8:8a:60:69:c6:48 on re0
Nov 30 13:07:18 pr5 kernel: arp: 192.168.0.156 moved from b8:8a:60:69:c6:48 to 28:f1:0e:27:20:7c on re0
Nov 30 13:24:44 pr5 su[1489]: user1 to root on /dev/pts/2
 
Your disk might be dying. Try checking SMART; smartctl -a /dev/ada0
 
That's odd:
Code: 
 smartctl -a /dev/ada0
smartctl: Command not found.

Code: 
uname -a
FreeBSD pr5.blank.blank 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64
root@pr5:/usr/home/parkrow #
 
Note you can still have a ZFS scrub succeed (with no errors) while your drive is tipping towards failure. As you can see in the logs, it will retry a command a number of times at the device driver level; as long as it succeeds before running out of retries, nothing gets reported back up to the ZFS layer that "this took a retry".

If this is a new device, I would check power and data cables are well seated. If it's an older drive / system that has worked without issue for years and has important data, I would start planning to replace the drive.
 
Here's the results of smartctl -a /dev/ada0, I don't have any experience reading these things so i can't be sure what it means:

Code: 
smartctl -a /dev/ada0
smartctl 7.4 2023-08-01 r5530 [FreeBSD 13.1-RELEASE amd64] (local build)
Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     Crucial/Micron Client SSDs
Device Model:     CT480BX500SSD1
Serial Number:    2209E613B7AD
LU WWN Device Id: 5 00a075 1e613b7ad
Firmware Version: M6CR054
User Capacity:    480,103,981,056 bytes [480 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
TRIM Command:     Available
Device is:        In smartctl database 7.3/5528
ATA Version is:   ACS-3 T13/2161-D revision 4
SATA Version is:  SATA 3.3, 6.0 Gb/s (current: 3.0 Gb/s)
Local Time is:    Thu Nov 30 15:59:52 2023 GMT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00)    Offline data collection activity
                    was never started.
                    Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)    The previous self-test routine completed
                    without error or no self-test has ever
                    been run.
Total time to complete Offline
data collection:         (  120) seconds.
Offline data collection
capabilities:              (0x11) SMART execute Offline immediate.
                    No Auto Offline data collection support.
                    Suspend Offline collection upon new
                    command.
                    No Offline surface scan supported.
                    Self-test supported.
                    No Conveyance Self-test supported.
                    No Selective Self-test supported.
SMART capabilities:            (0x0002)    Does not save SMART data before
                    entering power-saving mode.
                    Supports SMART auto save timer.
Error logging capability:        (0x01)    Error logging supported.
                    General Purpose Logging supported.
Short self-test routine
recommended polling time:      (   2) minutes.
Extended self-test routine
recommended polling time:      (  10) minutes.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   100   100   000    Pre-fail  Always       -       0
  5 Reallocate_NAND_Blk_Cnt 0x0032   100   100   010    Old_age   Always       -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       10575
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       24
171 Program_Fail_Count      0x0032   100   100   000    Old_age   Always       -       0
172 Erase_Fail_Count        0x0032   100   100   000    Old_age   Always       -       0
173 Ave_Block-Erase_Count   0x0032   100   100   000    Old_age   Always       -       3
174 Unexpect_Power_Loss_Ct  0x0032   100   100   000    Old_age   Always       -       18
180 Unused_Reserve_NAND_Blk 0x0033   100   100   000    Pre-fail  Always       -       40
183 SATA_Interfac_Downshift 0x0032   100   100   000    Old_age   Always       -       0
184 Error_Correction_Count  0x0032   100   100   000    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0022   074   068   000    Old_age   Always       -       26 (Min/Max 23/32)
196 Reallocated_Event_Count 0x0032   100   100   000    Old_age   Always       -       0
197 Current_Pending_ECC_Cnt 0x0032   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0032   100   100   000    Old_age   Always       -       0
202 Percent_Lifetime_Remain 0x0030   100   100   001    Old_age   Offline      -       0
206 Write_Error_Rate        0x000e   100   100   000    Old_age   Always       -       0
210 Success_RAIN_Recov_Cnt  0x0032   100   100   000    Old_age   Always       -       0
246 Total_LBAs_Written      0x0032   100   100   000    Old_age   Always       -       993588552
247 Host_Program_Page_Count 0x0032   100   100   000    Old_age   Always       -       31049642
248 FTL_Program_Page_Count  0x0032   100   100   000    Old_age   Always       -       10745856
249 Unkn_CrucialMicron_Attr 0x0032   100   100   000    Old_age   Always       -       0
250 Read_Error_Retry_Rate   0x0032   100   100   000    Old_age   Always       -       0
251 Unkn_CrucialMicron_Attr 0x0032   100   100   000    Old_age   Always       -       860264042
252 Unkn_CrucialMicron_Attr 0x0032   100   100   000    Old_age   Always       -       1
253 Unkn_CrucialMicron_Attr 0x0032   100   100   000    Old_age   Always       -       0
254 Unkn_CrucialMicron_Attr 0x0032   100   100   000    Old_age   Always       -       12
223 Unkn_CrucialMicron_Attr 0x0032   100   100   000    Old_age   Always       -       0

SMART Error Log not supported

SMART Self-test log structure revision number 1
No self-tests have been logged.  [To run self-tests, use: smartctl -t]

Selective Self-tests/Logging not supported

The above only provides legacy SMART information - try 'smartctl -x' for more
 
Doesn't look like the drive itself feels things have gone south. Re-seat (disconnect and reconnect) power and (both ends) data cables, and see if it goes away.

If it continues, you might be able to get a warranty replacement; it looks like it carries a 3-year warranty.
 
The original errors in your log are CRC errors on the interface, meaning the electrical connection between motherboard and disk is not perfect. Most likely, the problem is cables (data or power), although theoretically it could also be the drive or motherboard ends.
 
Hmmm, the errors have ceased for now. I'll give the cables a re-seat and see how it goes. Thanks all.
 
exeter said:
Hmmm, the errors have ceased for now. I'll give the cables a re-seat and see how it goes. Thanks all.
Click to expand...
Sometimes poor cabling will also result in zfs crc errors showing in zpool status (if you've seen that). Typically people just clear them out and go about their day unless the drive faults.
 
Cable might not be plugged in entirely. And a bit of vibration could cause it to connect and disconnect. Properly reseating the cable usually fixes that.
 
You must log in or register to reply here.
Back
Top