9.1-REL Kernel Panic (panic: cancel_mkdir_dotdot: Lost inodedep)

junovitch@

Developer
Good evening FreeBSD fans. My box suffered a crashed today and I'm not completely sure what to make of it. I'm not completely confident if this is purely software related or could have a hardware component because of the cheap SSD I'm running on. Any insight into the below info would be appreciated.

Abbreviated dmesg following a reboot.
Code:
panic: cancel_mkdir_dotdot: Lost inodedep
cpuid = 2
KDB: stack backtrace:
#0 0xffffffff809208a6 at kdb_backtrace+0x66
#1 0xffffffff808ea8be at panic+0x1ce
#2 0xffffffff80b162aa at newdirrem+0x5da
#3 0xffffffff80b1663a at softdep_setup_directory_change+0xaa
#4 0xffffffff80b278fc at ufs_dirrewrite+0x1ac
#5 0xffffffff80b323fa at ufs_rename+0x104a
#6 0xffffffff80c68406 at VOP_RENAME_APV+0x46
#7 0xffffffff8098c40d at kern_renameat+0x4cd
#8 0xffffffff80bd7ae6 at amd64_syscall+0x546
#9 0xffffffff80bc3447 at Xfast_syscall+0xf7
Uptime: 68d13h3m49s
Copyright (c) 1992-2012 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-RELEASE #0 r243825: Tue Dec  4 09:23:10 UTC 2012
    root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64

...

ada0 at ahcich0 bus 0 scbus0 target 0 lun 0
ada0: <OCZ-AGILITY2 3.5 1.33> ATA-8 SATA 2.x device
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 85857MB (175836528 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4

...

Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0
Root mount waiting for: usbus0

...

Trying to mount root from ufs:/dev/ada0p2 [rw]...
WARNING: / was not properly dismounted
ZFS filesystem version 5
ZFS storage pool version 28
em1: link state changed to UP

Contents of /var/crash/minfree (This was the only file in /var/crash)
Code:
2048

More to follow.
 
The output of smartctl -a /dev/ada0. This is my SSD boot drive running a UFS2 partion created by the installer. The only change after installation was to run tunefs -t enable /dev/ada0p2

Code:
smartctl 6.0 2012-10-10 r3643 [FreeBSD 9.1-RELEASE amd64] (local build)
Copyright (C) 2002-12, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Model Family:     SandForce Driven SSDs
Device Model:     OCZ-AGILITY2 3.5
Serial Number:    OCZ-H79S3R89PDO9J75E
LU WWN Device Id: 5 e83a97 fd027f85e
Firmware Version: 1.33
User Capacity:    90,028,302,336 bytes [90.0 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Device is:        In smartctl database [for details use: -P show]
ATA Version is:   ATA8-ACS T13/1699-D revision 6
SATA Version is:  SATA 2.6, 3.0 Gb/s
Local Time is:    Mon Feb 18 02:42:01 2013 UTC
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x02)	Offline data collection activity
					was completed without error.
					Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0)	The previous self-test routine completed
					without error or no self-test has ever 
					been run.
Total time to complete Offline 
data collection: 		(    0) seconds.
Offline data collection
capabilities: 			 (0x7f) SMART execute Offline immediate.
					Auto Offline data collection on/off support.
					Abort Offline collection upon new
					command.
					Offline surface scan supported.
					Self-test supported.
					Conveyance Self-test supported.
					Selective Self-test supported.
SMART capabilities:            (0x0003)	Saves SMART data before entering
					power-saving mode.
					Supports SMART auto save timer.
Error logging capability:        (0x01)	Error logging supported.
					General Purpose Logging supported.
Short self-test routine 
recommended polling time: 	 (   1) minutes.
Extended self-test routine
recommended polling time: 	 (   5) minutes.
Conveyance self-test routine
recommended polling time: 	 (   2) minutes.
SCT capabilities: 	       (0x003d)	SCT Status supported.
					SCT Error Recovery Control supported.
					SCT Feature Control supported.
					SCT Data Table supported.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   119   099   050    Pre-fail  Always       -       0/223074148
  5 Retired_Block_Count     0x0033   100   100   003    Pre-fail  Always       -       0
  9 Power_On_Hours_and_Msec 0x0032   100   100   000    Old_age   Always       -       14516h+53m+23.330s
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       61
171 Program_Fail_Count      0x0032   000   000   000    Old_age   Always       -       0
172 Erase_Fail_Count        0x0032   000   000   000    Old_age   Always       -       0
174 Unexpect_Power_Loss_Ct  0x0030   000   000   000    Old_age   Offline      -       28
177 Wear_Range_Delta        0x0000   000   000   000    Old_age   Offline      -       1
181 Program_Fail_Count      0x0032   000   000   000    Old_age   Always       -       0
182 Erase_Fail_Count        0x0032   000   000   000    Old_age   Always       -       0
187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0022   030   030   000    Old_age   Always       -       30 (Min/Max 30/30)
195 ECC_Uncorr_Error_Count  0x001c   119   099   000    Old_age   Offline      -       0/223074148
196 Reallocated_Event_Count 0x0033   100   100   000    Pre-fail  Always       -       0
231 SSD_Life_Left           0x0013   100   100   010    Pre-fail  Always       -       0
233 SandForce_Internal      0x0000   000   000   000    Old_age   Offline      -       3968
234 SandForce_Internal      0x0032   000   000   000    Old_age   Always       -       2368
241 Lifetime_Writes_GiB     0x0032   000   000   000    Old_age   Always       -       2368
242 Lifetime_Reads_GiB      0x0032   000   000   000    Old_age   Always       -       6016

SMART Error Log not supported

SMART Self-test Log not supported

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.
 
The contents of /var/log/all.log with duplicates of the above removed. I had to run through a manual fsck from single user mode.

Code:
...
Feb 18 02:20:18 <console.info> xts-bsd kernel: Entropy harvesting: interrupts ethernet point_to_point kickstart.
Feb 18 02:20:18 <console.info> xts-bsd kernel: Starting file system checks:
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** SU+J Recovering /dev/ada0p2
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Reading 33554432 byte journal from inode 4.
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Building recovery table.
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Resolving unreferenced inode list.
Feb 18 02:20:18 <console.info> xts-bsd kernel: /dev/ada0p2: Inode 5717544 link count 0 invalid
Feb 18 02:20:18 <console.info> xts-bsd kernel: /dev/ada0p2: UNEXPECTED SU+J INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: /dev/ada0p2: INTERNAL ERROR: GOT TO reply()
Feb 18 02:20:18 <console.info> xts-bsd kernel: /dev/ada0p2: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY.
Feb 18 02:20:18 <console.info> xts-bsd kernel: Automatic file system check failed; help!
Feb 18 02:20:18 <console.info> xts-bsd kernel: ERROR: ABORTING BOOT (sending SIGTERM to parent)!
Feb 18 02:20:18 <console.info> xts-bsd kernel: Feb 17 21:03:53 init: /bin/sh on /etc/rc terminated abnormally, going to single user mode
Feb 18 02:20:18 <console.info> xts-bsd kernel: Enter full pathname of shell or RETURN for /bin/sh: # uptime
Feb 18 02:20:18 <console.info> xts-bsd kernel: 2:16AM  up  5:13, 0 users, load averages: 0.00, 0.00, 0.00
Feb 18 02:20:18 <console.info> xts-bsd kernel: # fsck
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** /dev/ada0p2
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: USE JOURNAL? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** SU+J Recovering /dev/ada0p2
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Reading 33554432 byte journal from inode 4.
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECOVER? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Building recovery table.
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Resolving unreferenced inode list.
Feb 18 02:20:18 <console.info> xts-bsd kernel: Inode 5717544 link count 0 invalid
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SU+J INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: FALLBACK TO FULL FSCK? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Skipping journal, falling through to full fsck
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Last Mounted on /
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Root file system
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Phase 1 - Check Blocks and Sizes
Feb 18 02:20:18 <console.info> xts-bsd kernel: INCORRECT BLOCK COUNT I=7881966 (8 should be 0)
Feb 18 02:20:18 <console.info> xts-bsd kernel: CORRECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Phase 2 - Check Pathnames
Feb 18 02:20:18 <console.info> xts-bsd kernel: BAD TYPE VALUE  I=5553813  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=143 MTIME=Mar 27 22:06 2009 
Feb 18 02:20:18 <console.info> xts-bsd kernel: FILE=?
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: FIX? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: BAD TYPE VALUE  I=5714773  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=263 MTIME=Feb  3 04:53 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: FILE=?
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: FIX? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: BAD TYPE VALUE  I=5714773  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=263 MTIME=Feb  3 04:53 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: FILE=?
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: FIX? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: /var/spool/smtpd/queue/fa/fa37f0b1 IS AN EXTRANEOUS HARD LINK TO DIRECTORY /var/spool/smtpd/incoming/fa37f0b1
Feb 18 02:20:18 <console.info> xts-bsd kernel: REMOVE? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Phase 3 - Check Connectivity
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=6514299  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=5632 MTIME=Jan  5 22:49 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: NO lost+found DIRECTORY
Feb 18 02:20:18 <console.info> xts-bsd kernel: CREATE? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=6514299 CONNECTED. PARENT WAS I=5553822
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=6514212  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1536 MTIME=Jan  5 22:49 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=6514212 CONNECTED. PARENT WAS I=5553822
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=6180502  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=7680 MTIME=Jan  5 22:49 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=6180502 CONNECTED. PARENT WAS I=5553822
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=5717546  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan 13 02:19 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=5717546 CONNECTED. PARENT WAS I=5717544
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=5714776  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan 13 00:00 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=5714776 CONNECTED. PARENT WAS I=5714773
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=5714775  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan  6 01:01 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=5714775 CONNECTED. PARENT WAS I=5714773
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=5618108  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=512 MTIME=Jan  5 22:51 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=5618108 CONNECTED. PARENT WAS I=5553813
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF DIR  I=5553823  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan  5 22:49 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: RECONNECT? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: DIR I=5553823 CONNECTED. PARENT WAS I=5553821
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Phase 4 - Check Reference Counts
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=4815370  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=715 MTIME=Feb 14 02:07 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] CLEAR? [yn] CLEAR? [yn] CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=4815373  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=391 MTIME=Feb 14 03:46 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=4815556  OWNER=logcheck MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=925 MTIME=Feb 17 21:02 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=5553823  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan  5 22:49 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=5618108  OWNER=root MODE=40755
 
Continued from above.

Code:
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=512 MTIME=Jan  5 22:51 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=5714775  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan  6 01:01 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=5714776  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan 13 00:00 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=5717546  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1024 MTIME=Jan 13 02:19 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6099615  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb  3 06:36 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=6180502  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=7680 MTIME=Jan  5 22:49 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=6514212  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1536 MTIME=Jan  5 22:49 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=6514299  OWNER=root MODE=40755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=5632 MTIME=Jan  5 22:49 2013  COUNT 0 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT INCREASING
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNEXPECTED SOFT UPDATE INCONSISTENCY
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848051  OWNER=1002 MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=1146 MTIME=Feb 17 05:00 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848231  OWNER=88 MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 14 02:07 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848232  OWNER=88 MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=303 MTIME=Feb 17 03:30 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] CLEAR? [yn] CLEAR? [yn] CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848233  OWNER=88 MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 14 02:07 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848234  OWNER=88 MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 14 02:07 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848235  OWNER=88 MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 14 02:07 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848238  OWNER=root MODE=100640
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=237 MTIME=Feb 14 03:00 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=6848243  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 14 02:07 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=7557515  OWNER=root MODE=100644
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb  3 06:36 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=7624711  OWNER=root MODE=100755
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=410434 MTIME=Feb  3 05:11 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=7719998  OWNER=root MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 17 05:00 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=7719999  OWNER=root MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 17 05:00 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: UNREF FILE I=7720000  OWNER=root MODE=100600
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=0 MTIME=Feb 17 05:00 2013 
Feb 18 02:20:18 <console.info> xts-bsd kernel: CLEAR? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: LINK COUNT DIR I=7881962  OWNER=_smtpd MODE=40700
Feb 18 02:20:18 <console.info> xts-bsd kernel: SIZE=512 MTIME=Feb 17 21:02 2013  COUNT 3 SHOULD BE 2
Feb 18 02:20:18 <console.info> xts-bsd kernel: ADJUST? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ** Phase 5 - Check Cyl groups
Feb 18 02:20:18 <console.info> xts-bsd kernel: FREE BLK COUNT(S) WRONG IN SUPERBLK
Feb 18 02:20:18 <console.info> xts-bsd kernel: SALVAGE? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: SUMMARY INFORMATION BAD
Feb 18 02:20:18 <console.info> xts-bsd kernel: SALVAGE? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: BLK(S) MISSING IN BIT MAPS
Feb 18 02:20:18 <console.info> xts-bsd kernel: SALVAGE? [yn] 
Feb 18 02:20:18 <console.info> xts-bsd kernel: 701311 files, 2730963 used, 17324212 free (37644 frags, 2160821 blocks, 0.2% fragmentation)
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ***** FILE SYSTEM MARKED CLEAN *****
Feb 18 02:20:18 <console.info> xts-bsd kernel: 
Feb 18 02:20:18 <console.info> xts-bsd kernel: ***** FILE SYSTEM WAS MODIFIED *****
Feb 18 02:20:18 <console.info> xts-bsd kernel: # return
Feb 18 02:20:18 <console.info> xts-bsd kernel: Setting hostuuid: 03020100-0504-0706-0809-0a0b0c0d0e0f.
Feb 18 02:20:18 <console.info> xts-bsd kernel: Setting hostid: 0x25736c8e.
 
Based on my reading, the code that gave me the error I saw was changed in Nov 2012. The changes were at line 8582 in the code.

http://svnweb.freebsd.org/base/head/sys/ufs/ffs/ffs_softdep.c?view=log&pathrev=242492

So I'm not sure what to make of the issue. There was barely any hard drive activity at the time. All that should have been going on is syslogd writing into /var/log at the time of the crash. It was just after 2100 on 17 Feb and when I didn't found out NFS wasn't working over 5 hours later I discovered it was in single user mode. My actions were to give a "y" to all the manual fsck actions provided. I didn't discover any files within the base system corrupted or damaged in some way when I ran freebsd-update IDS.

Any ideas?
 
SU+J is not necessary on an SSD, and disabling it can avoid some problems. That said, some of the OCZ drives are famously unreliable.

PS: post log files on pastebin.com rather than in multiple messages.
 
I've done some searching and you're right, I've come across a handful of folks that recommend disabling the journal and leaving the soft updates. I've found one that recommended disabling both. I came across your guide at the link below as well and there is certainly some good information there that I would've used when I built my setup. Have you just disabled the journal on yours? I suppose it's worth a try.

http://www.wonkity.com/~wblock/docs/html/ssd.html
 
junovitch said:
I've done some searching and you're right, I've come across a handful of folks that recommend disabling the journal and leaving the soft updates. I've found one that recommended disabling both. I came across your guide at the link below as well and there is certainly some good information there that I would've used when I built my setup. Have you just disabled the journal on yours? I suppose it's worth a try.

I don't use SU+J on anything, disks or SSD. I had problems with early versions of it, SSDs don't need it, and I use dump(8) for backup, which is locked out when SU+J is enabled.
 
Thanks, wblock@. I've gone ahead and disabled SU+J and I'll continue to monitor this a bit more closely.

Here were my actions.

Drop to Single-User Mode.
# shutdown now

Remount read-only, disable SU+J, and reboot to put into effect.
# mount -o ro /dev/ada0p2
# tunefs -j disable /dev/ada0p2
# reboot

And for good measure after the reboot I've reclaimed the space used by the journal.
# rm /.sujournal
 
Back
Top