Frequent "UNEXPECTED SOFT UPDATE INCONSISTENCY" errors

D

darrendavid

Hey all-

I recently built up a new home NAS using FreeBSD 10.0-RELEASE, along with a Startech hardware RAID1 configuration for 2 Samsung 128GB SSD system disks. That all sounded like an awesome idea, but I've been seeing regular "UNEXPECTED SOFT UPDATE INCONSISTENCY" errors, which cause hangs on reboot, requiring that I dig in over IPMI in single-user-mode to resolve. Meh! Previously, I was running FreeBSD 8.x and FreeBSD 9.x on an Atom motherboard for years without a single soft update issue. Some thoughts and questions:

  • While I have pretty much swapped out all hardware, the only really different thing in terms of storage is the Startech RAID1 card. Could that simply be the culprit?
  • Could there be some issue with FreeBSD 10 and some other userland software I'm running? More often than not, I'm seeing /var/log as the offending partition, FWIW.

Sadly, I kind of threw the baby out with the bathwater, so there's no single smoking gun here. Any thoughts as to where I might look to diagnose the issue would be greatly appreciated.

Thanks in advance,
Darren David
 
Boot into single user mode and turn off soft updates journaling with tunefs -j disable for each filesystem. More than a few people have had trouble with SUJ, and I do not use it at all. Its value is that it does not take as long to run fsck(8) after a power loss or other unexpected reset, but that's not a problem on SSDs anyway.

If problems continue, suspect the Startech card. Please post the output of pciconf -lv so we can identify it. It is probably a Silicon Image chipset, and some of those work acceptably well, while some do not. If the motherboard SATA ports are available, running gmirror(8) or even a ZFS mirror are generally better options than an inexpensive RAID card, more reliable and the disks are portable.
 
Are you regularly hard power cycling the system? If you shutdown cleanly everything should get flushed to disk properly. A possibility is that the RAID card is lying to the OS and the OS is getting told everything has been flushed to disk when it hasn't. If that was the issue, @wblock@'s suggestion to try software RAID or ZFS mirror would resolve it. If you need to do something to hold you over, here's an option to keep in mind. In your /etc/rc.conf:
Code: 
fsck_y_enable="YES"
This could be a dangerous assumption, some file system issues you may want to say no to and take different actions to deal with those files. If it really is just some log files like you said, it might be a safe assumption in your case. Personally, I do this because I would rather the system be online with some lost files than sitting there waiting for me to pull up it's IPMI terminal. My UFS root is just the OS and jails; all my important data is on a ZFS RAIDZ.
 
Last edited by a moderator:
Thanks for the notes, all. I'll try disabling softupdates first. I had considered ZFS mirroring my system drives, but creating a bootable ZFS mirror seemed like more work than I was willing to put in.

One question that is most likely inappropriate for this forum, but I'll ask nonethless (And please redirect me to the proper forum if so) -- it is possible to turn my hardware RAID1 setup into a ZFS mirror setup without wiping the disks and starting from scratch? I know I can use the Startech in JBOD mode, but I'd prefer a happier way to migrate from HW to SW RAID that doesn't involve losing my whole system when I delete the HW RAID.

Thanks again,
Darren
 
Yes, migrating should be possible. Back up everything first.

Move one of the drives from the Startech board to a motherboard SATA connector.
Create a new ZFS mirror on that drive. Root on ZFS, unless you boot from a different drive.
Copy the data from the old mirror to the new ZFS one.
Shut down and remove the Startech card and its one remaining drive.
Boot and test the one-drive ZFS mirror. Make sure it boots correctly and all data is present.
When everything is correct, connect the one remaining Startech drive and add it to the ZFS mirror. It will begin "resilvering" to the new drive. This is better on ZFS than on a hardware mirror, because ZFS only copies blocks in use.
 
You must log in or register to reply here.
Back
Top