FreeBSD 10-RELEASE ZFS - Data Corruption

[rmahangoe]

Hi all,

Can someone help me with this problem I'm stuck with?

I have storage with 100TB of data. My OS is FreeBSD 10.0-RELEASE + ZFS. Server is ProLiant DL360 G6. RAID set: RAID6

Problem: Corrupted Data.

The problem is: after two hard drives failed I've replaced them with two new drives and rebuilt the RAID set. But now I can't access my files on this storage.

What I've tried:

I have tried to mount storage read-only

zpool export storage & zpool import "storage"
zpool import -fFX storage
zpool scrub "storage"

But still I can't access my data on this server.

Error message:

root@amcmhp-msa600:/storage/srcmaterial/test1 # zpool status
  pool: storage
 state: ONLINE
status: One or more devices has experienced an error resulting in data
	corruption.  Applications may be affected.
action: Restore the file in question if possible.  Otherwise restore the
	entire pool from backup.
   see: http://illumos.org/msg/ZFS-8000-8A
  scan: scrub repaired 2.51G in 137h55m with 72315668 errors on Mon Jul 14 06:32:54 2014
config:

	NAME        STATE     READ WRITE CKSUM
	storage     ONLINE       0     0    16
	  da2       ONLINE       0     0    60

errors: 72315712 data errors, use '-v' for a list

 

[SirDice]

There is no error correction in your current setup. I'm assuming you are using the RAID functionality of the controller and not of ZFS? In that case you are SoL and will need to restore the file(s) from backups. You do have backups? RAID is not a substitution for backups.

ZFS is a great filesystem but to get the most out of it you need to use separate disks and not the RAID functionality of the controller. If ZFS manages everything you will have fault-tolerance and error correction. RAID controllers typically only have fault-tolerance and no error correction.