Hello, I have a server running FreeBSD 9-current built on March 23rd (generic kernel) with a 3TB striped zpool (6x500gb) drives (used for mirror space). Up until recently this has been okay, with some minimal issues, however last week the machine became unresponsive. I was able to make TCP connections and get ICMP echo responses, but no CLI would come up.
Once the machine rebooted it would no longer mount my ZFS filesystems. I booted into single user mode, was able to mount one of the filesystems (tank/varlog) but the important one (tank/ftp 1.5TB used) would not come up and the system 'froze' again. A zpool scrub came up clean. Rebooted and left the machine to turn and see what would happen if anything. At first I received and error for "not enough space to start" or similar.
I have now disabled ZFS (zfs_enable=NO) and tried over the weekend to mount the offending filesystem and now have error "aacd0 hard error cmd=read", when mounting top(1) was running and would show the zfs process change between tx->tx and zio-> states.
I assume the hard error means one of the hard drives went bad but maybe somebody here would be able to help in case I'm missing something.
Also, Dedup is on for the pool and compression=gzip for tank/varlog only. Nothing out of the ordinary shows in zpool history.
Once the machine rebooted it would no longer mount my ZFS filesystems. I booted into single user mode, was able to mount one of the filesystems (tank/varlog) but the important one (tank/ftp 1.5TB used) would not come up and the system 'froze' again. A zpool scrub came up clean. Rebooted and left the machine to turn and see what would happen if anything. At first I received and error for "not enough space to start" or similar.
I have now disabled ZFS (zfs_enable=NO) and tried over the weekend to mount the offending filesystem and now have error "aacd0 hard error cmd=read", when mounting top(1) was running and would show the zfs process change between tx->tx and zio-> states.
I assume the hard error means one of the hard drives went bad but maybe somebody here would be able to help in case I'm missing something.
Also, Dedup is on for the pool and compression=gzip for tank/varlog only. Nothing out of the ordinary shows in zpool history.
Code:
FreeBSD 9.0-CURRENT #1: Thu Mar 24 08:26:39 UTC 2011
nick@:/usr/obj/usr/src/sys/GENERIC amd64
WARNING: WITNESS option enabled, expect reduced performance.
CPU: Intel(R) Pentium(R) D CPU 3.00GHz (3010.71-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0xf44 Family = f Model = 4 Stepping = 4
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x649d<SSE3,DTES64,MON,DS_CPL,EST,CNXT-ID,CX16,xTPR>
AMD Features=0x20000800<SYSCALL,LM>
TSC: P-state invariant
real memory = 2147483648 (2048 MB)
avail memory = 2046611456 (1951 MB)
Code:
zpool status
pool: tank
state: ONLINE
scan: scrub repaired 0 in 8h15m with 0 errors on Sat Apr 9 02:08:20 2011
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
aacd0 ONLINE 0 0 0
aacd1 ONLINE 0 0 0
aacd2 ONLINE 0 0 0
aacd3 ONLINE 0 0 0
ad8 ONLINE 0 0 0
ad10 ONLINE 0 0 0
errors: No known data errors
zfs list
NAME USED AVAIL REFER MOUNTPOINT
tank 1.38T 1.36T 31K none
tank/ftp 1.37T 1.36T 1.37T /ftp
tank/varlog 50.0M 9.95G 50.0M /var/log