Hi there! My first post - greatly appreciate your help in advance (and please be kind 
) ...
I have been running a headless server at home for ~5 years now which acts as a fileserver/mediaserver and generally gives me an excuse to play with FreeBSD. The server was originally a FreeNAS box but I moved to FreeBSD probably ~3 years ago (and had no problems importing the various ZFS pools).
The problem is that the computer has always been really, really slow. I'm not sure if it was that way at the outset but it certainly was before I switched to FreeBSD and it still is today.
By slow I mean it takes ~25-30 minutes from power on to the login screen. The console (whether I plug a keyboard/display directly to the machine or ssh from a windows machine) is also really unresponsive. Simple commands like
Over the years, I have read various posts here and experimented with the solutions of others to no avail. A month ago I decided to upgrade the hardware thinking that could be the culprit. The original machine had a relatively low powered combo motherboard/CPU (ASRock E350M1). I have now gone to an intel skylake setup (intel i7 6700 on a MSI B150I motherboard, 32GB ram) but the problem persists.
Notably, since the problems emerged I have replaced the cpu, motherboard, ram and the hard drive that hosts the root files. All that is really left are the 4x 2TB HDD's that host the zfs fileserver (which leads me to think that the problem is somewhere there).
I appreciate that this is a pretty general problem so please let me know what further information I can provide that might help you diagnose the issue.
Again greatly appreciate any help people can offer. Thanks - Simon
Here is a dump of a few things that may be useful...
See attached file...
) ...I have been running a headless server at home for ~5 years now which acts as a fileserver/mediaserver and generally gives me an excuse to play with FreeBSD. The server was originally a FreeNAS box but I moved to FreeBSD probably ~3 years ago (and had no problems importing the various ZFS pools).
The problem is that the computer has always been really, really slow. I'm not sure if it was that way at the outset but it certainly was before I switched to FreeBSD and it still is today.
By slow I mean it takes ~25-30 minutes from power on to the login screen. The console (whether I plug a keyboard/display directly to the machine or ssh from a windows machine) is also really unresponsive. Simple commands like
ls -l often hang for ~10 seconds before they produce output. Interestingly, sometimes the responsiveness issues disappear completely and the machine becomes lightning fast (for a period).Over the years, I have read various posts here and experimented with the solutions of others to no avail. A month ago I decided to upgrade the hardware thinking that could be the culprit. The original machine had a relatively low powered combo motherboard/CPU (ASRock E350M1). I have now gone to an intel skylake setup (intel i7 6700 on a MSI B150I motherboard, 32GB ram) but the problem persists.
Notably, since the problems emerged I have replaced the cpu, motherboard, ram and the hard drive that hosts the root files. All that is really left are the 4x 2TB HDD's that host the zfs fileserver (which leads me to think that the problem is somewhere there).
I appreciate that this is a pretty general problem so please let me know what further information I can provide that might help you diagnose the issue.
Again greatly appreciate any help people can offer. Thanks - Simon
Here is a dump of a few things that may be useful...
uname -rp
Code:
11.0-RELEASE-p1 amd64 dmesg
Code:
Copyright (c) 1992-2016 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-RELEASE-p1 #0 r306420: Thu Sep 29 01:43:23 UTC 2016
root@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
FreeBSD clang version 3.8.0 (tags/RELEASE_380/final 262564) (based on LLVM 3.8.0)
VT(efifb): resolution 800x600
CPU: Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (3408.17-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x506e3 Family=0x6 Model=0x5e Stepping=3
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
Features2=0x7ffafbff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
AMD Features2=0x121<LAHF,ABM,Prefetch>
Structured Extended Features=0x29c6fbf<FSGSBASE,TSCADJ,SGX,BMI1,HLE,AVX2,SMEP,BMI2,ERMS,INVPCID,RTM,NFPUSG,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PROCTRACE>
XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 34359738368 (32768 MB)
avail memory = 33129832448 (31595 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s) x 2 hardware threads
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-119 on motherboard
random: entropy device external interface
kbd1 at kbdmux0
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff8101c950, 0) error 19
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
cryptosoft0: <software crypto> on motherboard
acpi0: <ALASKA A M I > on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 24000000 Hz quality 950
Event timer "HPET" frequency 24000000 Hz quality 550
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1808-0x180b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
ahci0: <ASMedia ASM1061 AHCI SATA controller> port 0xe050-0xe057,0xe040-0xe043,0xe030-0xe037,0xe020-0xe023,0xe000-0xe01f mem 0xdf210000-0xdf2101ff irq 16 at device 0.0 on pci1
ahci0: AHCI v1.20 with 2 6Gbps ports, Port Multiplier supported
ahcich0: <AHCI channel> at channel 0 on ahci0
ahcich1: <AHCI channel> at channel 1 on ahci0
vgapci0: <VGA-compatible display> port 0xf000-0xf03f mem 0xde000000-0xdeffffff,0xc0000000-0xcfffffff irq 16 at device 2.0 on pci0
vgapci0: Boot video device
xhci0: <Intel Sunrise Point USB 3.0 controller> mem 0xdf310000-0xdf31ffff irq 16 at device 20.0 on pci0
xhci0: 32 bytes context size, 64-bit DMA
usbus0 on xhci0
pci0: <simple comms> at device 22.0 (no driver attached)
ahci1: <Intel Sunrise Point AHCI SATA controller> port 0xf090-0xf097,0xf080-0xf083,0xf060-0xf07f mem 0xdf328000-0xdf329fff,0xdf32d000-0xdf32d0ff,0xdf32c000-0xdf32c7ff irq 16 at device 23.0 on pci0
ahci1: AHCI v1.31 with 6 6Gbps ports, Port Multiplier not supported
ahcich2: <AHCI channel> at channel 0 on ahci1
ahcich3: <AHCI channel> at channel 1 on ahci1
ahcich4: <AHCI channel> at channel 2 on ahci1
ahcich5: <AHCI channel> at channel 3 on ahci1
ahcich6: <AHCI channel> at channel 4 on ahci1
ahcich7: <AHCI channel> at channel 5 on ahci1
ahciem0: <AHCI enclosure management bridge> on ahci1
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 29.0 on pci0
pci2: <ACPI PCI bus> on pcib2
re0: <RealTek 8168/8111 B/C/CP/D/DP/E/F/G PCIe Gigabit Ethernet> port 0xd000-0xd0ff mem 0xdf104000-0xdf104fff,0xdf100000-0xdf103fff irq 16 at device 0.0 on pci2
re0: Using 1 MSI-X message
re0: Chip rev. 0x54000000
re0: MAC rev. 0x00100000
miibus0: <MII bus> on re0
rgephy0: <RTL8251 1000BASE-T media interface> PHY 1 on miibus0
rgephy0: none, 10baseT, 10baseT-FDX, 10baseT-FDX-flow, 100baseTX, 100baseTX-FDX, 100baseTX-FDX-flow, 1000baseT-FDX, 1000baseT-FDX-master, 1000baseT-FDX-flow, 1000baseT-FDX-flow-master, auto, auto-flow
re0: Using defaults for TSO: 65518/35/2048
re0: Ethernet address: 4c:cc:6a:68:54:ea
re0: netmap queues/slots: TX 1/256, RX 1/256
pcib3: <ACPI PCI-PCI bridge> irq 17 at device 29.1 on pci0
pci3: <ACPI PCI bus> on pcib3
pci3: <network> at device 0.0 (no driver attached)
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
pci0: <memory> at device 31.2 (no driver attached)
hdac0: <Intel Sunrise Point HDA Controller> mem 0xdf320000-0xdf323fff,0xdf300000-0xdf30ffff irq 16 at device 31.3 on pci0
acpi_button0: <Sleep Button> on acpi0
acpi_button1: <Power Button> on acpi0
acpi_tz0: <Thermal Zone> on acpi0
acpi_tz1: <Thermal Zone> on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcffff,0xd0000-0xd7fff on isa0
ppc0: cannot reserve I/O port range
uart1: <16550 or compatible> at port 0x2f8 irq 3 on isa0
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
est4: <Enhanced SpeedStep Frequency Control> on cpu4
est5: <Enhanced SpeedStep Frequency Control> on cpu5
est6: <Enhanced SpeedStep Frequency Control> on cpu6
est7: <Enhanced SpeedStep Frequency Control> on cpu7
usbus0: 5.0Gbps Super Speed USB v3.0
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
nvme cam probe device init
hdacc0: <Realtek ALC887 HDA CODEC> at cad 0 on hdac0
hdaa0: <Realtek ALC887 Audio Function Group> at nid 1 on hdacc0
pcm0: <Realtek ALC887 (Rear Analog)> at nid 20 and 24,26 on hdaa0
pcm1: <Realtek ALC887 (Front Analog)> at nid 27 and 25 on hdaa0
hdacc1: <Intel (0x2809) HDA CODEC> at cad 2 on hdac0
hdaa1: <Intel (0x2809) Audio Function Group> at nid 1 on hdacc1
pcm2: <Intel (0x2809) (HDMI/DP 8ch)> at nid 3 on hdaa1
ugen0.1: <0x8086> at usbus0
uhub0: <0x8086 XHCI root HUB, class 9/0, rev 3.00/1.00, addr 1> on usbus0
uhub0: 22 ports with 22 removable, self powered
ugen0.2: <vendor 0x04d9> at usbus0
ukbd0: <vendor 0x04d9 USB Keyboard, class 0/0, rev 1.10/1.01, addr 1> on usbus0
kbd2 at ukbd0
usb_alloc_device: set address 3 failed (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
ses0 at ahciem0 bus 0 scbus8 target 0 lun 0
ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
ada0 at ahcich1 bus 0 scbus1 target 0 lun 0
ada0: <WDC WD10EADS-00M2B0 01.00A01> ATA8-ACS SATA 2.x device
ada0: Serial Number WD-WCAV54351888
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada0: Command Queueing enabled
ada0: 953869MB (1953525168 512 byte sectors)
ada1 at ahcich2 bus 0 scbus2 target 0 lun 0
ada1: <WDC WD20EARX-00PASB0 51.0AB51> ATA8-ACS SATA 3.x device
ada1: Serial Number WD-WCAZAH393502
ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada1: Command Queueing enabled
ada1: 1907729MB (3907029168 512 byte sectors)
ada1: quirks=0x1<4K>
ada2 at ahcich3 bus 0 scbus3 target 0 lun 0
ada2: <WDC WD20EARX-008FB0 51.0AB51> ATA8-ACS SATA 3.x device
ada2: Serial Number WD-WCAZAJ816862
ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada2: Command Queueing enabled
ada2: 1907729MB (3907029168 512 byte sectors)
ada2: quirks=0x1<4K>
ada3 at ahcich4 bus 0 scbus4 target 0 lun 0
ada3: <WDC WD20EARX-008FB0 51.0AB51> ATA8-ACS SATA 3.x device
ada3: Serial Number WD-WCAZAJ781220
ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada3: Command Queueing enabled
ada3: 1907729MB (3907029168 512 byte sectors)
ada3: quirks=0x1<4K>
ada4 at ahcich5 bus 0 scbus5 target 0 lun 0
ada4: <WDC WD20EZRZ-00Z5HB0 80.00A80> ACS-2 ATA SATA 3.x device
ada4: Serial Number WD-WMC4N0L02HTN
ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes)
ada4: Command Queueing enabled
ada4: 1907729MB (3907029168 512 byte sectors)
SMP: AP CPU #1 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #6 Launched!
Timecounter "TSC-low" frequency 1704084402 Hz quality 1000
Trying to mount root from zfs:zroot/ROOT/default []...
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
usbd_req_re_enumerate: addr=3, set address failed! (USB_ERR_TIMEOUT, ignored)
usbd_setup_device_desc: getting device descriptor at addr 3 failed, USB_ERR_TIMEOUT
ugen0.3: <Unknown> at usbus0 (disconnected)
uhub_reattach_port: could not allocate new device
re0: link state changed to DOWN
re0: link state changed to UP
warning: KLD '/boot/kernel/uhid.ko' is newer than the linker.hints file
uhid0: <vendor 0x04d9 USB Keyboard, class 0/0, rev 1.10/1.01, addr 1> on usbus0 zpool list
Code:
NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT
Popper 7.25T 5.43T 1.82T - 22% 74% 1.00x ONLINE -
zroot 864G 4.18G 860G - 0% 0% 1.00x ONLINE - zpool status alerts an error (this is the first time this has come up and I haven't been able to eliminate it yet but I don't believe this is contributing to the overall problem)
Code:
pool: Popper
state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
the pool may no longer be accessible by software that does not support
the features. See zpool-features(7) for details.
scan: resilvered 32K in 0h0m with 0 errors on Sun Jan 8 19:32:39 2017
config:
NAME STATE READ WRITE CKSUM
Popper ONLINE 0 0 0
raidz1-0 ONLINE 0 0 0
ada4p2 ONLINE 0 0 0
ada2p2 ONLINE 0 0 0
ada3p2 ONLINE 0 0 0
ada1p2 ONLINE 0 0 0
errors: No known data errors
pool: zroot
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://illumos.org/msg/ZFS-8000-8A
scan: none requested
config:
NAME STATE READ WRITE CKSUM
zroot ONLINE 0 0 0
ada0p3 ONLINE 0 0 0
errors: 1 data errors, use '-v' for a list sysinfo -aSee attached file...
