Very slow copy speed

hornyakb

hornyakb

I'm new at FreeBSD. I have recently built my new NAS based on FreeBSD. My hardware configuration: G640, 8 GB RAM, M1015. It contains 10 x 3 TB HDD (RAID-Z2) plus 500 GB (UFS) for system.

bonnie++ results look fine. ZFS 400 MB/s write, 930 MB/s read. System HDD 70 MB/s read/write.

But, if I try to copy a file from the system HDD (which contains my downloads) to the ZFS, then I can only achieve 7-8 MB/s. I have tried some network copy too. ZFS is ok, but I can copy only at about 7-8 MB/s from the download HDD. The system HDD is connected to the SATA2 interface of the motherboard (GA-H77-DS3H).

I don't think this is the limit. Any suggestions?

Thanks in advance.
 
What does the memory usage say during copy? ZFS is known to suffer when other file systems (UFS/MSDOS/...) are being used as it will shrink it's ARC when these want to cache data.
 
Code: 
CPU:  0.8% user,  0.0% nice,  1.6% system,  0.0% interrupt, 97.6% idle
Mem: 153M Active, 4630M Inact, 2553M Wired, 141M Cache, 796M Buf, 130M Free
Swap: 4096M Total, 1300K Used, 4095M Free

zdb | grep ashift
Code: 
ashift: 12
 
ashift is ZFS block size, not alignment. Please show the output of gpart show. Or if ZFS is on the bare drives without partitions, please confirm that.
 
Code: 
=>        34  5860533101  da0  GPT  (2.7T)
          34           6       - free -  (3.0k)
          40  5860300000    1  freebsd-zfs  (2.7T)
  5860300040      233095       - free -  (113M)
This is only one drive, I assume, you don't need all.

Thanks in advance.
 
Well, this looks pretty well aligned to me.

The memory information should be monitored during the copy process. When I copy data from any external device (USB disk, camera, ...) to my server the memory available to ZFS shrinks as Inactive grows - pushing against the ARC and making ZFS crawl. So if your copy process starts out fast and gets slower when the inactive memory starts pushing against wired - that may be the problem. You may try to unmount/remount your downloads and then continue. Or maybe you can simply mv those downloads to ZFS - that will delete the files and thus also free the inactive memory used to cache it.

You may also try to copy these downloads by network from a different box. In case of me syncing my camera to the server it is faster to connect it to my laptop and rsync the USB device using SSH.
 
I don't think it's memory related problem. 8 GB memory should be enough, and I have monitored memory usage during copy. Nothing unusual.

Otherwise, if I copy files to the ZFS over the network, then the copy speed is about 80 MB/s. The read speed (over network) from the 500 GB drive is slow too (around 7-8 MB/s). That's why I think that the HDD is the bottleneck, but I don't think that is a hardware problem. I have checked it under a different operating system and the read/write speed was 70 MB/s.
 
Do you, perhaps, have error messages in the dmesg? Maybe you need to switch off AHCI on that drive. But then again, your starting post states that bonnie++ gives you 70 MB/s on that drive.
 
Good - now try to run bonnie++ in parallel, one instance on UFS and the other instance on the ZFS pool.

The reason for that test is that the test files get deleted after they are no longer used, freeing the associated cache memory also. When using these in parallel, you can see if both file systems compete for memory.
 
You must log in or register to reply here.
Back
Top