FreeBSD 8RC2 and ZFS - slow writing

[freelsd]

Hi.
I use 8.0RC2 but have slow writing on disk:

PHP Code:

DSP         15.8T   488G    697      2  15.7M   244K
DSP         15.8T   488G    499     33  9.91M   798K
DSP         15.8T   488G     64     43  3.89M   646K
DSP         15.8T   488G     85      1  2.86M   174K 


This is really critical problem.

PHP Code:

8.0-RC2 FreeBSD 8.0-RC2 #0: Thu Nov  5 14:01:00 MSK 2009     drug@asrv31.qwarta.ru:/usr/obj/usr/src/sys/MYKERN  amd64 


zpool version 13
# cat /boot/loader.conf

PHP Code:

isp_load="YES"
isp_2400_load="YES"
kern.maxusers=2048
zfs_load="YES"
#vm.kmem_size_max="999M"// in 7.2 there are this parametrs
#vm.kmem_size="999M"
#vfs.zfs.arc_max="448M"
#vfs.zfs.prefetch_disable=1
#vfs.zfs.zil_disable=1
#vfs.zfs.cache_flush_disable=1
vm.kmem_size_max="1024M"
vm.kmem_size="1024M"
vfs.zfs.arc_max="100M" 


cat /etc/sysctl.conf

PHP Code:

security.bsd.see_other_uids=0
kern.maxvnodes=400000 


[User23]

Hm, dont know if this will help. On http://wiki.freebsd.org/ZFSTuningGuide they said:

amd64

FreeBSD 7.2+ has improved kernel memory allocation strategy and no tuning may be necessary on systems with more than 2 GB of RAM.

[freelsd]

Yea I see, without tuning and with him I have this problem. And it is critical for me.

[User23]

Did you used the 8.0 RC1 before and got any trouble? Iam using the 8.0 RC1 (without FC) with ZFS filesystem version 13 and ZFS storage pool version 13 .I dont got any bad behaviour with it. The fact that the ZFS file and pool version does not change between the RC1 and the RC2 let me believe that we have to search the problem somewhere else.

First i would try to boot without ACPI (oh i hate it ^^).
Second make a diff between RC1 and RC2 version of the drivers for your FC controller to find out if something change.

[phoenix]

How is your pool configured? What kinds of vdevs are in use (mirror, raidz1, raidz2, etc)? How many disks are in each vdev?

For raidz vdevs, you will get the write performance of a single drive, per vdev. Thus, if your pool has 1 raidz1 vdev of 5 disks, you will get the write performance of 1 disk. If your pool has 2 raidz1 vdevs of 5 disks, you will get the write performance of 2 disks. And so on.

If using raidz vdevs, do *NOT* use more than 9 disks per vdevs. You will get horrible write performance, and you will most likely be unable to replace any drive in it. Scrub and resilver operations will never finish due to disk thrashing.

The sweet spot for raidz2 vdevs seems to be 6 disks. The sweet spot for raidz1 seems to be 4 disks.

And, why are you limiting the ARC to 100 MB? Do you only have 1 GB of RAM? Depending on your workload, you should give up to 1/2 of your RAM to the ARC.

[freelsd]

No any zraids, just 1 disk in Zpool

        NAME        STATE     READ WRITE CKSUM
        DSR         ONLINE       0     0     0
          da1       ONLINE       0     0     0

[freelsd]

Originally Posted by User23

Did you used the 8.0 RC1 before and got any trouble? Iam using the 8.0 RC1 (without FC) with ZFS filesystem version 13 and ZFS storage pool version 13 .I dont got any bad behaviour with it. The fact that the ZFS file and pool version does not change between the RC1 and the RC2 let me believe that we have to search the problem somewhere else.

First i would try to boot without ACPI (oh i hate it ^^).
Second make a diff between RC1 and RC2 version of the drivers for your FC controller to find out if something change.

No, before 8.0RC2 was 7.2 .

[User23]

Uhm, let ask one stupid question...

How you figured out your write speeds are "low"?

DSP         15.8T   488G    697      2  15.7M   244K 
DSP         15.8T   488G    499     33  9.91M   798K 
DSP         15.8T   488G     64     43  3.89M   646K 
DSP         15.8T   488G     85      1  2.86M   174K

This is just the output from:

zpool iostat

like

zpool iostat
               capacity     operations    bandwidth
pool         used  avail   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
backup     2.61T  2.83T    166     33  2.61M  2.09M

right?

This output say nothing about max read/write performance! It just tell about the usage over a ? amount of time. Please use iozone or something like that to figure out your write speeds. And please post them

[freelsd]

I have 97% used ZFS.
Is it true for FreeBSD:

Keep pool space under 80% utilization to maintain pool performance. Currently, pool performance can degrade when a pool is very full and file systems are updated frequently, such as on a busy mail server. Full pools might cause a performance penalty, but no other issues. If the primary workload is immutable files (write once, never remove), then you can keep a pool in the 95-98% utilization range. Keep in mind that even with mostly static content in the 95-98% range, write, read, and resilvering performance might suffer.

http://www.solarisinternals.com/wiki...ractices_Guide