Free Memory (ZFS)

D

DarkenedReality

Hey Everyone,

Was just wondering if there was a way to tune and/or force the page handler to turn a large portion of inactive memory into free memory (preferably when going below a hard minimum of available free memory) ?? The reason I ask is that I'm currently running ZFS and am experiencing a drastic drop in performance when the system is running low on free memory.

Upon boot with about 5.5GB of free memory, I usually achieve ~70-100 MB/s file transfers to the zpool. ZFS quickly eats up this spare memory (just as quickly thrown into inactive) and the speed drops down to a steady 10-20 MB/s. Eventually (around 30MB of free memory) the page handler wakes up and starts freeing up inactive memory before the transfer can slow down any further.

I've read threw dozens upon dozens of pages concerning zfs issues (some that even bring up this exact issue/cause) and tried tuning it every way I can think of. Using perl to send a huge allocation request (forcing the page handler to free up some memory) seems to bring the transfer speeds back up for a short time before the free memory is exhausted again.

Here's the particulars,

Code: 
# uname -a
FreeBSD Foo. 8.1-RELEASE FreeBSD 8.1-RELEASE #0: Mon Jul 19 02:36:49 UTC 2010     root@mason.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64
Code: 
# zpool status
  pool: Luigi
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        Luigi       ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            ad14    ONLINE       0     0     0
            ad16    ONLINE       0     0     0
            ad18    ONLINE       0     0     0

errors: No known data errors
Code: 
# cat /boot/loader.conf
.......This is just the last settings I tested
vm.kmem_size="3072m"
vfs.zfs.arc_max="1024m"
vfs.zfs.prefetch_disable="1"
vfs.zfs.zio.use_uma="0"
vfs.zfs.txg.timeout="5"
vfs.zfs.vdev.max_pending=1
vfs.zfs.vdev.min_pending=1
Any help would be greatly appreciated (Sorry about the novel and the fact that its not about lasers and/or dinosaurs :P ),
Thanks,

Dark D
 
This comes up every other month or so on the mailing lists. Reading through the archives for -stable, -current, or -fs would be very informative. Many, many, many patches have floated through the lists regarding this.

8.2 should fix these issues, as the ZFSv15 patch set includes a bunch of memory usage fixes.
 
The settings listed under the loader.conf output where just the last settings I was testing. I've tried a kmem_size of 9216 (1.5x the physical 6gb as suggested on the mailing list) and with a arc_max of 5120 and every variation in between with the same results. Its not so much that zfs is eating all my memory, its more that it exhausts all the free memory faster then the page handler can recycle the inactive memory.
 
phoenix said:
This comes up every other month or so on the mailing lists. Reading through the archives for -stable, -current, or -fs would be very informative. Many, many, many patches have floated through the lists regarding this.

8.2 should fix these issues, as the ZFSv15 patch set includes a bunch of memory usage fixes.
Click to expand...

Thanks Pheonix :) I'll give them a try.

Dark D
 
You must log in or register to reply here.
Back
Top