Hi,
I also tried my luck with ZFS on
FreeBSD - 9.0-RELEASE
Intel® Core™ i5 CPU 760 @ 2.80GHz (4xCore)
64 bit
8GB RAM
4GB file based swap
Now, obviously one can read lots of things, tips and tweaks for FreeBSD System in order to use ZFS "proper". Now, some of those things you'll fall over the most when reading are basically the following four sysctl nodes in order to get a stable system without kernel panics:
Please correct me if I'm wrong here but as far as I understood those variables is that vfs.zfs.arc_* sets the frame/boundaries for the Layer 1 cache of ZFS. vfs.zfs.arc_min is what system should always keep full while vfs.zfs.arc_max represents buffer for higher loads. Also this Layer 1 cache depends and acts within the given frame of vm.kmem_size.
Basically to be on the same page here
Now some people advice you to give vm.kmem_size upto 16GB on a 8GB RAM system. This sounds absurd to me since system can not even make use of 16GB since there is only 8GB RAM installed?! Or would this mean you also need a 8GB swap in order to not run out of pages here?
Basically my intuition tells me to let vm.kmem_size = Installed RAM. In fact I figured FreeBSD automatically defaults vm.kmem_size to 94% of hw.realmem - which sounds healthy - while the default vm.kmem_size_max = 314572.8 MB just doesn't sound healthy to me?! Why such a big number? Where should all this be stored in such a "worst case scenario"? Wouldn't it sound more reasonalbe to let vm.kmem_size_max = 94% of hw.realmem PLUS the amount of currently setup swap file's/drive's size?
Is there any reasons against my theory?
Anyway, after some tests I figured to get kernel panics under control by decreasing vfs.zfs.arc_max to 4G (50% of my RAM) and at the same time increase vfs.zfs.arc_min to 1G (12.5% of my RAM).
Now fine, I don't have kernel Panics anymore (right now) - but note that at the same time there is a samba DC, apache, and some other services running on the system as well - and frankly I'm afraid that the system may run into a kernel panic again as soon as those service get some higher load ... ;/ - which is totally unacceptable for a productive system!
This doesn't give me butterflies in my stomach - in fact it makes me feel like FreeBSD's memory management is not well enough off to deal with high load, since I expect it to let a lower prio app wait until the higher prio one finished its work?! But I'm probably just missing out on the right sysctl variables to restrict/prevent kernel panics at all. Is there any suggestions for this based on facts? Or how Can I make sure other services don't disturb ZFS and the other way around and at the same time make full use of given RAM resources?
So how do we milk this cow?
Thanks
I also tried my luck with ZFS on
FreeBSD - 9.0-RELEASE
Intel® Core™ i5 CPU 760 @ 2.80GHz (4xCore)
64 bit
8GB RAM
4GB file based swap
Now, obviously one can read lots of things, tips and tweaks for FreeBSD System in order to use ZFS "proper". Now, some of those things you'll fall over the most when reading are basically the following four sysctl nodes in order to get a stable system without kernel panics:
- vm.kmem_size Size of kernel memory
- vm.kmem_size_max Maximum size of kernel memory
- vfs.zfs.arc_min Minimum ARC size
- vfs.zfs.arc_max Maximum ARC size
- vm.kmem_size 7921.0 MB
- vm.kmem_size_max 314572.8 MB
- vfs.zfs.arc_min 862.1 MB
- vfs.zfs.arc_max 6897.0 MB
Please correct me if I'm wrong here but as far as I understood those variables is that vfs.zfs.arc_* sets the frame/boundaries for the Layer 1 cache of ZFS. vfs.zfs.arc_min is what system should always keep full while vfs.zfs.arc_max represents buffer for higher loads. Also this Layer 1 cache depends and acts within the given frame of vm.kmem_size.
Basically to be on the same page here
- Layer 1 cache ==> slice of system RAM
- Layer 2 cache ==> Additional Storage Medium (i.e. SSD)
Now some people advice you to give vm.kmem_size upto 16GB on a 8GB RAM system. This sounds absurd to me since system can not even make use of 16GB since there is only 8GB RAM installed?! Or would this mean you also need a 8GB swap in order to not run out of pages here?
Basically my intuition tells me to let vm.kmem_size = Installed RAM. In fact I figured FreeBSD automatically defaults vm.kmem_size to 94% of hw.realmem - which sounds healthy - while the default vm.kmem_size_max = 314572.8 MB just doesn't sound healthy to me?! Why such a big number? Where should all this be stored in such a "worst case scenario"? Wouldn't it sound more reasonalbe to let vm.kmem_size_max = 94% of hw.realmem PLUS the amount of currently setup swap file's/drive's size?
Is there any reasons against my theory?
Anyway, after some tests I figured to get kernel panics under control by decreasing vfs.zfs.arc_max to 4G (50% of my RAM) and at the same time increase vfs.zfs.arc_min to 1G (12.5% of my RAM).
Now fine, I don't have kernel Panics anymore (right now) - but note that at the same time there is a samba DC, apache, and some other services running on the system as well - and frankly I'm afraid that the system may run into a kernel panic again as soon as those service get some higher load ... ;/ - which is totally unacceptable for a productive system!
This doesn't give me butterflies in my stomach - in fact it makes me feel like FreeBSD's memory management is not well enough off to deal with high load, since I expect it to let a lower prio app wait until the higher prio one finished its work?! But I'm probably just missing out on the right sysctl variables to restrict/prevent kernel panics at all. Is there any suggestions for this based on facts? Or how Can I make sure other services don't disturb ZFS and the other way around and at the same time make full use of given RAM resources?
So how do we milk this cow?
Thanks
