ZFS ARC cache larger then the pool itself?


Just of curiosity did anybody noticed this interesting stuff?

I got this VM with 16G RAM and 6GB disk.
The disk has only one pool - zroot and looks to me like the ARC cache takes more memory than the pool size is.

root@system:~ # zpool list
zroot  5.97G  3.84G  2.13G         -    70%    64%  1.00x  ONLINE  -
root@system:~ #

And what top reports is:
root@system:~ # top

last pid: 46132;  load averages:  0.23,  0.25,  0.19                                                                                                             up 7+05:01:55  18:55:14
32 processes:  2 running, 30 sleeping
CPU:  0.5% user,  0.0% nice,  0.5% system,  0.2% interrupt, 98.8% idle
Mem: 198M Active, 313M Inact, 13G Wired, 1931M Free
ARC: 12G Total, 2743M MFU, 9411M MRU, 272K Anon, 36M Header, 377M Other
Swap: 2048M Total, 2048M Free

Even the only MRU cache is larger than the pool so most used data is more than the pool size and not to talk about that the pool is only 70% full.

Anybody have idea is this a data duplication in the cache or somehow it is expected behavior?

Thank you.
Interesting ... would you mind to install sysutils/zfs-stats package ? and try to get detailed ARC info: zfs-stats -A

Description    :
zfs-stats displays ZFS statistics in human-readable format including
ARC, L2ARC, zfetch (DMU) and vdev cache statistics.

Done, this is the result:

root@system:~ # zfs-stats -A

ZFS Subsystem Report               Sun Jan  8 20:40:52 2017

ARC Summary: (HEALTHY)
   Memory Throttle Count:           0

ARC Misc:
   Deleted:               7
   Recycle Misses:               0
   Mutex Misses:               0
   Evict Skips:               1

ARC Size:               85.39%   12.41   GiB
   Target Size: (Adaptive)       100.00%   14.53   GiB
   Min Size (Hard Limit):       12.50%   1.82   GiB
   Max Size (High Water):       8:1   14.53   GiB

ARC Size Breakdown:
   Recently Used Cache Size:   64.49%   9.37   GiB
   Frequently Used Cache Size:   35.51%   5.16   GiB

ARC Hash Breakdown:
   Elements Max:               132.54k
   Elements Current:       99.74%   132.20k
   Collisions:               310.27k
   Chain Max:               4
   Chains:                   4.01k


Bug submitted: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=216364
Last edited:
well the bug report says is closed because is not a bug .. so is a feature ? Thats really nice when something that is pretty bad become something we want
Well, Andriy Gapon may be right about the compression of the data on the disk and it being uncompressed in the cache.
It doesn't matter now because since 11.1-RELEASE we have now compressed data stored in the cache.