ZFS Does ZFS copy a file twice with mmap?

rbranco

rbranco

I'm finally reading the Second Edition of "The Design and Implementation of the FreeBSD Operating System" and found this passsage:

> The ZFS filesystem integrated from OpenSolaris is the one exception to the integrated buffer cache. ZFS has its own set of memory that it manages by itself. Files that are mmap’ed from ZFS must be copied to the virtual-memory managed memory. In addition to requiring two copies of the file in memory, extra copying occurs every time an mmap’ed ZFS file is being accessed through the read and write interfaces.

Is this still the case? I thought this was an issue only on Linux.

How can I test mmap performance vs UFS2?
 
rbranco said:
I'm finally reading the Second Edition of "The Design and Implementation of the FreeBSD Operating System" and found this passsage:

> The ZFS filesystem integrated from OpenSolaris is the one exception to the integrated buffer cache. ZFS has its own set of memory that it manages by itself. Files that are mmap’ed from ZFS must be copied to the virtual-memory managed memory. In addition to requiring two copies of the file in memory, extra copying occurs every time an mmap’ed ZFS file is being accessed through the read and write interfaces.

Is this still the case? I thought this was an issue only on Linux.
Click to expand...

Still the case everywhere with ZFS except on Solaris.

But I think read might not be affected. The comments in the code say write only.

How can I test mmap performance vs UFS2?
Click to expand...

For readonly seeks:
github.com

GitHub - cracauer/cseekbench: Filesystem and raw device seek benchmark

Filesystem and raw device seek benchmark. Contribute to cracauer/cseekbench development by creating an account on GitHub.
github.com github.com

You can easily modify it to write instead, but you probably need the odd fsync(2) spread out.
 
cracauer@ said:
But I think read might not be affected. The comments in the code say write only.
Click to expand...

I finished reading chapter 10 about ZFS:

> • ZFS caches its data in its ARC that is not part of the unified-memory cache managed by the virtual memory. The result is that when mmap is used on a ZFS file, read faults are first read into the ARC and then copied to a page in the unified-memory cache. When dirty unified-memory pages are flushed, they must be copied to an ARC buffer and then written by ZFS. To ensure coherency whenever mmap has been used on a ZFS file, reads and writes to that file need to check whether, for each page in the transfer, the requested page is present in the unified-memory cache and, if present, use that page to do the I/O. If the page is not present in the unified-memory cache, then the I/O can proceed normally from the ARC. This approach provides coherency between memory-mapped and I/O access at the expense of wasted memory due to having two copies of the file in memory and extra overhead caused by the need to copy the contents between the two copies.
> Similarly, when using sendfile on a file in ZFS, it must be copied from the ARC to the unified-memory cache thereby losing the benefits of the no-copy semantics of sendfile. The primary use of sendfile is by Web-server applications like Apache.
> Integrating ZFS’s ARC into the unified-memory cache would require massive changes.
 
rbranco said:
The result is that when mmap is used on a ZFS file, read faults are first read into the ARC and then copied to a page in the unified-memory cache.
Click to expand...
Besides as discussed in the book, Kirk McKusick also mentions these different paths in (from ca. 2:40 min):
A new addition as of OpenZFS 2.3 (upcoming for FreeBSD) is direct IO, where the ZFS ARC is bypassed—mostly.

___
* Edit: the slides link seems to have died there. Gunion slides are available on the EuroBSDCon 2023 page:
eurobsdcon2023-kirk_mckusick-gunion.pdf
 
You must log in or register to reply here.
Back
Top