O_DIRECT - The Problem That Grew Up With Multi-Threading

Unjustified paranoia; all too common in 2025 computer science though. If you, as a programmer, cannot properly manage your buffers then you should not be allowed to program in the first place.

So there are caveats. There always have been caveats to most computer design paradigms, to spite all attempts to turn programming into an assembly line plug-n-play process.
 
A statement that caught my attention:

This isn’t a bug that can be fixed. It’s an architectural limitation that has finally caught up with us.

The article goes on to say Linus called it out in 2002!

Interestingly, O_DIRECT works in ZFS. Hmmm, I wonder why? (Sarcasm) The article tells you. +1 ZFS.
 
An old and well known problem. For example discussed here:

Considerations for the use of direct I/O (O_DIRECT)

When a file is opened in the O_DIRECT mode (direct I/O mode), GPFS transfers data directly between the user buffer and the file on the disk.
www.ibm.com

Doing data checksums or parity calculations in the kernel, while the writing user space process is still modifying the page, is clearly a no-no. I'm completely not surprised that BtrFS is broken here. As is performing the write operation in such a fashion that it can leave the system damaged and the data unreadable (what seems to happen for MDRAID). This can be implemented correctly, and the performance penalty is the price of being able to read the data back. I keep saying the same thing over and over: RAID (and redundancy features such as checksums) have to be implemented in close connection with the file system, probably within it, like ZFS does.
 
You must log in or register to reply here.
Back
Top