'du' behavior

[ANOKNUSA]

Or are you telling me that I have to insert a sleep(10) or something like that before to use du ? Is that what you mean ?

Yes. One second was actually enough when I tested it. du is simply being executed after the ZFS userland process finishes, but before ZFS is able to complete its background tasks.

 

[ASX]

Yes. One second was actually enough when I tested it.

The problem I see with this approach is to determine how much time ZFS require to completed its background taks.

Therefore an arbitrary sleep could not be enough in certain situations, in short a very weak workaround.

Using du -A would be an alternative if there is no use of sparse-file.

 

[ASX]

I'm adding here, and may be I should open a new PR about ls(1)

ls -s show the same behavior of du, i.e. may report an incorrect number of used blocks.

The nice things is that both commands works with some other flags, ls -l always provide the correct filesize so does du -A (other than the sparse-file case).

 

[phoenix]

This is why, if you search zfs-related mailing lists, they all recommend ignoring du and df when using ZFS as they won't give you the results you expect.

Instead, use zfs list. That will show you the correct information.

If you search the zfs-discuss mailing list archives, there's several long-posts that describe how du, df, zfs list, and zpool list interact and why they show different values. I've also made similar postings to the freebsd-fs list years ago.

Basically, the fs-related tools of the past don't work on the next-gen fs like zfs and btrfs. Stop relying on them, and use the fs-related specific tools.

 

[ASX]

I really appreciate the discussion, even if I need to disagree.

Basically, the fs-related tools of the past don't work on the next-gen fs like zfs and btrfs. Stop relying on them, and use the fs-related specific tools.

What happen then to shell scripts compatibility ? Assert something like that is likely saying to throw away the old scripts and rewrite all from scratch.

And the worst is the case when you may not notice the problem at all.

What I find surprising, is that from one side 'du' is considered "unreliable", as you suggest, and from another side it is difficult to get it recognized as a bug.

Instead, use zfs list. That will show you the correct information.

If the above is true, I see no reason why the underlying code (zfs command) could not be integrated in 'du' to get the correct result.

 

[kpa]

I would be pointless to change du(1) to recognize different filesystems, that's the wrong way to fix the problem. The du(1) command is really nothing else but a simple frontend that trusts that the stat(2) etc. system calls return correct information, it should left as it is. Where the "problem" should be fixed is the VFS layer that works to present all the different filesystems in a uniform way, that's the source of this problem and if there is a workable solution at all it should be done at the VFS layer.

 

[ASX]

kpa, I agree. I wrote "if the above is true".

 

[ASX]

To clarify and avoid misunderstandings:

du may be formally correct, in that it show exactly the used blocks at a specific point in time, but the fact is that for any practical purpose this behavior is meaningless.

The use of du has always been about looking for "used blocks", being irrelevant if the blocks have been already written or not because of the underlying I/O subsystem.