Solved When timestamps change

Hello guys,

can you help me to understand when the timestamps of a file (creation time, last modify time, last access time) change regarding copy, move or download operations ?
And for which filesystems (fat16/32, exfat and ntfs) these changes happens ?

thanks in advance.
 
The question you are asking is at the wrong level. In Posix (generally, in Unix land), a file system object (file, directory, ...) has the three time stamps you mentioned. The Posix standard sets exact rules on the effect which file system operations have on those three time stamps. For example: creating a file (with the creat() call) sets the ctime; writing to a file with the write() call sets the mtime, and reading from a file with the read() call sets the atime. Furthermore, the utimes() system call allows a program to modify the atime and the mtime. While the rules for files seem simple at this level, it gets exceedingly complicated when taking account the times on more complex objects (directories, soft links), and for other operations (such as chmod or stat). While I have worked on implementing file systems, I do not claim that I can remember these rules by heart, so you'll just have to read the posix standard documents yourself.

The reason your questions are at the wrong level: the rules for file stamps apply to system calls. But the operations you are talking about (copy, move, download) are much more complex than individual system calls, and can be built many different ways. For example: are you using cp, scp, rsync, tar, dump, ... to perform the copy? Most of these programs have the habit of transferring file "attributes" too, and updating the resulting file to have the "correct" attributes. So knowing about the effect of individual read() and write() operations is pointless, since those type of programs will call utimes() at the end anyhow, and wipe all that stuff out.

Now, once you add Windows-style file systems into the mix, it gets seriously crazy. All three file systems you mention (fat, exfat, ntfs) come from Windows, and Windows implements some of these times differently. In particular, the creation time of a Windows file has very different semantics from the ctime of a Unix file; I know that some file systems implement that by keeping four time stamps for each object, and using complicated tricks to differentiate between the two versions of the ctime. When using windows file systems in a Unix environment, some compromises have to be made, and those compromises will be different for different file systems on different Unix implementation.

To make matters worse: atime updates can be turned off, and in many production environments are (to help with performance). And the granularity of these time stamps is different on various file systems. AFAIK, most native Unix file systems keep them with at least 1-second granularity and update them correctly, but some of the Windows file systems aren't even capable of storing them at that granularity (if I remember, right, atime is only stored with 1-day granularity on FAT), and some Windows file systems are guaranteed to have better granularity (I think NTFS inherited its time-stamp accuracy from Digital's VMS, and it is milliseconds).

And if you think you understand it now, you're still wrong: Most Unix machines are configured with the system clock running in UTC, and individual processes running in time zones. In that case, the time stamp stored in the file system is in UTC, and is being converted to local time when it is displayed. But most old-style Windows file systems (definitely fat) don't have a concept of time zone, and most real-world file systems are set in local time, so time zone becomes a mount concept rather than a process concept. This tends to lead to hilarity, like mounting the SD card from your digital camera on Unix and discovering that you've been taking lots of pictures at 3am.

So far, I haven't answered the question in the slightest. And as explained above, it is not possible to give a simple, coherent answer. I could give you an answer that is simple but wrong: If the file systems you are talking about were real Unix file systems, and if the copy/move/download operation were implemented trivially (by calling open() on the source file, creat() on the target file, and then a sequence of read() on the source and write() on the target, followed by two close() calls), then the ctime of the target file would be when it was created, and mtime and atime would be when the last byte was written to it, independent of what the source file was. But this example is pretty unrealistic.

Please restate your question more concretely: What is your real problem?
 
Hi ralph,

While I have worked on implementing file systems
I did not know you've worked on the implementation of filesystems; reading this post, this statement has escaped me. Who better than you can answer my question!

Please restate your question more concretely: What is your real problem?
You had expressed perplexity on the correctness formulation of the question and you asked me what my real problem was. You were right.
But first I needed to understand the "layered logic" and the "status concept" of a filesystem as you have masterfully explained in the post: Changes of the Timestamps.
Only now I am able to formulate the question concretely.
Relatively on what you wrote, it seems that there is no a deterministic answer or at least not always there is an answer.

QUESTION:
once you add Windows-style file systems into the mix, it gets seriously crazy. All three file systems you mention (fat, exfat, ntfs) come from Windows, and Windows implements some of these times differently.
Is there or not a 1:1 corrispondence between the FAT(16/32) filesystem drivers of the Windows O.S. and the FAT(16/32) filesystem drivers of the FreeBSD O.S. ?
It seems the answer is no, because Windows implements some of these times differently.

The same question regards the NTFS filesystem (I use the ntfs-3g drivers on my FreeBSD machine) and also the Apple HFSplus filesystem.

Please, Let me give you a concrete example:

Man 1 gives me three usb pen drives, the first fat16/32 formatted (on a Windows machine), the second one NTFS formatted (on a Windows machine), the third one HFS plus formatted (on an APPLE machine).
He asks to me to find the timestamps of the files there stored.

To do this I mount read-only (-atime) the usb sticks on /mnt and use the FreeBSD command stat -x /mnt/*.*

Are the Access Time, Modify Time and Change Time the real ones, that is those really "registered" by the Windows/Apple OSes ?
Yes, no, depends on .... "mistery" of the filesystem drivers implementation ?

I hope to hear you.

Thanks in advance.
 
Long reply at the earliest tonight; only have a few minutes this afternoon, and don't get home until 11pm this evening.
 
Oops, I just saw that I forgot to answer this question.

If you look at a modern Posix stat structure, such as the one FreeBSD uses, you see that it has room for four different time stamps (ctime, mtime, atime and birthtime), and each of those in turn is a timespec structure, which consists of an integer number of whole seconds, plus a number of nanoseconds. This is actually relatively easy to see in Python: "s = os.stat('test file')", then do a "dir(s)" which shows the four st_xxxtime fields. If you print one of the four, Python reports it in floating point (it converts the seconds + nanoseconds to a floating point number).

This means that any sensible implementation of Windows file systems (NTFS and FAT) is capable of displaying the full detail of the time stamps that NTFS and FAT keep: NTFS by default has better resolution that 1s, and both have the birthtime instead of the ctime (the birthtime is also known as the file creation time, while Unix'es ctime is the last modification time of the inode, which is often but not always the file creation time).

Apple HFS is a non-problem, since it is a Unix file system (actually closely related to and derived from *BSD's UFS and FFS file systems), and keeps the normal Unix file attributes.

Now, it is theoretically possible that an idiot has implemented these file systems in FreeBSD, and managed to garble the time stamps. I find that extremely unlikely.

So what I said above is: Yes, FreeBSD (or any other modern Posix-compliant OS) is capable of reading and displaying the time attributes of the Windows file systems in their full richness. That is obviously not a coincidence; the Posix standard was changed to allow this (in the late 90s this was not yet the case). However, the opposite is not true: The Windows file systems are not capable of actually storing the Posix time stamps completely. For example, they don't have nanosecond granularity, nor can FAT store the atime in full resolution (I think traditional FAT keeps the atime only in 1-day intervals).
 
FreeBSD (or any other modern Posix-compliant OS) is capable of reading and displaying the time attributes of the Windows file systems in their full richness. That is obviously not a coincidence; the Posix standard was changed to allow this (in the late 90s this was not yet the case). However, the opposite is not true: The Windows file systems are not capable of actually storing the Posix time stamps completely.

Very important !!! To remember !

Thanks very much ralph !

Best Regards.
 
Back
Top