Wow activity here again; I was worried that I had inadvertently killed this thread with my last reply. I had stopped checking at one point so I didn't see until now see there is discussion here once more.
To clarify my comparisons of kqueue to inotify/fsevents I'll give a quick and dirty overview of the process for monitoring a file-system tree:
Setup:
With kqueue, the user-space developer must:
- Enumerate the entire directory structure recursively
- Open an FD for every directory and file
- Add accounting to lookup by FD as well as by path (needed later)
With inotify:
- Create inotify context
- Enumerate the entire directory structure recursively
- Register a watch in the context for every directory (not files)
- Add accounting to lookup path by watch id (needed later)
With fsevents:
- Register a single watch for the directory tree (no recursion)
Watching for changes:
With kqueue:
- Loop over kevent giving it a giant array of fds from the accounting to get list of FDs changed
- For each FD indicating change:
- Lookup in accounting by FD
- If a directory changed, scan dir to figure out what the change was
- Use the by path accounting to determine file/directory additions and removals
- If a file/directory was added, recursively add them (as in setup above)
- If a file/directory was removed, recursively remove them from accounting and close the fds
With inotify:
- Loop over read of inotify context
- For each object read:
- Lookup the path by watch id (provided in object) and add file/dir name (also provided) to get full path of change
- Take action on the path in the object
- If a directory was added, recursively add directories (as in setup above)
- If a directory was removed, recursively remove directories from accounting and unregister with the context
With fsevents:
- Read the path of the updated file/directory given in the callback
- Take action on this path
I hope with the above it makes it more clear why I'd prefer something akin to the fsevents API oven inotify as it's kinder on the user-land developer. With fsevents there is one object to open per monitored directory tree and it gives rich contextual information in the callback. The only accounting needed by the user-land program is which directory trees are monitored and close that monitor if the entire directory tree disappears. With inotify the user-land program must have accounting for every directory, but at least the notification there yields the path of what was modified or added and what the change in the FS was. The most work is kqueue since the user-land program only has the fd of the FS change and no other contextual information. This requires all the accounting I mentioned above.
As per the rationale for having FS monitoring, I can attest to the difference myself. I used to be running ZFS on Ubuntu and I transitioned over to FreeNAS. When I was on Ubuntu, if I rsynced a file over to my media directories, within seconds it was detected, scanned, and playable on the Plex client running on the TV. When I changed to FreeNAS, I had to wait for a periodic scan to pick up the new file or manually kick it off myself. I truly missed the automatic FS monitoring. At the moment, I'm currently transitioning my media storage/server back to ZFS on Ubuntu and when I would copy a file to a media pool on the Ubuntu side, it'd show up in Plex in seconds once more without any interaction on my part. It was quite satisfying to automatic FS change detection once more and reminded me of how much I missed it.
So I'll appeal for a new FS change API that gives rich callback information, such as path of the change, what the change was (add/modify/delete), and this be done on a directory-tree as a whole.
