Missing inotify events (in .git directory)

I may speculate that Git most of the time uses atomic file updates which are done like this:

1. A file's contents is read into memory (and modified).
2. The modified contents is written into a separate file (usually located in the same directory as the original one, and having a randomized (mktemp-style) name.
3. The new file is then rename(2)d -d over the original one; this operation guarantees that every observer trying to open the file using its name will get either the old contents or the new one.

Such updates are seen by inotify(7) as moved_to events—since a file "reappears" in a directory.

To answer your question separately for git 2.24.1 on Linux 4.19.95:

• Why are these events missing on these files?

You don't see IN_MODIFY/IN_CLOSE_WRITE events because git clone will always try to use hard links for files under the .git/objects directory. When cloning over the network or across file system boundaries, these events will appear again.

• What can be done about it? Specifically, how can I respond to the completion of writes to these files? Note: ideally I would like to respond when writing is "finished" to avoid needlessly/(incorrectly) uploading "unfinished" writing.

In order to catch modification of hard links you have to set up a handler for the inotify CREATE event which follows and keeps track of those links. Please note that a simple CREATE can also mean that a nonempty file was created. Then, on IN_MODIFY/IN_CLOSE_WRITE to any of the files you have to trigger the same action on all linked files as well. Obviously you also have to remove that relationship on the DELETE event.

A simpler and more robust approach would probably be to just periodically hash all the files and check if the content of a file has changed.

---

Correction

After checking the git source code closely and running git with strace, I found that git does use memory mapped files, but mostly for reading content. See the usage of xmmap which is always called with PROT_READ only.. Therefore my previous answer below is NOT the correct answer. Nevertheless for informational purpose I would still like to keep it here:

• You don't see IN_MODIFY events because packfile.c uses mmap for file access and inotify does not report modifications for mmaped files.

  From the inotify manpage:

  The inotify API does not report file accesses and modifications that may occur because of mmap(2), msync(2), and munmap(2).

Based on this accepted answer I'd assume there might be some difference in the events based on the protocol being used (i.e. ssh or https).

Do you observe the same behavior when monitoring cloning from the local filesystem with the --no-hardlinks option?

$ git clone [email protected]:user/repo.git
# set up watcher for new dir
$ git clone --no-hardlinks repo new-repo

Your observed behavior on running the experiment on both a linux and Mac host probably eliminates this open issue being the cause https://github.com/docker/for-mac/issues/896 but adding just incase.