inotify with NFS
inotify requires support from the kernel to work. When an application tracks a directory, it asks the kernel to inform it when those changes occur. When the change occurs, in addition to writing those changes to disk, the kernel also notifies the watching process.
On a remote NFS machine, the change is not visible to the kernel; it happens entirely remotely. NFS predates inotify and there is no network level support for it in NFS, or anything equivalent.
If you want to get around this, You can run a service on the storage server (since that kernel will always see changes to the filesystem) that brokers inotify requests for remote machines, and forward the data to the remote clients.
Edit: It seems odd to me that NFS should be blamed for its lack of support for inotify.
Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems in 1984, wikipedia article
However:
Inotify (inode notify) is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem. [...] It has been included in the mainline Linux kernel from release 2.6.13 (June 18, 2005 ) [...]. wikipedia article
It's hard to expect a portable network protocol/application to support a specific kernel feature developed for a different operating system and that appeared more than twenty years later. Even if it did include extensions for it, they would not be available or useful on other operating systems.
*emphasis mine in all cases
Another problem with this; Lets suppose we are not using a network at all, but rather, a local filesystem with good inotify support: ext3 (suppose its mounted at /mnt/foo). But instead of a real disk, the filesystem is mounted from a loopback device ; and the underlying file is in turn accessible at a different location in the vfs (say, /var/images/foo.img).
Now, you're not supposed to modify mounted ext3 filesystems, But it's still reasonably safe to do so if the change is to file contents instead of metadata.
So suppose a clever user modifies the file system image (/var/images/foo.img) in a hex editor, replacing a file's contents with some other data, while at the same time an inotify watch is observing the same file on the mounted filesystem.
There's no reasonable way one can arrange for inotify to always inform the watching process of this sort of change. Although there are probably some gyrations that could be take to make ext3 notice and honor the change, none of that would apply to, say, the xfs drtiver, which is otherwise quite similar.
Nor should it. You're cheating!. inotify can only inform you of changes that occured through the vfs at the actual mountpoint being watched. If the changes occured outside that VFS, because of a change to the underlying data, inotify can't help you and isn't designed to solve that problem.
Have you considered using a message queue for network notification?
To anyone who has come across this question in the search for an answer of why bind mounting on Docker will not detect file changes from host directory (for hot reloading of an app), it's because the propagation of file changes between host and container is not communicated to the container kernel.
Only changes from the container itself is communicated to the kernel. Solution for this is to have your live reload utility turn on "polling mode" instead of using fsnotify.