Fix deadlock in Remove (linux/inotify)

Several people have reported this issue where if you are using a
single goroutine to watch for fs events and you call Remove in
that goroutine it can deadlock. The cause for this is that the Remove
was made synchronous by PR #73. The reason for this was to try and
ensure that maps were no longer leaking.

In this PR: IN_IGNORE was used as the event to ensure map cleanup.
This worked fine when Remove() was called and the next event was
IN_IGNORE, but when a different event was received the main goroutine
that's supposed to be reading from the Events channel would be stuck
waiting for the sync.Cond, which would never be hit because the select
would then block waiting for someone to receive the non-IN_IGNORE event
from the channel so it could proceed to process the IN_IGNORE event that
was waiting in the queue. Deadlock :)

Removing the synchronization then created two nasty races where Remove
followed by Remove would error unnecessarily, and one where Remove
followed by an Add could result in the maps being cleaned up AFTER the
Add call which means the inotify watch is active, but our maps don't
have the values anymore. It then becomes impossible to delete the
watches via the fsnotify code since it checks it's local data before
calling InotifyRemove.

This code attempts to use IN_DELETE_SELF as a means to know when a watch
was deleted as part of an unlink(). That means that we didn't delete the
watch via the fsnotify lib and we should clean up our maps since that
watch no longer exists. This allows us to clean up the maps immediately
when calling Remove since we no longer try to synchronize cleanup
using IN_IGNORE as the sync point.

- Fix #195
- Fix #123
- Fix #115
2 files changed
tree: 95b6c61860de89ebfaa370b07c1067405bbeefc1
  1. .github/
  2. .editorconfig
  3. .gitignore
  4. .travis.yml
  5. AUTHORS
  6. CHANGELOG.md
  7. CONTRIBUTING.md
  8. example_test.go
  9. fen.go
  10. fsnotify.go
  11. fsnotify_test.go
  12. inotify.go
  13. inotify_poller.go
  14. inotify_poller_test.go
  15. inotify_test.go
  16. integration_darwin_test.go
  17. integration_test.go
  18. kqueue.go
  19. LICENSE
  20. open_mode_bsd.go
  21. open_mode_darwin.go
  22. README.md
  23. windows.go
README.md

File system notifications for Go

GoDoc Go Report Card

fsnotify utilizes golang.org/x/sys rather than syscall from the standard library. Ensure you have the latest version installed by running:

go get -u golang.org/x/sys/...

Cross platform: Windows, Linux, BSD and macOS.

AdapterOSStatus
inotifyLinux 2.6.27 or later, Android*Supported Build Status
kqueueBSD, macOS, iOS*Supported Build Status
ReadDirectoryChangesWWindowsSupported Build status
FSEventsmacOSPlanned
FENSolaris 11In Progress
fanotifyLinux 2.6.37+
USN JournalsWindowsMaybe
PollingAllMaybe

* Android and iOS are untested.

Please see the documentation and consult the FAQ for usage information.

API stability

fsnotify is a fork of howeyc/fsnotify with a new API as of v1.0. The API is based on this design document.

All releases are tagged based on Semantic Versioning. Further API changes are planned, and will be tagged with a new major revision number.

Go 1.6 supports dependencies located in the vendor/ folder. Unless you are creating a library, it is recommended that you copy fsnotify into vendor/github.com/fsnotify/fsnotify within your project, and likewise for golang.org/x/sys.

Contributing

Please refer to CONTRIBUTING before opening an issue or pull request.

Example

See example_test.go.

FAQ

When a file is moved to another directory is it still being watched?

No (it shouldn't be, unless you are watching where it was moved to).

When I watch a directory, are all subdirectories watched as well?

No, you must add watches for any directory you want to watch (a recursive watcher is on the roadmap #18).

Do I have to watch the Error and Event channels in a separate goroutine?

As of now, yes. Looking into making this single-thread friendly (see howeyc #7)

Why am I receiving multiple events for the same file on OS X?

Spotlight indexing on OS X can result in multiple events (see howeyc #62). A temporary workaround is to add your folder(s) to the Spotlight Privacy settings until we have a native FSEvents implementation (see #11).

How many files can be watched at once?

There are OS-specific limits as to how many watches can be created:

  • Linux: /proc/sys/fs/inotify/max_user_watches contains the limit, reaching this limit results in a “no space left on device” error.
  • BSD / OSX: sysctl variables “kern.maxfiles” and “kern.maxfilesperproc”, reaching these limits results in a “too many open files” error.

Related Projects