a quick review of file watchers
File watchers. I always forget about those and never use then, but I constantly feel like I need them. So I made this list to stop searching everywhere for those things which are surprisingly hard to find in a search engine.
Generic
Those tools an watch files or trees of files and execute whatever.
caretaker
https://github.com/grego/caretaker
- 2020
- Rust
- MIT
- No Debian package
- requires a TOML config file
chokidar
https://github.com/kimmobrunfeldt/chokidar-cli
- 2015-2019
- Javascript
- MIT
- No Debian package
direvent
https://www.gnu.org.ua/software/direvent/
- 2013-2019, GNU project since 2014
- C
- GPL-3
- Debian package, since 2015 (stretch), out of date (5.1 vs 5.2)
- requires a config file to operate
entr
http://eradman.com/entrproject/
- 2012-2019
- C
- ISC
- Debian package since 2014 (jessie), up to date
- finds file from stdin
- requires one watch per file, which quickly runs over the builtin limits, so requires sysctl hacks to actually work on even a medium-sized project
- has special hacks to reload browser
- can clear screen between calls
fluffy
https://github.com/tinkershack/fluffy
- 2018 (no official release)
- C
- Unlicense
- No Debian package
- Linux-specific
- Streams events to standard output
- also a library
fswatch
http://emcrisostomo.github.io/fswatch/
- 2014-2018
- C++
- GPL-3
- Debian package since 2017 (buster), up to date
- outputs changesets using a specific syntax, so requires more commandline voodoo
gamin
https://people.gnome.org/~veillard/gamin/
- 2004-2007
- Debian package since 2005
- not a commandline tool
incron
- 2006-2012, 2014-2015
- C++
- GPL-2
- Debian package since 2007
- Watches files and launches command in a cron-like (and rather obscure) syntax
- no event deduplication
inoticoming
https://tracker.debian.org/pkg/inoticoming
- 2007-2012
- C
- GPL-2
- Debian package is upstream
- watches directory, fires command
- yet another wrapper for inotify
- built for reprepro
- no event deduplication
inotify-hookable
https://metacpan.org/pod/App::Inotify::Hookable
- 2012-2016
- Debian package since 2013 (jessie)
- Perl
- GPL-1+
- yet another wrapper for inotify
- no event deduplication, but can "buffer" multiple events together with a timeout
inotify-tools
https://github.com/rvoicilas/inotify-tools/
- 2006-2018?
- C
- GPL-2
- Debian package since 2006
- canonical wrapper around inotify (Linux-specific)
- somewhat difficult commandline interface
- no event deduplication
notify-rs
https://github.com/notify-rs/notify
- 2016-2022
- Rust
- CC0 / Artistic
- Debian package since 2022
- cross-platform library, not a commandline tool
- used by
cargo watch
, watchexec (RFP), and Python's watchfiles which features a CLI tool
peat
- 2012-2016
- Python
- GPL-3
- Not in Debian
- relatively short Python script
- runs a command when files changed, specified on stdin
systemd .path units
https://www.freedesktop.org/software/systemd/man/systemd.path.html
- 2010-2019
- C
- GPL-2
- Debian package since 2010
- activates a system or user "unit" on inotify changes
Update: I tried to make this work for calibre but somehow it didn't work:
# this doesn't actually work. either it doesn't notices changes from git, or it
# doesn't notify calibre-server.service, or it does and that doesn't trigger a
# restart, but the thing doesn't restart as i would expect
[Path]
PathModified=/srv/books/metadata.db
PathModified=/srv/books
PathChanged=/srv/books/metadata.db
PathChanged=/srv/books
[Unit]
Description=calibre content server
After=network.target
... ie. it doesn't restart the service on changes to any of those files.
watchexec
https://github.com/watchexec/watchexec
- 2016-2019
- Rust
- Apache-2.0
- No Debian package (bug 946546)
- supports
.gitignore
files and filename patterns - merges multiple events
- fast polling
- can clear screen between calls
- waits for command to complete to launch again
simple commandline, ie. this worked for me:
watchexec -w ./feed2exec/ -e py -d 2000 -v -p tox
it automatically picked up my
.gitignore
directory, which elegantly avoided the loops I have had in watchman because of the files generated by tox
watchman
http://facebook.github.io/watchman/
- 2013-2019
- C++, Python
- Apache-2
- Debian package, since 2019 (bullseye)
- can watch multiple directories
- waits for settling
somewhat complex client/server architecture, sends output to logfiles, although watchman-make provides a nice little wrapper:
watchman-make -p '**/*.py' --run tox
Unfortunately this doesn't work for my Python project, as it picks up
setup.py
writing the_version.py
file as a change, and constantely runs the tests, because there's no way to ignore files inwatchman-make
.
Web development
grip (markdown)
https://github.com/joeyespo/grip
- 2012-2018
- Python
- MIT
- Debian package since 2016 (stretch)
- rebuilds markdown files and servers them over HTTP with autoreload
- relies on GitHub's API for rendering (???)
livereload
https://github.com/lepture/python-livereload
- 2012-2019
- Python
- BSD-3
- Debian package since 2013 (jessie), up to date
- watches a directory and triggers a web browser livereload extension
- also supports arbitrary commands through custom Python scripts
- integration with Python frameworks (Django, Flask, Bottle)
frameworks
Many frameworks automatically reload files when they change. A few examples:
- Flask - built-in for code, set
TEMPLATES_AUTO_RELOAD
for templates - Django -
runserver
will reload all files at every requests, unlesspyinotify
is installed, in which case it will do so more smartly - iPython - has a autoreload extension
Unit tests
autotest
https://metacpan.org/pod/distribution/App-autotest/scripts/autotest
- 2012-2015
- Perl
- Artistic license
- Rerun tests on file change
- Perl only
- No Debian package
conttest
https://github.com/eigenhombre/continuous-testing-helper
- 2012-2018 (no official release)
- Python
- No license
- No Debian package
- Reruns tests (but also any command) on file change in the current directory
- Part of a continuous testing strategy on the desktop
tdaemon
https://github.com/brunobord/tdaemon
- 2008-2014 (no official release)
- Python
- MIT
- No Debian package
- inspiration for the above
- restricted to a subset of Python test frameworks
Test::Continuous
https://metacpan.org/pod/Test::Continuous
- 2008-2015
- Perl
- Artistic license
- Run tests on file change
- Perl only
- No Debian package
File synchronization
I will not go through a list of all the file synchronization tools here. Most of them have some sort of "wake-up" system to notify file changes, but they are considered out of scope here unless they can also launch custom commands. But just for the sake of the argument, I am aware of:
gitwatch
https://github.com/gitwatch/gitwatch
- 2012-2020
- GPLv3
- relies on inotifywait
- just commits and pushes to git after a delay
inosync
https://github.com/hollow/inosync
- 2008-2010 (archived upstream)
- Python
- BSD
- Debian package since 2009 (jessie), in RFA
- combination of rsync and inotify
- similar to lsyncd
lsyncd
https://github.com/axkibe/lsyncd
- 2012-2018
- Lua, C
- GPL-2
- Debian package since 2010 (jessie), up to date
- spawns rsync on file changes
- Lua configuration can be leveraged to do other things than sync
Intrusion detection
Here again, there are many filesystem integrity checkers and intrusion detection systems (IDS), but they are not relevant here unless they can also execute arbitrary commands. But, again, here's an idea of the other stuff that is out there that might be a better fit than trying to fit a square peg in this round hole:
fail2ban
- 2004-2018
- Python
- GPL-2
- Debian package since 2005, out of date
- parses logfiles and executes commands
- handles multiple services and patterns and can escalate
iwatch
http://iwatch.sourceforge.net/
- 2006-2009
- Perl
- GPL-2+
- Debian package since 2006
- emails on file changes
- can also execute commands, but primarily designed as an intrusion detection system
- XML configuration (???)
sshguard
- 2007-2019
- C
- ISC
- Debian package since 2007, out of date
- similar to fail2ban
Other
inotify-info
https://github.com/mikesart/inotify-info
- 2021-...
- C++
- MIT
- Debian package
- simply lists active inotify watches
kfmon (kobo launcher)
https://github.com/NiLuJe/kfmon
- 2016-2019
- C
- GPL-3
- No Debian package
- Launches programs when icons are activated on a Kobo reader
I've probably missed half a million such programs so this might be expanded with time and your comments.
Update: I've added a bunch of tools. I was relunctant to add all those old inotify wrappers because I don't find them as interesting as the newer ones - they're really hard to use! - but I guess it's worth mentioning them even if just to criticise them.
timetrack
https://github.com/joshmcguigan/timetrack
- 2018-2019
- Rust
- Apache-2.0, MIT
- No Debian package
- tracks filesystem changes to report time spent on different things, see also this discussion on selfspy for other alternatives
You also have inotifywait/inotifywatch, inotify-hookable, lsyncd (can do more than just sync)
Perl also has Continuous Testing features based in file monitoring. There are at least
Neither one seems to be in Debian stable, but they are only one
cpanm
call away.https://github.com/tinkershack/fluffy Not really meant for firing arbitrary commands, however it's output is super-'clean' & easy to parse. Also comes with a libfluffy library as well.