other sections
WATCHDOG(3) | watchdog | WATCHDOG(3) |
NAME¶
watchdog - watchdog Documentation Python API library and shell utilities to monitor file system events.DIRECTORY MONITORING MADE EASY WITH¶
- •
- A cross-platform API.
- •
- A shell tool to run commands in response to directory changes.
Get started quickly with a simple example in quickstart.
EASY INSTALLATION¶
You can use pip to install watchdog quickly and easily:$ pip install watchdog
Need more help with installing? See installation.
INSTALLATION¶
watchdog requires Python 2.5 or above to work. If you are using a Linux/FreeBSD/Mac OS X system, you already have Python installed. However, you may wish to upgrade your system to Python 2.7 at least, because this version comes with updates that can reduce compatibility problems. See a list of installation-dependencies.Installing from PyPI using pip¶
$ pip install watchdog
Installing from source tarballs¶
$ wget -c http://pypi.python.org/packages/source/w/watchdog/watchdog-0.7.0.tar.gz $ tar zxvf watchdog-0.7.0.tar.gz $ cd watchdog-0.7.0 $ python setup.py install
Installing from the code repository¶
$ git clone --recursive git://github.com/gorakhargosh/watchdog.git $ cd watchdog $ python setup.py install
Dependencies¶
watchdog depends on many libraries to do its job. The following is a list of dependencies you need based on the operating system you are using.Operating system Dependency (row) | Windows | Linux 2.6 | 0.0 Mac OS X/ Darwin 168u | BSD |
XCode | Yes | |||
PyYAML | Yes | Yes | Yes | Yes |
argh | Yes | Yes | Yes | Yes |
argparse | Yes | Yes | Yes | Yes |
select_backport (Python 2.5/2.6) | Yes | Yes | ||
pathtools | Yes | Yes | Yes | Yes |
a lot of luck | Yes | |||
Installing Dependencies¶
The watchmedo script depends on PyYAML which links with LibYAML. On Mac OS X, you can use homebrew to install LibYAML:brew install libyaml
On Linux, use your favorite package manager to install LibYAML. Here's how you do it on Ubuntu:
sudo aptitude install libyaml-dev
On Windows, please install PyYAML using the binaries they provide.
Supported Platforms (and Caveats)¶
watchdog uses native APIs as much as possible falling back to polling the disk periodically to compare directory snapshots only when it cannot use an API natively-provided by the underlying operating system. The following operating systems are currently supported: WARNING:Differences between behaviors of these native API are
noted below.
- Linux 2.6+
- Linux kernel version 2.6 and later come with an API called inotify that programs can use to monitor file system events. watchdog can use this feature by relying on a third party library for python called PyInotify. (Future releases may remove this dependency.)
- Mac OS X
- The Darwin kernel/OS X API maintains two ways to monitor directories for file system events:
- •
- kqueue
- •
- FSEvents
watchdog can use whichever one is available, preferring FSEvents over kqueue(2). kqueue(2) uses open file descriptors for monitoring and the current implementation uses Mac OS X File System Monitoring Performance Guidelines to open these file descriptors only to monitor events, thus allowing OS X to unmount volumes that are being watched without locking them. NOTE:
More information about how watchdog uses
kqueue(2) is noted in BSD Unix variants. Much of this
information applies to Mac OS X as well.
- BSD variants come with kqueue which programs can use to monitor
- changes to open file descriptors. Because of the way kqueue(2) works, watchdog needs to open these files and directories in read-only non-blocking mode and keep books about them. watchdog will automatically open file descriptors for all new files/directories created and close those for which are deleted. NOTE:
The maximum number of open file descriptor per process
limit on your operating system can hinder watchdog's ability to monitor
files.
You should ensure this limit is set to at least 1024 (or a value suitable
to your usage). The following command appended to your ~/.profile
configuration file does this for you:
ulimit -n 1024
- Windows Vista and later
- The Windows API on Windows XP provides the ReadDirectoryChangesW function, which operates in either synchronous or asynchronous mode. watchdog currently contains implementation for the synchronous approach and use additional API functionality only available in Windows Vista and later. NOTE:
Since renaming is not the same operation as movement on
Windows, watchdog tries hard to convert renames to movement events.
Also, because the ReadDirectoryChangesW API function returns
rename/movement events for directories even before the underlying I/O is
complete, watchdog may not be able to completely scan the moved
directory in order to successfully queue movement events for files and
directories within it.
- OS Independent Polling
- watchdog also includes a fallback-implementation that polls watched directories for changes by periodically comparing snapshots of the directory tree. NOTE:
Windows caveats again.
Because Windows has no concept of inodes as Unix-y platforms do, there is
no current reliable way of determining file/directory movement on Windows
without help from the Windows API.
You can use hashing for only those files in which you are interested in your
event handlers to determine this, although it is rather slow. watchdog
does not attempt to handle this on Windows. It is left to your
discretion.
QUICKSTART¶
Below we present a simple example that monitors the current directory recursively (which means, it will traverse any sub-directories) to detect changes. Here is what we will do with the API:- 1.
- Create an instance of the watchdog.observers.Observer thread class.
- 2.
- Implement a subclass of watchdog.events.FileSystemEventHandler (or as in our case, we will use the built-in watchdog.events.LoggingEventHandler, which already does).
- 3.
- Schedule monitoring a few paths with the observer instance attaching the event handler.
- 4.
- Start the observer thread and wait for it generate events without blocking our main thread.
By default, an watchdog.observers.Observer instance will not monitor sub-directories. By passing recursive=True in the call to watchdog.observers.Observer.schedule() monitoring entire directory trees is ensured.
A Simple Example¶
The following example program will monitor the current directory recursively for file system changes and simply log them to the console:import sys import time import logging from watchdog.observers import Observer from watchdog.events import LoggingEventHandler if __name__ == "__main__": logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S') path = sys.argv[1] if len(sys.argv) > 1 else '.' event_handler = LoggingEventHandler() observer = Observer() observer.schedule(event_handler, path, recursive=True) observer.start() try: while True: time.sleep(1) except KeyboardInterrupt: observer.stop() observer.join()
To stop the program, press Control-C.
API REFERENCE¶
watchdog.events¶
- module
- watchdog.events
- synopsis
- File system events and event handlers.
- author
- yesudeep@google.com (Yesudeep Mangalapilly)
Event Classes¶
- class watchdog.events.FileSystemEvent(event_type, src_path, is_directory=False)
- Bases: object Immutable type that represents a file system event that is triggered when a change occurs on the monitored file system. All FileSystemEvent objects are required to be immutable and hence can be used as keys in dictionaries or be added to sets.
- event_type
- The type of the event as a string.
- is_directory
- True if event was emitted for a directory; False otherwise.
- src_path
- Source path of the file system object that triggered this event.
- class watchdog.events.FileSystemMovedEvent(src_path, dest_path, is_directory)
- Bases: watchdog.events.FileSystemEvent File system event representing any kind of file system movement.
- dest_path
- The destination path of the move event.
- class watchdog.events.FileMovedEvent(src_path, dest_path)
- Bases: watchdog.events.FileSystemMovedEvent File system event representing file movement on the file system.
- class watchdog.events.DirMovedEvent(src_path, dest_path)
- Bases: watchdog.events.FileSystemMovedEvent File system event representing directory movement on the file system.
- sub_moved_events(_walker=<function walk at 0x2b498b623e60>)
- Generates moved events for file system objects within the moved directory.
- Parameters
- _walker -- Walker used to walk directory trees os.walk() style. Sanity tests use this parameter to inject a mock walker that behaves like os.walk().
- Returns
- iterable of event objects of type FileMovedEvent and DirMovedEvent.
- class watchdog.events.FileModifiedEvent(src_path)
- Bases: watchdog.events.FileSystemEvent File system event representing file modification on the file system.
- class watchdog.events.DirModifiedEvent(src_path)
- Bases: watchdog.events.FileSystemEvent File system event representing directory modification on the file system.
- class watchdog.events.FileCreatedEvent(src_path)
- Bases: watchdog.events.FileSystemEvent File system event representing file creation on the file system.
- class watchdog.events.DirCreatedEvent(src_path)
- Bases: watchdog.events.FileSystemEvent File system event representing directory creation on the file system.
- class watchdog.events.FileDeletedEvent(src_path)
- Bases: watchdog.events.FileSystemEvent File system event representing file deletion on the file system.
- class watchdog.events.DirDeletedEvent(src_path)
- Bases: watchdog.events.FileSystemEvent File system event representing directory deletion on the file system.
Event Handler Classes¶
- class watchdog.events.FileSystemEventHandler
- Bases: object Base file system event handler that you can override methods from.
- dispatch(event)
- Dispatches events to the appropriate methods.
- Parameters
- event (FileSystemEvent) -- The event object representing the file system event.
- on_any_event(event)
- Catch-all event handler.
- Parameters
- event (FileSystemEvent) -- The event object representing the file system event.
- on_created(event)
- Called when a file or directory is created.
- Parameters
- event (DirCreatedEvent or FileCreatedEvent) -- Event representing file/directory creation.
- on_deleted(event)
- Called when a file or directory is deleted.
- Parameters
- event (DirDeletedEvent or FileDeletedEvent) -- Event representing file/directory deletion.
- on_modified(event)
- Called when a file or directory is modified.
- Parameters
- event (DirModifiedEvent or FileModifiedEvent) -- Event representing file/directory modification.
- on_moved(event)
- Called when a file or a directory is moved or renamed.
- Parameters
- event (DirMovedEvent or FileMovedEvent) -- Event representing file/directory movement.
- class watchdog.events.PatternMatchingEventHandler(patterns=None, ignore_patterns=None, ignore_directories=False, case_sensitive=False)
- Bases: watchdog.events.FileSystemEventHandler Matches given patterns with file paths associated with occurring events.
- case_sensitive
- (Read-only) True if path names should be matched sensitive to case; False otherwise.
- dispatch(event)
- Dispatches events to the appropriate methods.
- Parameters
- event (FileSystemEvent) -- The event object representing the file system event.
- ignore_directories
- (Read-only) True if directories should be ignored; False otherwise.
- ignore_patterns
- (Read-only) Patterns to ignore matching event paths.
- patterns
- (Read-only) Patterns to allow matching event paths.
- class watchdog.events.RegexMatchingEventHandler(regexes=['.*'], ignore_regexes=[], ignore_directories=False, case_sensitive=False)
- Bases: watchdog.events.FileSystemEventHandler Matches given regexes with file paths associated with occurring events.
- case_sensitive
- (Read-only) True if path names should be matched sensitive to case; False otherwise.
- dispatch(event)
- Dispatches events to the appropriate methods.
- Parameters
- event (FileSystemEvent) -- The event object representing the file system event.
- ignore_directories
- (Read-only) True if directories should be ignored; False otherwise.
- ignore_regexes
- (Read-only) Regexes to ignore matching event paths.
- regexes
- (Read-only) Regexes to allow matching event paths.
- class watchdog.events.LoggingEventHandler
- Bases: watchdog.events.FileSystemEventHandler Logs all the events captured.
watchdog.observers.api¶
- module
- watchdog.observers.api
- synopsis
- Classes useful to observer implementers.
- author
- yesudeep@google.com (Yesudeep Mangalapilly)
Immutables¶
- class watchdog.observers.api.ObservedWatch(path, recursive)
- Bases: object An scheduled watch.
- Parameters
- •
- path -- Path string.
- •
- recursive -- True if watch is recursive; False otherwise.
- is_recursive
- Determines whether subdirectories are watched for the path.
- path
- The path that this watch monitors.
Collections¶
- class watchdog.observers.api.EventQueue(maxsize=0)
- Bases: watchdog.utils.bricks.SkipRepeatsQueue Thread-safe event queue based on a special queue that skips adding the same event ( FileSystemEvent) multiple times consecutively. Thus avoiding dispatching multiple event handling calls when multiple identical events are produced quicker than an observer can consume them.
Classes¶
- class watchdog.observers.api.EventEmitter(event_queue, watch, timeout=1)
- Bases: watchdog.utils.DaemonThread Producer daemon thread base class subclassed by event emitters that generate events and populate a queue with them.
- Parameters
- •
- event_queue (watchdog.events.EventQueue) -- The event queue to populate with generated events.
- •
- watch (ObservedWatch) -- The watch to observe and produce events for.
- •
- timeout (float) -- Timeout (in seconds) between successive attempts at reading events.
- queue_event(event)
- Queues a single event.
- Parameters
- event (An instance of watchdog.events.FileSystemEvent or a subclass.) -- Event to be queued.
- queue_events(timeout)
- Override this method to populate the event queue with events per interval period.
- Parameters
- timeout (float) -- Timeout (in seconds) between successive attempts at reading events.
- timeout
- Blocking timeout for reading events.
- watch
- The watch associated with this emitter.
- class watchdog.observers.api.EventDispatcher(timeout=1)
- Bases: watchdog.utils.DaemonThread Consumer daemon thread base class subclassed by event observer threads that dispatch events from an event queue to appropriate event handlers.
- Parameters
- timeout (float) -- Event queue blocking timeout (in seconds).
- dispatch_events(event_queue, timeout)
- Override this method to consume events from an event queue, blocking on the queue for the specified timeout before raising queue.Empty.
- Parameters
- •
- event_queue (EventQueue) -- Event queue to populate with one set of events.
- •
- timeout (float) -- Interval period (in seconds) to wait before timing out on the event queue.
- Raises
- queue.Empty
- event_queue
- The event queue which is populated with file system events by emitters and from which events are dispatched by a dispatcher thread.
- timeout
- Event queue block timeout.
- class watchdog.observers.api.BaseObserver(emitter_class, timeout=1)
- Bases: watchdog.observers.api.EventDispatcher Base observer.
- add_handler_for_watch(event_handler, watch)
- Adds a handler for the given watch.
- Parameters
- •
- event_handler (watchdog.events.FileSystemEventHandler or a subclass) -- An event handler instance that has appropriate event handling methods which will be called by the observer in response to file system events.
- •
- watch (An instance of ObservedWatch or a subclass of ObservedWatch) -- The watch to add a handler for.
- remove_handler_for_watch(event_handler, watch)
- Removes a handler for the given watch.
- Parameters
- •
- event_handler (watchdog.events.FileSystemEventHandler or a subclass) -- An event handler instance that has appropriate event handling methods which will be called by the observer in response to file system events.
- •
- watch (An instance of ObservedWatch or a subclass of ObservedWatch) -- The watch to remove a handler for.
- schedule(event_handler, path, recursive=False)
- Schedules watching a path and calls appropriate methods specified in the given event handler in response to file system events.
- Parameters
- •
- event_handler (watchdog.events.FileSystemEventHandler or a subclass) -- An event handler instance that has appropriate event handling methods which will be called by the observer in response to file system events.
- •
- path (str) -- Directory path that will be monitored.
- •
- recursive (bool) -- True if events will be emitted for sub-directories traversed recursively; False otherwise.
- Returns
- An ObservedWatch object instance representing a watch.
- unschedule(watch)
- Unschedules a watch.
- Parameters
- watch (An instance of ObservedWatch or a subclass of ObservedWatch) -- The watch to unschedule.
- unschedule_all()
- Unschedules all watches and detaches all associated event handlers.
watchdog.observers¶
- module
- watchdog.observers
- synopsis
- Observer that picks a native implementation if available.
- author
- yesudeep@google.com (Yesudeep Mangalapilly)
Classes¶
- watchdog.observers.Observer
- alias of InotifyObserver
You can also import platform specific classes directly and use it instead of Observer. Here is a list of implemented observer classes.:
Class | Platforms | Note |
inotify.InotifyObserver | Linux 2.6.13+ | inotify(7) based observer |
fsevents.FSEventsObserver | Mac OS X | FSEvents based observer |
kqueue.KqueueObserver | Mac OS X and BSD with kqueue(2) | kqueue(2) based observer |
read_directory_changes.WindowsApiObserver | MS Windows | Windows API-based observer |
polling.PollingObserver | Any | fallback implementation |
watchdog.utils¶
- module
- watchdog.utils
- synopsis
- Utility classes and functions.
- author
- yesudeep@google.com (Yesudeep Mangalapilly)
Functions¶
- watchdog.utils.has_attribute(ob, attribute)
- hasattr() swallows exceptions. has_attribute() tests a Python object for the presence of an attribute.
- Parameters
- •
- ob -- object to inspect
- •
- attribute -- str for the name of the attribute.
- watchdog.utils.load_class(dotted_path)
- Loads and returns a class definition provided a dotted path specification the last part of the dotted path is the class name and there is at least one module name preceding the class name. Notes: You will need to ensure that the module you are trying to load exists in the Python path. Examples: - module.name.ClassName # Provided module.name is in the Python path. - module.ClassName # Provided module is in the Python path. What won't work: - ClassName - modle.name.ClassName # Typo in module name. - module.name.ClasNam # Typo in classname.
- watchdog.utils.load_module(module_name)
- Imports a module given its name and returns a handle to it.
- watchdog.utils.read_text_file(file_path, mode='rb')
- Returns the contents of a file after opening it in read-only mode.
- Parameters
- •
- file_path -- Path to the file to be read from.
- •
- mode -- Mode string.
Classes¶
- class watchdog.utils.DaemonThread
- Bases: threading.Thread Daemon thread convenience class, sets a few properties and makes writing daemon threads a little easier.
- daemon
- A boolean value indicating whether this thread is a daemon thread (True) or not (False). This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False. The entire Python program exits when no alive non-daemon threads are left.
- ident
- Thread identifier of this thread or None if it has not been started. This is a nonzero integer. See the thread.get_ident() function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited.
- isAlive()
- Return whether the thread is alive. This method returns True just before the run() method starts until just after the run() method terminates. The module function enumerate() returns a list of all alive threads.
- is_alive()
- Return whether the thread is alive. This method returns True just before the run() method starts until just after the run() method terminates. The module function enumerate() returns a list of all alive threads.
- join(timeout=None)
- Wait until the thread terminates. This blocks the calling thread until the thread whose join() method is called terminates -- either normally or through an unhandled exception or until the optional timeout occurs. When the timeout argument is present and not None, it should be a floating point number specifying a timeout for the operation in seconds (or fractions thereof). As join() always returns None, you must call isAlive() after join() to decide whether a timeout happened -- if the thread is still alive, the join() call timed out. When the timeout argument is not present or None, the operation will block until the thread terminates. A thread can be join()ed many times. join() raises a RuntimeError if an attempt is made to join the current thread as that would cause a deadlock. It is also an error to join() a thread before it has been started and attempts to do so raises the same exception.
- name
- A string used for identification purposes only. It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor.
- on_thread_stop()
- Override this method instead of stop(). stop() calls this method. Note that this method is called immediately after the daemon thread is signaled to halt.
- run()
- Method representing the thread's activity. You may override this method in a subclass. The standard run() method invokes the callable object passed to the object's constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.
- should_keep_running()
- Determines whether the daemon thread should continue running.
- should_stop()
- Determines whether the daemon thread should stop.
- start()
- Start the thread's activity. It must be called at most once per thread object. It arranges for the object's run() method to be invoked in a separate thread of control. This method will raise a RuntimeError if called more than once on the same thread object.
- stop()
- Signals the daemon thread to stop.
watchdog.utils.dirsnapshot¶
- module
- watchdog.utils.dirsnapshot
- synopsis
- Directory snapshots and comparison.
- author
- yesudeep@google.com (Yesudeep Mangalapilly)
- Where are the moved events? They 'disappeared'
- This implementation does not take partition boundaries into consideration. It will only work when the directory tree is entirely on the same file system. More specifically, any part of the code that depends on inode numbers can break if partition boundaries are crossed. In these cases, the snapshot diff will represent file/directory movement as created and deleted events. Windows does not have any concept of inodes, which prevents this snapshotter from determining file or directory renames/movement on it. The snapshotter does not try to handle this on Windows. File or directory movement will show up as creation and deletion events. Please do not use this on a virtual file system mapped to a network share.
Classes¶
- class watchdog.utils.dirsnapshot.DirectorySnapshot(path, recursive=True, walker_callback=<function <lambda> at 0x2b498e65cd70>, _copying=False)
- Bases: object A snapshot of stat information of files in a directory.
- Parameters
- •
- path (str) -- The directory path for which a snapshot should be taken.
- •
- recursive (bool) -- True if the entired directory tree should be included in the snapshot; False otherwise.
- •
- walker_callback -- A function with the signature walker_callback(path, stat_info) which will be called for every entry in the directory tree.
- has_inode(inode)
- Determines if the inode exists.
- Parameters
- inode -- inode number.
- path_for_inode(inode)
- Determines the path that an inode represents in a snapshot.
- Parameters
- inode -- inode number.
- paths
- List of file/directory paths in the snapshot.
- stat_info(path)
- Returns a stat information object for the specified path from the snapshot.
- Parameters
- path -- The path for which stat information should be obtained from a snapshot.
- stat_info_for_inode(inode)
- Determines stat information for a given inode.
- Parameters
- inode -- inode number.
- stat_snapshot
- Returns a dictionary of stat information with file paths being keys.
- class watchdog.utils.dirsnapshot.DirectorySnapshotDiff(ref_dirsnap, dirsnap)
- Bases: object Compares two directory snapshots and creates an object that represents the difference between the two snapshots.
- Parameters
- •
- ref_dirsnap (DirectorySnapshot) -- The reference directory snapshot.
- •
- dirsnap (DirectorySnapshot) -- The directory snapshot which will be compared with the reference snapshot.
- dirs_created
- List of directories that were created.
- dirs_deleted
- List of directories that were deleted.
- dirs_modified
- List of directories that were modified.
- dirs_moved
- List of directories that were moved. Each event is a two-tuple the first item of which is the path that has been renamed to the second item in the tuple.
- files_created
- List of files that were created.
- files_deleted
- List of files that were deleted.
- files_modified
- List of files that were modified.
- files_moved
- List of files that were moved. Each event is a two-tuple the first item of which is the path that has been renamed to the second item in the tuple.
CONTRIBUTING¶
Welcome hacker! So you have got something you would like to see in watchdog? Whee. This document will help you get started.Important URLs¶
watchdog uses git to track code history and hosts its code repository at github. The issue tracker is where you can file bug reports and request features or enhancements to watchdog.Before you start¶
Ensure your system has the following programs and libraries installed before beginning to hack:- 1.
- Python
- 2.
- git
- 3.
- ssh
- 4.
- XCode (on Mac OS X)
- 5.
- select_backport (on BSD/Mac OS X if you're using Python 2.5/2.6)
Setting up the Work Environment¶
watchdog makes extensive use of zc.buildout to set up its work environment. You should get familiar with it. Steps to setting up a clean environment:- 1.
- Fork the code repository into your github account. Let us call you hackeratti for the sake of this example. Replace hackeratti with your own username below.
- 2.
- Clone your fork and setup your environment:
$ git clone --recursive git@github.com:hackeratti/watchdog.git $ cd watchdog $ python tools/bootstrap.py --distribute $ bin/buildout
IMPORTANT:
Re-run bin/buildout every time you make a change
to the buildout.cfg file.
That's it with the setup. Now you're ready to hack on watchdog.
Enabling Continuous Integration¶
The repository checkout contains a script called autobuild.sh which you must run prior to making changes. It will detect changes to Python source code or restructuredText documentation files anywhere in the directory tree and rebuild sphinx documentation, run all tests using nose, and generate coverage reports. Start it by issuing this command in the watchdog directory checked out earlier:$ tools/autobuild.sh ...
Happy hacking! Found a bug in or want a feature added to watchdog? You can fork the official code repository or file an issue ticket at the issue tracker. You can also ask questions at the official mailing list. You may also want to refer to hacking for information about contributing code or documentation to watchdog.
- •
- genindex
- •
- modindex
- •
- search
AUTHOR¶
Yesudeep MangalapillyCOPYRIGHT¶
2010, Yesudeep MangalapillyApril 19, 2014 | 0.7.0 |