globwatch

README

globwatch

A file system glob watcher for golang.

globwatch provides a file system watcher that detects changes recursively for files that match a glob pattern.

Installation

globwatch is provided as a go module and requires go >= 1.18.

go get github.com/halimath/globwatch@main

Usage

Creating a Watcher

globwatch provides a Watcher that can be created using the New function. The function receives a fs.FS handle to watch, a pattern (see below) and a check interval.

Once a Watcher has been created, you can start the watch routine by invoking Start or StartContext. The second function expects a context.Context that - when done - causes the watcher to terminate. Otherwise you can invoke Close to finish watching. Once finished a Watcher cannot be restarted.

watcher, err := globwatch.New(fsys, "**/*_test.go", time.Millisecond)
if err != nil {
    // ...
}

if err := watcher.Start(); err != nil {
    // ...
}

// ..

watcher.Close()

Receiving changes

A Watcher communicates changes via a channel. The channel is available via the C method.

for e := range watcher.C() {
    fmt.Printf("%8s %s\n", e.Type, e.Path)
}

In addition you can subscribe for errors by reading from an errors channel available via the ErrorsChan method.

Pattern format

The pattern format used by globwatch works similar to the pattern format of .gitignore. It is completely compatible with the pattern format used by os.Glob or fs.Glob and extends it.

The format is specified as the following EBNF:

pattern = term, { '/', term };

term        = '**' | name;
name        = { charSpecial | group | escapedChar | '*' | '?' };
charSpecial = (* any unicode rune except '/', '*', '?', '[' and '\' *);
char        = (* any unicode rune *);
escapedChar = '\\', char;
group       = '[', [ '^' ] { escapedChar | groupChar | range } ']';
groupChar   = (* any unicode rune except '-' and ']' *);
range       = ( groupChar | escapedChar ), '-', (groupChar | escapedChar);

The format operators have the following meaning:

• any character (rune) matches the exactly this rune - with the following exceptions
• / works as a directory separator. It matches directory boundarys of the underlying system independently of the separator char used by the OS.
• ? matches exactly one non-separator char
• * matches any number of non-separator chars - including zero
• \ escapes a character's special meaning allowing * and ? to be used as regular characters.
• ** matches any number of nested directories. If anything is matched it always extends until a separator or the end of the name.
• Groups can be defined using the [ and ] characters. Inside a group the special meaning of the characters mentioned before is disabled but the following rules apply
  • any character used as part of the group acts as a choice to pick from
  • if the group's first character is a ^ the whole group is negated
  • a range can be defined using - matching any rune between low and high inclusive
  • Multiple ranges can be given. Ranges can be combined with choices.
  • The meaning of - and ] can be escacped using \

Performance

globwatch separates pattern parsing and matching. This can create a performance benefit when applied repeatedly. When reusing a precompiled pattern to match filenames globwatch outperforms filepath.Match with both simple and complex patterns. When not reusing the parsed pattern, filepath works much faster (but lacks the additional features).

Test	Execution time [ns/op]	Memory usage [B/op]	Allocations per op
filepath simple pattern	15.5	0	0
globwatch simple pattern (reuse)	3.9	0	0
globwatch simple pattern (noreuse)	495.0	1112	5
filepath complex pattern	226.2	0	0
globwatch complex pattern (reuse)	108.1	0	0
globwatch complex pattern (noreuse)	1103.0	2280	8
globwatch directory wildcard pattern (reuse)	111.7	0	0
globwatch directory wildcard pattern (noreuse)	1229.0	2280	8

License

Copyright 2022 Alexander Metzner.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Documentation

Overview

Package globwatch implements a filesystem watcher that supports recursive watching using an extended glob pattern syntax.

The glob pattern syntax is descibed in github.com/halimath/globwatch/pattern.

The watcher is implemented based on a directory polling which periodically uses fs.WalkDir to walk a directory and check each file for changes. The watcher does not rely on kernel support like inotify or kqueue. The decision to work around these kernel features was made to support a large number of files and directories to watch. Especially with kqueue on MacOS you can quickly hit the open files limit.

Index

• type Event
• type EventType
  • func (t EventType) String() string
• type Watcher
  • func New(fsys fs.FS, pat string, interval time.Duration) (*Watcher, error)
  • func (w *Watcher) C() <-chan Event
  • func (w *Watcher) Close()
  • func (w *Watcher) ErrorsChan() <-chan error
  • func (w *Watcher) Start() error
  • func (w *Watcher) StartContext(ctx context.Context) error

Types

type Event

type Event struct {
	// The event's type
	Type EventType
	// The full path of the file relative to the watched root
	Path string
}

Event models a single event reported by a Watcher for a single file.

type EventType

type EventType int

EventType defines the type of event for a changed file.

const (
	// Created reports that a new file has been created.
	Created EventType = iota + 1
	// Modified reports that an existing file has changed.
	Modified
	// Deleted reports that an existing file has been deleted (or moved away).
	Deleted
)

func (EventType) String

func (t EventType) String() string

String returns a string representation of t.

type Watcher

type Watcher struct {
	// contains filtered or unexported fields
}

Watcher implements glob watching. Events for changed files will be reported via C. Any error that occured during change detection will be reported vi Errors. Make sure you consume both channels or you will block change detection otherwise.

func New

func New(fsys fs.FS, pat string, interval time.Duration) (*Watcher, error)

New creates a new watcher. The watcher will use fsys to access the files and directories. It will use fsys as the root to watch. pat defines the pattern relative to fsys' root. interval defines how often to check for changes. A created watcher will not start watching for changes unless Start or StartContext is called.

func (*Watcher) C

func (w *Watcher) C() <-chan Event

C returns a channel used to receive change Events.

func (*Watcher) Close

func (w *Watcher) Close()

Close closes w. The change detection goroutine will be shutdown gracefully and both w.C and w.Errors will be closed before Close returns.

func (*Watcher) ErrorsChan

func (w *Watcher) ErrorsChan() <-chan error

ErrorsChan returns a channel used to receive errors during watching.

func (*Watcher) Start

func (w *Watcher) Start() error

Start starts watching using a default context. See StartContext.

func (*Watcher) StartContext

func (w *Watcher) StartContext(ctx context.Context) error

StartContext starts watching for changes. If ctx will be canceled w will be closed. The funtion reports any error that occured during initial file analysis.