README
¶
Simple no-nonsense file tailing.
Installation
go get -u github.com/jacobcase/gotail
Overview
gotail is a simple go module that provides regular file tailing. While there are a few other tail libraries, many of them are some combination of unmaintained, buggy, or overly complex.
gotail currently exposes a simple file polling implementation. It does not assume that you only want to read newline delimited text data, nor does it assume you want to consume it over a channel. Instead, implementations in this module expose file tailing as a simple io.ReadCloser that never reaches EOF, transparently consuming files as they are rotated.
gotail will NOT work correctly on files that are truncated.
Polling may be excessive for some applications. This module was designed with large and frequently written log files in mind, such as edge proxy logs.
Platforms
So far, gotail has only been testing on Linux. However, the poller implementation doesn't depend on any OS specific features that aren't abstracted by the OS package, so it should work on most systems.
Contributing
Contributions welcome! An fsnotify implementation would be nice and I may get around to adding it some day.
TODO
- File checkpoints for resuming on restart
- Readline implementation that can provide line aligned checkpoints.
- More thorough testing
- Test seeking
- Test correct offset tracking
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func DiscardErrorHandler ¶ added in v0.3.0
DiscardErrorHandler ignores all errors and always returns nil.
Types ¶
type Config ¶ added in v0.3.0
type Config struct {
// Path should be the location to a regular file. This value is
// not validated and is passed directly to os.Open().
Path string
// Interval is used for a few operations. For file polling, it is
// how frequently to check for new data. For the LineReader, it is
// also how long to wait before retrying.
Interval time.Duration
// Whence can be set to one of the Seek constants from the IO package.
// It only applies to the first file opened, as subsequent files will always be
// read from the beginning. io.SeekCurrent will behave the same as io.SeekStart.
// This will also be disregarded if the file doesn't initially exist on disk.
Whence int
// StartState is optional for resuming the first file opened from where it left
// off if the FileState matches.
StartState *FileState
// StopAtEOF will cause a tail to exit when it gets the first EOF.
// Useful for consumers to build tests.
StopAtEOF bool
}
Config is shared among a few types in this package to configure what and how to tail a file.
type FileState ¶ added in v0.2.1
type FileState struct {
Size int64 `json:",string"`
Position int64 `json:",string"`
Inode uint64 `json:",string"`
}
FileState describes some details about a regular file that can be used to compare it with another file on disk for a best guess on if they are the same file. It can also store the position of a file descriptor to allow seeking if reopening later to continue where it last left off.
func NewFileState ¶ added in v0.2.1
NewFileState will initialize a FileState with the inode, size, and position of the provided file. Currently does not support windows, or anything that isn't a *syscall.Stat_t or *unix.Stat_t in the underlying stat.
func NewFileStateFromPath ¶ added in v0.2.1
func (*FileState) SeekIfMatches ¶ added in v0.2.1
SeekIfMatches will try to determine if this FileState matches that of the file, which means they must have a matching Inode and the size of f must be at least as big as this FileState's Position. Otherwise it does nothing. The returned SeekInfo is always valid for f if the error is nil, though the Position is not updated so if the descriptor of f points beyond the start of the file, Position will need to be updated outside this method.
type LineReader ¶ added in v0.2.1
type LineReader struct {
// contains filtered or unexported fields
}
LineReader provides a way to transparently read \n or \r\n delimited lines across multiple files. The only method that is safe to call in parallel to other methods is Close().
func NewLineReader ¶ added in v0.2.1
func NewLineReader(c Config, h ErrorHandler) (*LineReader, error)
NewLineReader returns a LineReader that has an underlying Watcher created from c and will run unexpected errors through ErrorHandler h. If the error is an EOF or file not found error, it will not be passed to the error handler. If h is nil, errors will be ignored and will automatically retry.
func (*LineReader) Bytes ¶ added in v0.2.1
func (l *LineReader) Bytes() []byte
func (*LineReader) Close ¶ added in v0.2.1
func (l *LineReader) Close() error
Close cleans up any resources and should only be called once.
func (*LineReader) Err ¶ added in v0.3.0
func (l *LineReader) Err() error
Err returns any error that occurred that caused Next to return false. If it's set, it will generally be what was returned by the ErrorHandler.
func (*LineReader) FileState ¶ added in v0.2.1
func (l *LineReader) FileState() FileState
func (*LineReader) Next ¶ added in v0.2.1
func (l *LineReader) Next() bool
type WaitStatus ¶ added in v0.2.1
type WaitStatus struct {
// State contains the latest open file size, position of the
// descriptor, and file inode. This can be used to preserve
// the position between application starts for the same file.
State FileState
// File is a ready to use file the Watcher has determined
// is next to be read from. This file does NOT need to be
// closed by the consumer, as it should always be closed
// when a Watcher no longer considers it the latest to read
// from or the Watcher is closed.
File *os.File
// ReOpened, if true, indicates the file returned has just been
// opened. This will also be true for the first file opened, even
// though there wasn't one previously.
ReOpened bool
}
WaitStatus is the result of Watcher.Wait and should contain enough information for callers to setup for the next file Read.
type Watcher ¶ added in v0.3.0
type Watcher interface {
// Wait will block until there is more data to read, the watcher
// is closed, or there was an error checking if there was more data
// to read. Wait should always be safe to call again if there was
// an error previously, but calling again when closed returns true
// should be avoided.
Wait() (s WaitStatus, closed bool, err error)
// Close will stop the Watcher, cleanup any resources, and
// return the result of closing the currently open file if one
// is open.
Close() error
}
Watcher provides a simple interface to handle reading rotated files.
func NewPollingWatcher ¶ added in v0.3.0
NewPollingWatcher configures a Watcher that uses file polling to determine when there is more data to read. It doesn't support files that were truncated, and only supports regular files (no pipes).
Source Files