keepcurrent

README

keepcurrent

A library to periodically check for changes from the source and update a set of sinks. Optionally the sinks can be initialized from a different source for once.

One typical user case is to keep both the in memory configuration and local config file in sync with the remote one, and load the local config file when the program starts up to speed up a bit.

Documentation

Index

• Variables
• func ExpBackoff(base time.Duration, stop int) func(err error, tries int) time.Duration
• func ExpBackoffThenFail(base time.Duration, stop int, onFail func(err error)) func(err error, tries int) time.Duration
• type Runner
  • func New(from Source, to ...Sink) *Runner
  • func NewWithValidator(validate func(data []byte) error, from Source, to ...Sink) *Runner
  • func (runner *Runner) InitFrom(s Source)
  • func (runner *Runner) Start(interval time.Duration) func()
• type Sink
  • func ToChannel(ch chan []byte) Sink
  • func ToFile(path string) Sink
  • func ToFileWithPreprocessor(path string, preprocessor func(io.Reader) (io.Reader, error)) Sink
• type Source
  • func FromFile(path string) Source
  • func FromFileWithPreprocessor(path string, preprocessor func(io.ReadCloser) (io.ReadCloser, error)) Source
  • func FromTarGz(s Source, expectedName string) Source
  • func FromWeb(url string) Source
  • func FromWebWithClient(url string, client *http.Client) Source

Variables

var ErrUnmodified = errors.New("unmodified")

ErrUnmodified is the error to signal that the source has not been modified since the last sync.

Functions

func ExpBackoff

func ExpBackoff(base time.Duration, stop int) func(err error, tries int) time.Duration

ExpBackoff returns an OnSourceError handler which does exponential backoff starting with base, doubles for every retry, and stops retrying after 'stop' attempts.

func ExpBackoffThenFail

func ExpBackoffThenFail(base time.Duration, stop int, onFail func(err error)) func(err error, tries int) time.Duration

ExpBackoffThenFail does the same as ExpBackoff but also calls the onFail callback when it stops retrying.

Types

type Runner

type Runner struct {
	// If given, OnSourceError is called if there is any error fetching from
	// the source. tries is how many times has been tried and failed. It should
	// return the wait time before trying again, or zero to stop retrying.
	OnSourceError func(err error, tries int) time.Duration
	// If given, OnSinkError is called if there is any error writing to any of
	// the sinks. There is no retry logic as sinks are local and considered to
	// be more reliable than the source.
	OnSinkError func(sink Sink, err error)

	Validate func(data []byte) error
	// contains filtered or unexported fields
}

Runner runs the logic to synchronizes data from the source to the sinks

func New

func New(from Source, to ...Sink) *Runner

New construct a runner which synchronizes data from one source to one or more sinks

func NewWithValidator

func NewWithValidator(validate func(data []byte) error, from Source, to ...Sink) *Runner

Like New but with a function that validates data before sending it to the sinks

func (*Runner) InitFrom

func (runner *Runner) InitFrom(s Source)

InitFrom synchronizes data from the given source to configured sinks.

func (*Runner) Start

func (runner *Runner) Start(interval time.Duration) func()

Start starts the loop to actually synchronizes data with given interval. It returns a function to stop the loop.

type Sink

type Sink interface {
	UpdateFrom(io.Reader) error
	String() string
}

Sink represents somewhere the data can be written to

func ToChannel

func ToChannel(ch chan []byte) Sink

ToChannel constructs a sink which sends all data to the given channel.

func ToFile

func ToFile(path string) Sink

ToFile constructs a sink from the given file path. Writing to the file while reading from it (via FromFile) won't corrupt the file.

func ToFileWithPreprocessor

func ToFileWithPreprocessor(path string, preprocessor func(io.Reader) (io.Reader, error)) Sink

ToFileWithPreprocessor constructs a sink from the given file path while modifying the data before writing to disk.

type Source

type Source interface {
	// Fetch fetches the data from the source if modified since the designated time.
	Fetch(ifNewerThan time.Time) (io.ReadCloser, error)
}

Source represents somewhere any data can be fetched from

func FromFile

func FromFile(path string) Source

FromFile constructs a source from the given file path.

func FromFileWithPreprocessor

func FromFileWithPreprocessor(path string, preprocessor func(io.ReadCloser) (io.ReadCloser, error)) Source

FromFileWithPreprocessor constructs a source from the given file path, while modifying the file data using preprocessor function

func FromTarGz

func FromTarGz(s Source, expectedName string) Source

FromTarGz wraps a source to decompress one specific file from the gzipped tarball.

func FromWeb

func FromWeb(url string) Source

FromWeb constructs a source from the given URL.

func FromWebWithClient

func FromWebWithClient(url string, client *http.Client) Source

FromWebWithClient is the same as FromWeb but with a custom http.Client