bgwork

Documentation

Overview

Package bgwork handles background work or jobs distributed over many machines.

TODO: gather ideas - https://blog.gobuffalo.io/buffalo-v0-9-0-released-68fcf0844473 - https://github.com/gocraft/work - https://medium.com/@matryer/introducing-vice-go-channels-across-many-machines-bcac1147d7e2

Index

• func Async(fns ...func() error) <-chan error
• func AutoScaling(ctx context.Context, chanEvents <-chan interface{}, ...)
• func ScheduleWorkAt(ctx context.Context, opts ScheduleOptions, t func() error)
• func Wait(length int, block func(index int))
• func WaitContext(length int, block func(ctx context.Context, index int) error) error
• type ScalingOptions
• type ScheduleOptions

Functions

func Async

func Async(fns ...func() error) <-chan error

Async runs the given functions asynchronous. It returns all errors, if there are. The channel gets closed once all functions are done.

func AutoScaling

func AutoScaling(ctx context.Context, chanEvents <-chan interface{}, jobFn func(event interface{}), opt ScalingOptions)

AutoScaling runs the same task across min to max background workers. Idle workers are getting terminated until the min amount of workers get reached. There won't be more workers than max amount. AutoScaling blocks once called so it must start in its own goroutine.

func ScheduleWorkAt

func ScheduleWorkAt(ctx context.Context, opts ScheduleOptions, t func() error)

ScheduleWorkAt runs the task t

func Wait

func Wait(length int, block func(index int))

Wait is a better pattern around using sync.WaitGroup. For returning errors while working on a subtask of a common task, consider using package x/errgroup.

func WaitContext

func WaitContext(length int, block func(ctx context.Context, index int) error) error

WaitContext is a better pattern around using errgroup.Group. See function Wait. The returned error of the block function cancels the context.

Types

type ScalingOptions

type ScalingOptions struct {
	// MinWorkers defaults to 2
	MinWorkers uint16
	// MaxWorkers defaults to 16
	MaxWorkers uint16
	// WorkerCheckInterval defaults to one second. Duration when to check for
	// termination and/or creation of workers. Each interval the GetStatistics
	// function will be called.
	WorkerCheckInterval time.Duration
	// GetStatistics gets called each WorkerCheckInterval
	GetStatistics func(string)
}

ScalingOptions sets various configurations to AutoScaling function.

type ScheduleOptions

type ScheduleOptions struct {
	Log      log.Logger
	Truncate time.Duration
	Sleep    time.Duration // duration to sleep for the next call of function t
	Backoff  time.Duration // delay the next try in duration x in case of an error, default 5s.
}