jobtracker

README

jobtracker - a self terminating concurrent job queue for indeterminate workloads in golang

This library is primarily useful for technically-recursive work with an unknown size at the start of the work. It was originally created for goop where i had a need for a concurrent job queue where jobs have the ability to queue new jobs, which terminates itself once all work is done (only indicated by the lack of new jobs).

Installation

go get -u github.com/deletescape/jobtracker@latest

Examples

You can find some examples in the _examples/ directory.

Projects using jobtracker

• goop

Documentation

Index

• func DefaultNapper(naps int32, activeWorkers int32)
• type Context
• type JobTracker
  • func NewJobTracker(worker Worker, maxConcurrency int32, napper Napper) *JobTracker
  • func (jt *JobTracker) AddJob(job string)
  • func (jt *JobTracker) AddJobs(jobs ...string)
  • func (jt *JobTracker) StartAndWait(context Context, forceMaxConcurrency bool)
• type Napper
• type Worker

Functions

func DefaultNapper

func DefaultNapper(naps int32, activeWorkers int32)

DefaultNapper will sleep a minimum of 15 milliseconds per nap, but increasingly longer the more often workers go idle this ensures that idle workers do not take up too much cpu time, and should prevent accidental early exits

Types

type Context

type Context interface{}

type JobTracker

type JobTracker struct {
	// contains filtered or unexported fields
}

func NewJobTracker

func NewJobTracker(worker Worker, maxConcurrency int32, napper Napper) *JobTracker

func (*JobTracker) AddJob

func (jt *JobTracker) AddJob(job string)

func (*JobTracker) AddJobs

func (jt *JobTracker) AddJobs(jobs ...string)

func (*JobTracker) StartAndWait

func (jt *JobTracker) StartAndWait(context Context, forceMaxConcurrency bool)

type Napper

type Napper func(naps, activeWorkers int32)

type Worker

type Worker func(jt *JobTracker, job string, context Context)