tickerpool

README

TickerPool

A worker pool of timed tasks, balanced equally to prevent cpu spikes.

This package is for creating a pool of workers which fire on a constant interval. The workers are balanced to avoid cpu spikes by simply dividing the interval by the amount of workers and offsetting their execution by this fraction.

For example, you want to scrape 3 pages one every second you could create a time.NewTicker(time.Second) and scrape all 3 at once but it would be better to create a time.NewTicker(time.Second / 3) and scrape each page 1/3 of a second apart from each other.

Usage

Create a new TickerPool:

func main() {
    tp := tickerpool.NewTickerPool(time.Second)
}

Add some workers:

func addSomePages() {
    tp.Add("page1", func(){
        // scrape
    })
    tp.Add("page2", func(){
        // scrape
    })
    tp.Add("page3", func(){
        // scrape
    })
}

Maybe remove one or two when you don't need them any more:

func imDone() {
    tp.Remove("page1")
}

All while the TickerPool balances the timing between each call automatically.

Documentation

Index

• type TickerPool
  • func NewTickerPool(ctx context.Context, interval time.Duration) (*TickerPool, error)
  • func (tp *TickerPool) Add(name string, task func(context.Context)) (exists bool)
  • func (tp *TickerPool) Exists(name string) bool
  • func (tp *TickerPool) Remove(name string)

Types

type TickerPool

type TickerPool struct {
	Interval time.Duration
	// contains filtered or unexported fields
}

TickerPool represents a pool of workers set to perform a task periodically. A TickerPool has a set of workers which can grow and shrink dynamically.

func NewTickerPool

func NewTickerPool(ctx context.Context, interval time.Duration) (*TickerPool, error)

NewTickerPool creates a TickerPool with a set interval.

func (*TickerPool) Add

func (tp *TickerPool) Add(name string, task func(context.Context)) (exists bool)

Add schedules a new worker to spin up on the TickerPool's interval. The task will not fire right away but as soon as the new worker interval is calculated and the task is assigned a position.

func (*TickerPool) Exists

func (tp *TickerPool) Exists(name string) bool

Exists checks if a worker exists in the pool

func (*TickerPool) Remove

func (tp *TickerPool) Remove(name string)

Remove simply removes a worker from the pool, no other action is required as if there is a queue running, it will stop itself if there are no more workers.