workqueue

Documentation

Overview

Package workqueue provides a simple queue that supports the following features:

• Fair: items processed in the order in which they are added.
• Stingy: a single item will not be processed multiple times concurrently, and if an item is added multiple times before it can be processed, it will only be processed once.
• Multiple consumers and producers. In particular, it is allowed for an item to be reenqueued while it is being processed.
• Shutdown notifications.

Index

• func Parallelize(workers, pieces int, doWorkPiece DoWorkPieceFunc)
• type DelayingInterface
  • func NewDelayingQueue() DelayingInterface
• type DoWorkPieceFunc
• type Interface
• type Type
  • func New() *Type
  • func (q *Type) Add(item interface{})
  • func (q *Type) Done(item interface{})
  • func (q *Type) Get() (item interface{}, shutdown bool)
  • func (q *Type) Len() int
  • func (q *Type) ShutDown()
  • func (q *Type) ShuttingDown() bool

Functions

func Parallelize

func Parallelize(workers, pieces int, doWorkPiece DoWorkPieceFunc)

Parallelize is a very simple framework that allow for parallelizing N independent pieces of work.

Types

type DelayingInterface

type DelayingInterface interface {
	Interface
	// AddAfter adds an item to the workqueue after the indicated duration has passed
	AddAfter(item interface{}, duration time.Duration)
}

DelayingInterface is an Interface that can Add an item at a later time. This makes it easier to requeue items after failures without ending up in a hot-loop.

func NewDelayingQueue

func NewDelayingQueue() DelayingInterface

NewDelayingQueue constructs a new workqueue with delayed queuing ability

type DoWorkPieceFunc

type DoWorkPieceFunc func(piece int)

type Interface

type Interface interface {
	Add(item interface{})
	Len() int
	Get() (item interface{}, shutdown bool)
	Done(item interface{})
	ShutDown()
	ShuttingDown() bool
}

type Type

type Type struct {
	// contains filtered or unexported fields
}

Type is a work queue (see the package comment).

func New

func New() *Type

New constructs a new workqueue (see the package comment).

func (*Type) Add

func (q *Type) Add(item interface{})

Add marks item as needing processing.

func (*Type) Done

func (q *Type) Done(item interface{})

Done marks item as done processing, and if it has been marked as dirty again while it was being processed, it will be re-added to the queue for re-processing.

func (*Type) Get

func (q *Type) Get() (item interface{}, shutdown bool)

Get blocks until it can return an item to be processed. If shutdown = true, the caller should end their goroutine. You must call Done with item when you have finished processing it.

func (*Type) Len

func (q *Type) Len() int

Len returns the current queue length, for informational purposes only. You shouldn't e.g. gate a call to Add() or Get() on Len() being a particular value, that can't be synchronized properly.

func (*Type) ShutDown

func (q *Type) ShutDown()

Shutdown will cause q to ignore all new items added to it. As soon as the worker goroutines have drained the existing items in the queue, they will be instructed to exit.

func (*Type) ShuttingDown

func (q *Type) ShuttingDown() bool