queue

Documentation

Index

• Constants
• func MarkComplete(conn redis.Conn, qType string, token WorkerToken) error
• func PushOntoQueue(conn redis.Conn, qType string, queue string, tps int, value string, ...) error
• func StartDethrottler(redis *redis.Pool, quitter chan bool, wg *sync.WaitGroup, qType string)
• type Priority
• type WorkerToken
  • func PopFromQueue(conn redis.Conn, qType string) (WorkerToken, string, error)

Constants

const (
	// DefaultPriority is the normal priority for msgs, messages are sent first in first out
	DefaultPriority = 1

	// BulkPriority is our priority for bulk messages (sent in batches) These will only be
	// processed after all default priority messages are deault with
	BulkPriority = 0
)

const (
	// EmptyQueue means there are no items to retrive, caller should sleep and try again later
	EmptyQueue = WorkerToken("empty")

	// Retry means the caller should immediately call again to get the next value
	Retry = WorkerToken("retry")
)

Functions

func MarkComplete

func MarkComplete(conn redis.Conn, qType string, token WorkerToken) error

MarkComplete marks a task as complete for the passed in queue and queue result. It is important for callers to call this so that workers are evenly spread across all queues with jobs in them

func PushOntoQueue

func PushOntoQueue(conn redis.Conn, qType string, queue string, tps int, value string, priority Priority) error

PushOntoQueue pushes the passed in value to the passed in queue, making sure that no more than the specified transactions per second are popped off at a time. A tps value of 0 means there is no limit to the rate that messages can be consumed

func StartDethrottler

func StartDethrottler(redis *redis.Pool, quitter chan bool, wg *sync.WaitGroup, qType string)

StartDethrottler starts a goroutine responsible for dethrottling any queues that were throttled every second. The passed in quitter chan can be used to shut down the goroutine

Types

type Priority

type Priority int64

Priority represents the priority of an item in a queue

type WorkerToken

type WorkerToken string

WorkerToken represents a token that a worker should return when a task is complete

func PopFromQueue

func PopFromQueue(conn redis.Conn, qType string) (WorkerToken, string, error)

PopFromQueue pops the next available message from the passed in queue. If QueueRetry is returned the caller should immediately make another call to get the next value. A worker token of EmptyQueue will be returned if there are no more items to retrive. Otherwise the WorkerToken should be saved in order to mark the task as complete later.