taskqueue

README

Task Queue

Task is the interface to the smallest unit of SP background service interaction. Task scheduling and execution are directly related to the order of task arrival, so task queue is a relatively important basic interface used by all modules inside SP.

Concept

Task Queue With Limit

Task execution needs to consume certain resources. Different task types have large differences in Memory, Bandwidth, and CPU consumption. The available resources of the nodes executing the task are uneven. Therefore, resources need to be considered when scheduling tasks. The Task Queue With Limit is to consider resources.

Task Queue Strategy

Conventional queues cannot fully meet the requirements of tasks. For example, the retired strategy of tasks inside the queue, when the conventional queue is full, it cannot be pushed any more, however, tasks that fail after retries may need to be retired. For different types of task retired and pick up, etc. the strategies are different, the Task Queue Strategy is an interface that supports custom strategies.

Task Queue Types

TQueue

TQueue is the interface to task queue. The task queue is mainly used to maintain tasks are running. In addition to supporting conventional FIFO operations, task queue also has some customized operations for task. For example, Has, PopByKey.

TQueueWithLimit

TQueueWithLimit is the interface task queue that takes resources into account. Only tasks with less than required resources can be popped out.

TQueueOnStrategy

TQueueOnStrategy is a combination of TQueue and TQueueStrategy， it is the interface to task queue and the queue supports customize strategies to filter task for popping and retiring task.

TQueueOnStrategyWithLimit

TQueueOnStrategyWithLimit is a combination of TQueueWithLimit and TQueueStrategy，it is the interface to task queue that takes resources into account, and the queue supports customize strategies to filter task for popping and retiring task.

Documentation

Index

• func ScanTQueueBySubKey(queue TQueue, subKey task.TKey) ([]task.Task, error)
• func ScanTQueueWithLimitBySubKey(queue TQueueWithLimit, subKey task.TKey) ([]task.Task, error)
• type NewTQueue
• type NewTQueueOnStrategy
• type NewTQueueOnStrategyWithLimit
• type NewTQueueWithLimit
• type NilQueue
  • func (*NilQueue) Cap() int
  • func (*NilQueue) Has(task.TKey) bool
  • func (*NilQueue) Len() int
  • func (*NilQueue) Pop() task.Task
  • func (*NilQueue) PopByKey(task.TKey) task.Task
  • func (*NilQueue) PopByLimit(rcmgr.Limit) task.Task
  • func (*NilQueue) Push(task.Task) error
  • func (*NilQueue) ScanTask(func(task.Task))
  • func (*NilQueue) SetFilterTaskStrategy(func(task.Task) bool)
  • func (*NilQueue) SetRetireTaskStrategy(func(task.Task) bool)
  • func (*NilQueue) Top() task.Task
  • func (*NilQueue) TopByLimit(rcmgr.Limit) task.Task
• type TQueue
• type TQueueOnStrategy
• type TQueueOnStrategyWithLimit
• type TQueueStrategy
• type TQueueWithLimit

Functions

func ScanTQueueBySubKey

func ScanTQueueBySubKey(queue TQueue, subKey task.TKey) ([]task.Task, error)

func ScanTQueueWithLimitBySubKey

func ScanTQueueWithLimitBySubKey(queue TQueueWithLimit, subKey task.TKey) ([]task.Task, error)

Types

type NewTQueue

type NewTQueue = func(name string, cap int) TQueue

NewTQueue defines the new func type of TQueue

type NewTQueueOnStrategy

type NewTQueueOnStrategy = func(name string, cap int) TQueueOnStrategy

NewTQueueOnStrategy defines the new func type of TQueueOnStrategy

type NewTQueueOnStrategyWithLimit

type NewTQueueOnStrategyWithLimit = func(name string, cap int) TQueueOnStrategyWithLimit

NewTQueueOnStrategyWithLimit the new func type of TQueueOnStrategyWithLimit

type NewTQueueWithLimit

type NewTQueueWithLimit = func(name string, cap int) TQueueWithLimit

NewTQueueWithLimit defines the new func type of TQueueWithLimit

type NilQueue

type NilQueue struct{}

func (*NilQueue) Cap

func (*NilQueue) Cap() int

func (*NilQueue) Has

func (*NilQueue) Has(task.TKey) bool

func (*NilQueue) Len

func (*NilQueue) Len() int

func (*NilQueue) Pop

func (*NilQueue) Pop() task.Task

func (*NilQueue) PopByKey

func (*NilQueue) PopByKey(task.TKey) task.Task

func (*NilQueue) PopByLimit

func (*NilQueue) PopByLimit(rcmgr.Limit) task.Task

func (*NilQueue) Push

func (*NilQueue) Push(task.Task) error

func (*NilQueue) ScanTask

func (*NilQueue) ScanTask(func(task.Task))

func (*NilQueue) SetFilterTaskStrategy

func (*NilQueue) SetFilterTaskStrategy(func(task.Task) bool)

func (*NilQueue) SetRetireTaskStrategy

func (*NilQueue) SetRetireTaskStrategy(func(task.Task) bool)

func (*NilQueue) Top

func (*NilQueue) Top() task.Task

func (*NilQueue) TopByLimit

func (*NilQueue) TopByLimit(rcmgr.Limit) task.Task

type TQueue

type TQueue interface {
	// Top returns the top task in the queue, if the queue empty, returns nil.
	Top() task.Task
	// Pop pops and returns the top task in queue, if the queue empty, returns nil.
	Pop() task.Task
	// PopByKey pops the task by the task key, if the task does not exist , returns nil.
	PopByKey(task.TKey) task.Task
	// Has returns an indicator whether the task in queue.
	Has(task.TKey) bool
	// Push pushes the task in queue tail, if the queue len greater the capacity, returns error.
	Push(task.Task) error
	// Len returns the length of queue.
	Len() int
	// Cap returns the capacity of queue.
	Cap() int
	// ScanTask scans all tasks, and call the func one by one task.
	ScanTask(func(task.Task))
}

TQueue is the interface to task queue. The task queue is mainly used to maintain tasks are running. In addition to supporting conventional FIFO operations, task queue also has some customized operations for task. For example, Has, PopByKey.

type TQueueOnStrategy

type TQueueOnStrategy interface {
	TQueue
	TQueueStrategy
}

TQueueOnStrategy is the interface to task queue and the queue supports customize strategies to filter task for popping and retiring task.

type TQueueOnStrategyWithLimit

type TQueueOnStrategyWithLimit interface {
	TQueueWithLimit
	TQueueStrategy
}

TQueueOnStrategyWithLimit is the interface to task queue that takes resources into account, and the queue supports customize strategies to filter task for popping and retiring task.

type TQueueStrategy

type TQueueStrategy interface {
	// SetFilterTaskStrategy sets the callback func to filter task for popping or topping.
	SetFilterTaskStrategy(func(task.Task) bool)
	// SetRetireTaskStrategy sets the callback func to retire task, when the queue is full, it will be
	// called to retire tasks.
	SetRetireTaskStrategy(func(task.Task) bool)
}

TQueueStrategy is the interface to queue customize strategies, it supports filter task for popping and retiring task strategies.

type TQueueWithLimit

type TQueueWithLimit interface {
	// TopByLimit returns the top task that the LimitEstimate less than the param in the queue.
	TopByLimit(rcmgr.Limit) task.Task
	// PopByLimit pops and returns the top task that the LimitEstimate less than the param in the queue.
	PopByLimit(rcmgr.Limit) task.Task
	// PopByKey pops the task by the task key, if the task does not exist , returns nil.
	PopByKey(task.TKey) task.Task
	// Has returns an indicator whether the task in queue.
	Has(task.TKey) bool
	// Push pushes the task in queue tail, if the queue len greater the capacity, returns error.
	Push(task.Task) error
	// Len returns the length of queue.
	Len() int
	// Cap returns the capacity of queue.
	Cap() int
	// ScanTask scans all tasks, and call the func one by one task.
	ScanTask(func(task.Task))
}

TQueueWithLimit is the interface task queue that takes resources into account. Only tasks with less than required resources can be popped out.