taskqueue

package
v1.8.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 18, 2024 License: GPL-3.0 Imports: 5 Imported by: 0

README

Task Queue

Task is the interface to the smallest unit of SP background service interaction. Task scheduling and execution are directly related to the order of task arrival, so task queue is a relatively important basic interface used by all modules inside SP.

Concept

Task Queue With Limit

Task execution needs to consume certain resources. Different task types have large differences in Memory, Bandwidth, and CPU consumption. The available resources of the nodes executing the task are uneven. Therefore, resources need to be considered when scheduling tasks. The Task Queue With Limit is to consider resources.

Task Queue Strategy

Conventional queues cannot fully meet the requirements of tasks. For example, the retired strategy of tasks inside the queue, when the conventional queue is full, it cannot be pushed any more, however, tasks that fail after retries may need to be retired. For different types of task retired and pick up, etc. the strategies are different, the Task Queue Strategy is an interface that supports custom strategies.

Task Queue Types

TQueue

TQueue is the interface to task queue. The task queue is mainly used to maintain tasks are running. In addition to supporting conventional FIFO operations, task queue also has some customized operations for task. For example, Has, PopByKey.

TQueueWithLimit

TQueueWithLimit is the interface task queue that takes resources into account. Only tasks with less than required resources can be popped out.

TQueueOnStrategy

TQueueOnStrategy is a combination of TQueue and TQueueStrategy, it is the interface to task queue and the queue supports customize strategies to filter task for popping and retiring task.

TQueueOnStrategyWithLimit

TQueueOnStrategyWithLimit is a combination of TQueueWithLimit and TQueueStrategy,it is the interface to task queue that takes resources into account, and the queue supports customize strategies to filter task for popping and retiring task.

Documentation

Overview

Package taskqueue is a generated GoMock package.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ScanTQueueBySubKey

func ScanTQueueBySubKey(queue TQueue, subKey task.TKey) ([]task.Task, error)

func ScanTQueueWithLimitBySubKey

func ScanTQueueWithLimitBySubKey(queue TQueueWithLimit, subKey task.TKey) ([]task.Task, error)

Types

type MockTQueue added in v0.2.4

type MockTQueue struct {
	// contains filtered or unexported fields
}

MockTQueue is a mock of TQueue interface.

func NewMockTQueue added in v0.2.4

func NewMockTQueue(ctrl *gomock.Controller) *MockTQueue

NewMockTQueue creates a new mock instance.

func (*MockTQueue) Cap added in v0.2.4

func (m *MockTQueue) Cap() int

Cap mocks base method.

func (*MockTQueue) EXPECT added in v0.2.4

func (m *MockTQueue) EXPECT() *MockTQueueMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockTQueue) Has added in v0.2.4

func (m *MockTQueue) Has(arg0 task.TKey) bool

Has mocks base method.

func (*MockTQueue) Len added in v0.2.4

func (m *MockTQueue) Len() int

Len mocks base method.

func (*MockTQueue) Pop added in v0.2.4

func (m *MockTQueue) Pop() task.Task

Pop mocks base method.

func (*MockTQueue) PopByKey added in v0.2.4

func (m *MockTQueue) PopByKey(arg0 task.TKey) task.Task

PopByKey mocks base method.

func (*MockTQueue) Push added in v0.2.4

func (m *MockTQueue) Push(arg0 task.Task) error

Push mocks base method.

func (*MockTQueue) ScanTask added in v0.2.4

func (m *MockTQueue) ScanTask(arg0 func(task.Task))

ScanTask mocks base method.

func (*MockTQueue) Top added in v0.2.4

func (m *MockTQueue) Top() task.Task

Top mocks base method.

type MockTQueueMockRecorder added in v0.2.4

type MockTQueueMockRecorder struct {
	// contains filtered or unexported fields
}

MockTQueueMockRecorder is the mock recorder for MockTQueue.

func (*MockTQueueMockRecorder) Cap added in v0.2.4

func (mr *MockTQueueMockRecorder) Cap() *gomock.Call

Cap indicates an expected call of Cap.

func (*MockTQueueMockRecorder) Has added in v0.2.4

func (mr *MockTQueueMockRecorder) Has(arg0 any) *gomock.Call

Has indicates an expected call of Has.

func (*MockTQueueMockRecorder) Len added in v0.2.4

func (mr *MockTQueueMockRecorder) Len() *gomock.Call

Len indicates an expected call of Len.

func (*MockTQueueMockRecorder) Pop added in v0.2.4

func (mr *MockTQueueMockRecorder) Pop() *gomock.Call

Pop indicates an expected call of Pop.

func (*MockTQueueMockRecorder) PopByKey added in v0.2.4

func (mr *MockTQueueMockRecorder) PopByKey(arg0 any) *gomock.Call

PopByKey indicates an expected call of PopByKey.

func (*MockTQueueMockRecorder) Push added in v0.2.4

func (mr *MockTQueueMockRecorder) Push(arg0 any) *gomock.Call

Push indicates an expected call of Push.

func (*MockTQueueMockRecorder) ScanTask added in v0.2.4

func (mr *MockTQueueMockRecorder) ScanTask(arg0 any) *gomock.Call

ScanTask indicates an expected call of ScanTask.

func (*MockTQueueMockRecorder) Top added in v0.2.4

func (mr *MockTQueueMockRecorder) Top() *gomock.Call

Top indicates an expected call of Top.

type MockTQueueOnStrategy added in v0.2.4

type MockTQueueOnStrategy struct {
	// contains filtered or unexported fields
}

MockTQueueOnStrategy is a mock of TQueueOnStrategy interface.

func NewMockTQueueOnStrategy added in v0.2.4

func NewMockTQueueOnStrategy(ctrl *gomock.Controller) *MockTQueueOnStrategy

NewMockTQueueOnStrategy creates a new mock instance.

func (*MockTQueueOnStrategy) Cap added in v0.2.4

func (m *MockTQueueOnStrategy) Cap() int

Cap mocks base method.

func (*MockTQueueOnStrategy) EXPECT added in v0.2.4

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockTQueueOnStrategy) Has added in v0.2.4

func (m *MockTQueueOnStrategy) Has(arg0 task.TKey) bool

Has mocks base method.

func (*MockTQueueOnStrategy) Len added in v0.2.4

func (m *MockTQueueOnStrategy) Len() int

Len mocks base method.

func (*MockTQueueOnStrategy) Pop added in v0.2.4

func (m *MockTQueueOnStrategy) Pop() task.Task

Pop mocks base method.

func (*MockTQueueOnStrategy) PopByKey added in v0.2.4

func (m *MockTQueueOnStrategy) PopByKey(arg0 task.TKey) task.Task

PopByKey mocks base method.

func (*MockTQueueOnStrategy) Push added in v0.2.4

func (m *MockTQueueOnStrategy) Push(arg0 task.Task) error

Push mocks base method.

func (*MockTQueueOnStrategy) ScanTask added in v0.2.4

func (m *MockTQueueOnStrategy) ScanTask(arg0 func(task.Task))

ScanTask mocks base method.

func (*MockTQueueOnStrategy) SetFilterTaskStrategy added in v0.2.4

func (m *MockTQueueOnStrategy) SetFilterTaskStrategy(arg0 func(task.Task) bool)

SetFilterTaskStrategy mocks base method.

func (*MockTQueueOnStrategy) SetRetireTaskStrategy added in v0.2.4

func (m *MockTQueueOnStrategy) SetRetireTaskStrategy(arg0 func(task.Task) bool)

SetRetireTaskStrategy mocks base method.

func (*MockTQueueOnStrategy) Top added in v0.2.4

func (m *MockTQueueOnStrategy) Top() task.Task

Top mocks base method.

type MockTQueueOnStrategyMockRecorder added in v0.2.4

type MockTQueueOnStrategyMockRecorder struct {
	// contains filtered or unexported fields
}

MockTQueueOnStrategyMockRecorder is the mock recorder for MockTQueueOnStrategy.

func (*MockTQueueOnStrategyMockRecorder) Cap added in v0.2.4

Cap indicates an expected call of Cap.

func (*MockTQueueOnStrategyMockRecorder) Has added in v0.2.4

Has indicates an expected call of Has.

func (*MockTQueueOnStrategyMockRecorder) Len added in v0.2.4

Len indicates an expected call of Len.

func (*MockTQueueOnStrategyMockRecorder) Pop added in v0.2.4

Pop indicates an expected call of Pop.

func (*MockTQueueOnStrategyMockRecorder) PopByKey added in v0.2.4

func (mr *MockTQueueOnStrategyMockRecorder) PopByKey(arg0 any) *gomock.Call

PopByKey indicates an expected call of PopByKey.

func (*MockTQueueOnStrategyMockRecorder) Push added in v0.2.4

Push indicates an expected call of Push.

func (*MockTQueueOnStrategyMockRecorder) ScanTask added in v0.2.4

func (mr *MockTQueueOnStrategyMockRecorder) ScanTask(arg0 any) *gomock.Call

ScanTask indicates an expected call of ScanTask.

func (*MockTQueueOnStrategyMockRecorder) SetFilterTaskStrategy added in v0.2.4

func (mr *MockTQueueOnStrategyMockRecorder) SetFilterTaskStrategy(arg0 any) *gomock.Call

SetFilterTaskStrategy indicates an expected call of SetFilterTaskStrategy.

func (*MockTQueueOnStrategyMockRecorder) SetRetireTaskStrategy added in v0.2.4

func (mr *MockTQueueOnStrategyMockRecorder) SetRetireTaskStrategy(arg0 any) *gomock.Call

SetRetireTaskStrategy indicates an expected call of SetRetireTaskStrategy.

func (*MockTQueueOnStrategyMockRecorder) Top added in v0.2.4

Top indicates an expected call of Top.

type MockTQueueOnStrategyWithLimit added in v0.2.4

type MockTQueueOnStrategyWithLimit struct {
	// contains filtered or unexported fields
}

MockTQueueOnStrategyWithLimit is a mock of TQueueOnStrategyWithLimit interface.

func NewMockTQueueOnStrategyWithLimit added in v0.2.4

func NewMockTQueueOnStrategyWithLimit(ctrl *gomock.Controller) *MockTQueueOnStrategyWithLimit

NewMockTQueueOnStrategyWithLimit creates a new mock instance.

func (*MockTQueueOnStrategyWithLimit) Cap added in v0.2.4

Cap mocks base method.

func (*MockTQueueOnStrategyWithLimit) EXPECT added in v0.2.4

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockTQueueOnStrategyWithLimit) Has added in v0.2.4

Has mocks base method.

func (*MockTQueueOnStrategyWithLimit) Len added in v0.2.4

Len mocks base method.

func (*MockTQueueOnStrategyWithLimit) PopByKey added in v0.2.4

func (m *MockTQueueOnStrategyWithLimit) PopByKey(arg0 task.TKey) task.Task

PopByKey mocks base method.

func (*MockTQueueOnStrategyWithLimit) PopByLimit added in v0.2.4

func (m *MockTQueueOnStrategyWithLimit) PopByLimit(arg0 rcmgr.Limit) task.Task

PopByLimit mocks base method.

func (*MockTQueueOnStrategyWithLimit) Push added in v0.2.4

Push mocks base method.

func (*MockTQueueOnStrategyWithLimit) ScanTask added in v0.2.4

func (m *MockTQueueOnStrategyWithLimit) ScanTask(arg0 func(task.Task))

ScanTask mocks base method.

func (*MockTQueueOnStrategyWithLimit) SetFilterTaskStrategy added in v0.2.4

func (m *MockTQueueOnStrategyWithLimit) SetFilterTaskStrategy(arg0 func(task.Task) bool)

SetFilterTaskStrategy mocks base method.

func (*MockTQueueOnStrategyWithLimit) SetRetireTaskStrategy added in v0.2.4

func (m *MockTQueueOnStrategyWithLimit) SetRetireTaskStrategy(arg0 func(task.Task) bool)

SetRetireTaskStrategy mocks base method.

func (*MockTQueueOnStrategyWithLimit) TopByLimit added in v0.2.4

func (m *MockTQueueOnStrategyWithLimit) TopByLimit(arg0 rcmgr.Limit) task.Task

TopByLimit mocks base method.

type MockTQueueOnStrategyWithLimitMockRecorder added in v0.2.4

type MockTQueueOnStrategyWithLimitMockRecorder struct {
	// contains filtered or unexported fields
}

MockTQueueOnStrategyWithLimitMockRecorder is the mock recorder for MockTQueueOnStrategyWithLimit.

func (*MockTQueueOnStrategyWithLimitMockRecorder) Cap added in v0.2.4

Cap indicates an expected call of Cap.

func (*MockTQueueOnStrategyWithLimitMockRecorder) Has added in v0.2.4

Has indicates an expected call of Has.

func (*MockTQueueOnStrategyWithLimitMockRecorder) Len added in v0.2.4

Len indicates an expected call of Len.

func (*MockTQueueOnStrategyWithLimitMockRecorder) PopByKey added in v0.2.4

PopByKey indicates an expected call of PopByKey.

func (*MockTQueueOnStrategyWithLimitMockRecorder) PopByLimit added in v0.2.4

PopByLimit indicates an expected call of PopByLimit.

func (*MockTQueueOnStrategyWithLimitMockRecorder) Push added in v0.2.4

Push indicates an expected call of Push.

func (*MockTQueueOnStrategyWithLimitMockRecorder) ScanTask added in v0.2.4

ScanTask indicates an expected call of ScanTask.

func (*MockTQueueOnStrategyWithLimitMockRecorder) SetFilterTaskStrategy added in v0.2.4

func (mr *MockTQueueOnStrategyWithLimitMockRecorder) SetFilterTaskStrategy(arg0 any) *gomock.Call

SetFilterTaskStrategy indicates an expected call of SetFilterTaskStrategy.

func (*MockTQueueOnStrategyWithLimitMockRecorder) SetRetireTaskStrategy added in v0.2.4

func (mr *MockTQueueOnStrategyWithLimitMockRecorder) SetRetireTaskStrategy(arg0 any) *gomock.Call

SetRetireTaskStrategy indicates an expected call of SetRetireTaskStrategy.

func (*MockTQueueOnStrategyWithLimitMockRecorder) TopByLimit added in v0.2.4

TopByLimit indicates an expected call of TopByLimit.

type MockTQueueStrategy added in v0.2.4

type MockTQueueStrategy struct {
	// contains filtered or unexported fields
}

MockTQueueStrategy is a mock of TQueueStrategy interface.

func NewMockTQueueStrategy added in v0.2.4

func NewMockTQueueStrategy(ctrl *gomock.Controller) *MockTQueueStrategy

NewMockTQueueStrategy creates a new mock instance.

func (*MockTQueueStrategy) EXPECT added in v0.2.4

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockTQueueStrategy) SetFilterTaskStrategy added in v0.2.4

func (m *MockTQueueStrategy) SetFilterTaskStrategy(arg0 func(task.Task) bool)

SetFilterTaskStrategy mocks base method.

func (*MockTQueueStrategy) SetRetireTaskStrategy added in v0.2.4

func (m *MockTQueueStrategy) SetRetireTaskStrategy(arg0 func(task.Task) bool)

SetRetireTaskStrategy mocks base method.

type MockTQueueStrategyMockRecorder added in v0.2.4

type MockTQueueStrategyMockRecorder struct {
	// contains filtered or unexported fields
}

MockTQueueStrategyMockRecorder is the mock recorder for MockTQueueStrategy.

func (*MockTQueueStrategyMockRecorder) SetFilterTaskStrategy added in v0.2.4

func (mr *MockTQueueStrategyMockRecorder) SetFilterTaskStrategy(arg0 any) *gomock.Call

SetFilterTaskStrategy indicates an expected call of SetFilterTaskStrategy.

func (*MockTQueueStrategyMockRecorder) SetRetireTaskStrategy added in v0.2.4

func (mr *MockTQueueStrategyMockRecorder) SetRetireTaskStrategy(arg0 any) *gomock.Call

SetRetireTaskStrategy indicates an expected call of SetRetireTaskStrategy.

type MockTQueueWithLimit added in v0.2.4

type MockTQueueWithLimit struct {
	// contains filtered or unexported fields
}

MockTQueueWithLimit is a mock of TQueueWithLimit interface.

func NewMockTQueueWithLimit added in v0.2.4

func NewMockTQueueWithLimit(ctrl *gomock.Controller) *MockTQueueWithLimit

NewMockTQueueWithLimit creates a new mock instance.

func (*MockTQueueWithLimit) Cap added in v0.2.4

func (m *MockTQueueWithLimit) Cap() int

Cap mocks base method.

func (*MockTQueueWithLimit) EXPECT added in v0.2.4

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockTQueueWithLimit) Has added in v0.2.4

func (m *MockTQueueWithLimit) Has(arg0 task.TKey) bool

Has mocks base method.

func (*MockTQueueWithLimit) Len added in v0.2.4

func (m *MockTQueueWithLimit) Len() int

Len mocks base method.

func (*MockTQueueWithLimit) PopByKey added in v0.2.4

func (m *MockTQueueWithLimit) PopByKey(arg0 task.TKey) task.Task

PopByKey mocks base method.

func (*MockTQueueWithLimit) PopByLimit added in v0.2.4

func (m *MockTQueueWithLimit) PopByLimit(arg0 rcmgr.Limit) task.Task

PopByLimit mocks base method.

func (*MockTQueueWithLimit) Push added in v0.2.4

func (m *MockTQueueWithLimit) Push(arg0 task.Task) error

Push mocks base method.

func (*MockTQueueWithLimit) ScanTask added in v0.2.4

func (m *MockTQueueWithLimit) ScanTask(arg0 func(task.Task))

ScanTask mocks base method.

func (*MockTQueueWithLimit) TopByLimit added in v0.2.4

func (m *MockTQueueWithLimit) TopByLimit(arg0 rcmgr.Limit) task.Task

TopByLimit mocks base method.

type MockTQueueWithLimitMockRecorder added in v0.2.4

type MockTQueueWithLimitMockRecorder struct {
	// contains filtered or unexported fields
}

MockTQueueWithLimitMockRecorder is the mock recorder for MockTQueueWithLimit.

func (*MockTQueueWithLimitMockRecorder) Cap added in v0.2.4

Cap indicates an expected call of Cap.

func (*MockTQueueWithLimitMockRecorder) Has added in v0.2.4

Has indicates an expected call of Has.

func (*MockTQueueWithLimitMockRecorder) Len added in v0.2.4

Len indicates an expected call of Len.

func (*MockTQueueWithLimitMockRecorder) PopByKey added in v0.2.4

func (mr *MockTQueueWithLimitMockRecorder) PopByKey(arg0 any) *gomock.Call

PopByKey indicates an expected call of PopByKey.

func (*MockTQueueWithLimitMockRecorder) PopByLimit added in v0.2.4

func (mr *MockTQueueWithLimitMockRecorder) PopByLimit(arg0 any) *gomock.Call

PopByLimit indicates an expected call of PopByLimit.

func (*MockTQueueWithLimitMockRecorder) Push added in v0.2.4

Push indicates an expected call of Push.

func (*MockTQueueWithLimitMockRecorder) ScanTask added in v0.2.4

func (mr *MockTQueueWithLimitMockRecorder) ScanTask(arg0 any) *gomock.Call

ScanTask indicates an expected call of ScanTask.

func (*MockTQueueWithLimitMockRecorder) TopByLimit added in v0.2.4

func (mr *MockTQueueWithLimitMockRecorder) TopByLimit(arg0 any) *gomock.Call

TopByLimit indicates an expected call of TopByLimit.

type NewTQueue

type NewTQueue = func(name string, cap int) TQueue

NewTQueue defines the new func type of TQueue

type NewTQueueOnStrategy

type NewTQueueOnStrategy = func(name string, cap int) TQueueOnStrategy

NewTQueueOnStrategy defines the new func type of TQueueOnStrategy

type NewTQueueOnStrategyWithLimit

type NewTQueueOnStrategyWithLimit = func(name string, cap int) TQueueOnStrategyWithLimit

NewTQueueOnStrategyWithLimit the new func type of TQueueOnStrategyWithLimit

type NewTQueueWithLimit

type NewTQueueWithLimit = func(name string, cap int) TQueueWithLimit

NewTQueueWithLimit defines the new func type of TQueueWithLimit

type NilQueue

type NilQueue struct{}

func (*NilQueue) Cap

func (*NilQueue) Cap() int

func (*NilQueue) Has

func (*NilQueue) Has(task.TKey) bool

func (*NilQueue) Len

func (*NilQueue) Len() int

func (*NilQueue) Pop

func (*NilQueue) Pop() task.Task

func (*NilQueue) PopByKey

func (*NilQueue) PopByKey(task.TKey) task.Task

func (*NilQueue) PopByLimit

func (*NilQueue) PopByLimit(rcmgr.Limit) task.Task

func (*NilQueue) Push

func (*NilQueue) Push(task.Task) error

func (*NilQueue) ScanTask

func (*NilQueue) ScanTask(func(task.Task))

func (*NilQueue) SetFilterTaskStrategy

func (*NilQueue) SetFilterTaskStrategy(func(task.Task) bool)

func (*NilQueue) SetRetireTaskStrategy

func (*NilQueue) SetRetireTaskStrategy(func(task.Task) bool)

func (*NilQueue) Top

func (*NilQueue) Top() task.Task

func (*NilQueue) TopByLimit

func (*NilQueue) TopByLimit(rcmgr.Limit) task.Task

type TQueue

type TQueue interface {
	// Top returns the top task in the queue, if the queue empty, returns nil.
	Top() task.Task
	// Pop pops and returns the top task in queue, if the queue empty, returns nil.
	Pop() task.Task
	// PopByKey pops the task by the task key, if the task does not exist , returns nil.
	PopByKey(task.TKey) task.Task
	// Has returns an indicator whether the task in queue.
	Has(task.TKey) bool
	// Push pushes the task in queue tail, if the queue len greater the capacity, returns error.
	Push(task.Task) error
	// Len returns the length of queue.
	Len() int
	// Cap returns the capacity of queue.
	Cap() int
	// ScanTask scans all tasks, and call the func one by one task.
	ScanTask(func(task.Task))
}

TQueue is the interface to task queue. The task queue is mainly used to maintain tasks are running. In addition to supporting conventional FIFO operations, task queue also has some customized operations for task. For example, Has, PopByKey.

type TQueueOnStrategy

type TQueueOnStrategy interface {
	TQueue
	TQueueStrategy
}

TQueueOnStrategy is the interface to task queue and the queue supports customize strategies to filter task for popping and retiring task.

type TQueueOnStrategyWithLimit

type TQueueOnStrategyWithLimit interface {
	TQueueWithLimit
	TQueueStrategy
}

TQueueOnStrategyWithLimit is the interface to task queue that takes resources into account, and the queue supports customize strategies to filter task for popping and retiring task.

type TQueueStrategy

type TQueueStrategy interface {
	// SetFilterTaskStrategy sets the callback func to filter task for popping or topping.
	SetFilterTaskStrategy(func(task.Task) bool)
	// SetRetireTaskStrategy sets the callback func to retire task, when the queue is full, it will be
	// called to retire tasks.
	SetRetireTaskStrategy(func(task.Task) bool)
}

TQueueStrategy is the interface to queue customize strategies, it supports filter task for popping and retiring task strategies.

type TQueueWithLimit

type TQueueWithLimit interface {
	// TopByLimit returns the top task that the LimitEstimate less than the param in the queue.
	TopByLimit(rcmgr.Limit) task.Task
	// PopByLimit pops and returns the top task that the LimitEstimate less than the param in the queue.
	PopByLimit(rcmgr.Limit) task.Task
	// PopByKey pops the task by the task key, if the task does not exist , returns nil.
	PopByKey(task.TKey) task.Task
	// Has returns an indicator whether the task in queue.
	Has(task.TKey) bool
	// Push pushes the task in queue tail, if the queue len greater the capacity, returns error.
	Push(task.Task) error
	// Len returns the length of queue.
	Len() int
	// Cap returns the capacity of queue.
	Cap() int
	// ScanTask scans all tasks, and call the func one by one task.
	ScanTask(func(task.Task))
}

TQueueWithLimit is the interface task queue that takes resources into account. Only tasks with less than required resources can be popped out.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL