pool

package
v2.11.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 17, 2022 License: MIT Imports: 18 Imported by: 9

Documentation

Index

Constants

View Source
const (
	MB = 1024 * 1024
)
View Source
const NSEC_IN_SEC int64 = 1000000000 //nolint:stylecheck

NSEC_IN_SEC nanoseconds in second

View Source
const (
	// StopRequest can be sent by worker to indicate that restart is required.
	StopRequest = `{"stop":true}`
)

Variables

This section is empty.

Functions

func NewStaticPool

func NewStaticPool(ctx context.Context, cmd Command, factory ipc.Factory, conf interface{}, log *zap.Logger) (pool.Pool, error)

NewStaticPool creates new worker pool and task multiplexer. StaticPool will initiate with one worker.

Types

type Command

type Command func(cmd string) *exec.Cmd

type Config

type Config struct {
	// Debug flag creates new fresh worker before every request.
	Debug bool

	// Command used to override the server command with the custom one
	Command string `mapstructure:"command"`

	// NumWorkers defines how many sub-processes can be run at once. This value
	// might be doubled by Swapper while hot-swap. Defaults to number of CPU cores.
	NumWorkers uint64 `mapstructure:"num_workers"`

	// MaxJobs defines how many executions is allowed for the worker until
	// it's destruction. set 1 to create new process for each new task, 0 to let
	// worker handle as many tasks as it can.
	MaxJobs uint64 `mapstructure:"max_jobs"`

	// AllocateTimeout defines for how long pool will be waiting for a worker to
	// be freed to handle the task. Defaults to 60s.
	AllocateTimeout time.Duration `mapstructure:"allocate_timeout"`

	// DestroyTimeout defines for how long pool should be waiting for worker to
	// properly destroy, if timeout reached worker will be killed. Defaults to 60s.
	DestroyTimeout time.Duration `mapstructure:"destroy_timeout"`

	// Supervision config to limit worker and pool memory usage.
	Supervisor *SupervisorConfig `mapstructure:"supervisor"`
}

Config .. Pool config Configures the pool behavior.

func (*Config) InitDefaults

func (cfg *Config) InitDefaults()

InitDefaults enables default config values.

type ErrorEncoder

type ErrorEncoder func(err error, w worker.BaseProcess) (*payload.Payload, error)

ErrorEncoder encode error or make a decision based on the error type

type Options

type Options func(p *StaticPool)

func WithCustomErrEncoder

func WithCustomErrEncoder(errEnc ErrorEncoder) Options

func WithLogger

func WithLogger(z *zap.Logger) Options

type StaticPool

type StaticPool struct {
	// contains filtered or unexported fields
}

StaticPool controls worker creation, destruction and task routing. Pool uses fixed amount of stack.

func (*StaticPool) Destroy

func (sp *StaticPool) Destroy(ctx context.Context)

Destroy all underlying stack (but let them complete the task).

func (*StaticPool) Exec

func (sp *StaticPool) Exec(p *payload.Payload) (*payload.Payload, error)

Exec executes provided payload on the worker

func (*StaticPool) ExecWithTTL added in v2.9.0

func (sp *StaticPool) ExecWithTTL(ctx context.Context, p *payload.Payload) (*payload.Payload, error)

ExecWithTTL sync with pool.Exec method

func (*StaticPool) GetConfig

func (sp *StaticPool) GetConfig() interface{}

GetConfig returns associated pool configuration. Immutable.

func (*StaticPool) QueueSize added in v2.10.0

func (sp *StaticPool) QueueSize() uint64

func (*StaticPool) RemoveWorker

func (sp *StaticPool) RemoveWorker(wb worker.BaseProcess) error

func (*StaticPool) Reset

func (sp *StaticPool) Reset(ctx context.Context) error

func (*StaticPool) Workers

func (sp *StaticPool) Workers() (workers []worker.BaseProcess)

Workers returns worker list associated with the pool.

type SupervisorConfig

type SupervisorConfig struct {
	// WatchTick defines how often to check the state of worker.
	WatchTick time.Duration `mapstructure:"watch_tick"`

	// TTL defines maximum time worker is allowed to live.
	TTL time.Duration `mapstructure:"ttl"`

	// IdleTTL defines maximum duration worker can spend in idle mode. Disabled when 0.
	IdleTTL time.Duration `mapstructure:"idle_ttl"`

	// ExecTTL defines maximum lifetime per job.
	ExecTTL time.Duration `mapstructure:"exec_ttl"`

	// MaxWorkerMemory limits memory per worker.
	MaxWorkerMemory uint64 `mapstructure:"max_worker_memory"`
}

func (*SupervisorConfig) InitDefaults

func (cfg *SupervisorConfig) InitDefaults()

InitDefaults enables default config values.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL