control

Documentation

Overview

Package control controls which resources a query may consume.

The Controller manages the resources available to each query and ensures an optimal use of those resources to execute queries in a timely manner. The controller also maintains the state of a query as it goes through the various stages of execution and is responsible for killing currently executing queries when requested by the user.

The Controller manages when a query is executed. This can be based on anything within the query's requested resources. For example, a basic implementation of the Controller may decide to execute anything with a high priority before anything with a low priority. The implementation of the Controller will vary and change over time and this package may provide multiple implementations for different controller algorithms.

During execution, the Controller manages the resources used by the query and provides observabiility into what resources are being used and by which queries. The Controller also imposes limitations so a query that uses more than its allocated resources or more resources than available on the system will be aborted.

Index

• func StartSpanFromContext(ctx context.Context, operationName string, hist prometheus.Observer, ...) (*span, context.Context)
• type Config
• type Controller
  • func New(c Config) *Controller
  • func (c *Controller) PrometheusCollectors() []prometheus.Collector
  • func (c *Controller) Queries() []*Query
  • func (c *Controller) Query(ctx context.Context, compiler flux.Compiler) (flux.Query, error)
  • func (c *Controller) Shutdown(ctx context.Context) error
• type PriorityQueue
  • func (p *PriorityQueue) Peek() *Query
  • func (p *PriorityQueue) Pop() *Query
  • func (p *PriorityQueue) Push(q *Query)
• type Query
  • func (q *Query) Cancel()
  • func (q *Query) Concurrency() int
  • func (q *Query) Done()
  • func (q *Query) Err() error
  • func (q *Query) ID() QueryID
  • func (q *Query) Ready() <-chan map[string]flux.Result
  • func (q *Query) Spec() *flux.Spec
  • func (q *Query) State() State
  • func (q *Query) Statistics() flux.Statistics
• type QueryID
• type State
  • func (s State) String() string
• type Stringer

Functions

func StartSpanFromContext

func StartSpanFromContext(ctx context.Context, operationName string, hist prometheus.Observer, gauge prometheus.Gauge) (*span, context.Context)

Types

type Config

type Config struct {
	ConcurrencyQuota     int
	MemoryBytesQuota     int64
	ExecutorDependencies execute.Dependencies
	PPlannerOptions      []plan.PhysicalOption
	LPlannerOptions      []plan.LogicalOption
	Logger               *zap.Logger
	// MetricLabelKeys is a list of labels to add to the metrics produced by the controller.
	// The value for a given key will be read off the context.
	// The context value must be a string or an implementation of the Stringer interface.
	MetricLabelKeys []string
}

type Controller

type Controller struct {
	// contains filtered or unexported fields
}

Controller provides a central location to manage all incoming queries. The controller is responsible for queueing, planning, and executing queries.

func New

func New(c Config) *Controller

func (*Controller) PrometheusCollectors

func (c *Controller) PrometheusCollectors() []prometheus.Collector

PrometheusCollectors satisifies the prom.PrometheusCollector interface.

func (*Controller) Queries

func (c *Controller) Queries() []*Query

Queries reports the active queries.

func (*Controller) Query

func (c *Controller) Query(ctx context.Context, compiler flux.Compiler) (flux.Query, error)

Query submits a query for execution returning immediately. Done must be called on any returned Query objects.

func (*Controller) Shutdown

func (c *Controller) Shutdown(ctx context.Context) error

Shutdown will signal to the Controller that it should not accept any new queries and that it should finish executing any existing queries. This will return once the Controller's run loop has been exited and all queries have been finished or until the Context has been canceled.

type PriorityQueue

type PriorityQueue struct {
	// contains filtered or unexported fields
}

func (*PriorityQueue) Peek

func (p *PriorityQueue) Peek() *Query

func (*PriorityQueue) Pop

func (p *PriorityQueue) Pop() *Query

func (*PriorityQueue) Push

func (p *PriorityQueue) Push(q *Query)

type Query

type Query struct {
	// contains filtered or unexported fields
}

Query represents a single request.

func (*Query) Cancel

func (q *Query) Cancel()

Cancel will stop the query execution.

func (*Query) Concurrency

func (q *Query) Concurrency() int

Concurrency reports the number of goroutines allowed to process the request.

func (*Query) Done

func (q *Query) Done()

Done must always be called to free resources.

func (*Query) Err

func (q *Query) Err() error

Err reports any error the query may have encountered.

func (*Query) ID

func (q *Query) ID() QueryID

ID reports an ephemeral unique ID for the query.

func (*Query) Ready

func (q *Query) Ready() <-chan map[string]flux.Result

Ready returns a channel that will deliver the query results. It's possible that the channel is closed before any results arrive. In particular, if a query's context is cancelled during execution, the channel will be closed immediately and an error will be exposed via Err(). For this reason, a query should always be inspected for an error using Err().

func (*Query) Spec

func (q *Query) Spec() *flux.Spec

func (*Query) State

func (q *Query) State() State

State reports the current state of the query.

func (*Query) Statistics

func (q *Query) Statistics() flux.Statistics

Statistics reports the statisitcs for the query. The statisitcs are not complete until the query is finished.

type QueryID

type QueryID uint64

type State

type State int

State is the query state.

const (
	Created State = iota
	Compiling
	Queueing
	Planning
	Requeueing
	Executing
	Errored
	Finished
	Canceled
)

func (State) String

func (s State) String() string

type Stringer

type Stringer interface {
	String() string
}