execution

package
v0.7.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Aug 18, 2017 License: MIT Imports: 14 Imported by: 0

Documentation

Index

Constants

View Source
const (
	// UnknownVolatility describes an unset value. This is not
	// a valid return value for FlatExpression.Volatility().
	UnknownVolatility = iota
	// Volatile expressions can do anything, in particular return a
	// different result on every call
	Volatile
	// Stable expressions return the same result for the same input
	// values within a single statement execution
	Stable
	// Immutable expressions return the same result for the same input
	// values forever. One good hint to distinguish between Stable
	// and Immutable is that (in PostgreSQL) Immutable functions can
	// be used in functional indexes, while Stable functions can't.
	Immutable
)
View Source
const (
	MaxRangeTuples   float64 = 1<<20 - 1
	MaxRangeSec      float64 = 60 * 60 * 24
	MaxRangeMillisec float64 = 60 * 60 * 24 * 1000
)

Variables

This section is empty.

Functions

func CanBuildDefaultSelectExecutionPlan

func CanBuildDefaultSelectExecutionPlan(lp *LogicalPlan, reg udf.FunctionRegistry) bool

CanBuildDefaultSelectExecutionPlan checks whether the given statement allows to use an defaultSelectExecutionPlan.

func CanBuildFilterPlan

func CanBuildFilterPlan(lp *LogicalPlan, reg udf.FunctionRegistry) bool

CanBuildFilterPlan checks whether the given statement allows to use a filterPlan.

func CanBuildGroupbyExecutionPlan

func CanBuildGroupbyExecutionPlan(lp *LogicalPlan, reg udf.FunctionRegistry) bool

CanBuildGroupbyExecutionPlan checks whether the given statement allows to use an groupbyExecutionPlan.

func EvaluateFoldable

func EvaluateFoldable(expr parser.Expression, reg udf.FunctionRegistry) (data.Value, error)

EvaluateFoldable evaluates a foldable expression, i.e., one that is independent from the input row. Note that foldable is not necessarily equivalent to constant (e.g., the expression `random()` is foldable, but not constant), and also note that this function should not be used for frequent evaluation of the same expression due to performance reasons.

func EvaluateOnInput

func EvaluateOnInput(expr parser.Expression, input data.Value, reg udf.FunctionRegistry) (data.Value, error)

EvaluateOnInput evaluates a (not necessarily foldable) expression, given a Map that represents a row of data.

Types

type Evaluator

type Evaluator interface {
	// Eval evaluates the expression that this Evaluator represents
	// on the given input data. Note that in order to deal with joins and
	// meta information such as timestamps properly, the input data must have
	// the shape:
	//   {"alias_1": {"col_0": ..., "col_1": ...},
	//    "alias_1:meta:x": (meta datum "x" for alias_1's row),
	//    "alias_2": {"col_0": ..., "col_1": ...},
	//    "alias_2:meta:x": (meta datum "x" for alias_2's row),
	//    ...}
	// and every caller (in particular all execution plans)
	// must ensure that the data has this shape even if there's only one input
	// stream.
	//
	// Eval must NOT modify the input.
	Eval(input data.Value) (data.Value, error)
}

An Evaluator represents an expression such as `colX + 2` or `t1:col AND t2:col` and can be evaluated, given the actual data contained in one row.

func ExpressionToEvaluator

func ExpressionToEvaluator(ast FlatExpression, reg udf.FunctionRegistry) (Evaluator, error)

ExpressionToEvaluator takes one of the Expression structs that result from parsing a BQL Expression (see parser/ast.go) and turns it into an Evaluator that can be used to evaluate an expression given a particular input Value.

func FuncApp

func FuncApp(name string, f udf.UDF, ctx *core.Context, params []Evaluator) Evaluator

FuncApp represents evaluation of a function on a number of parameters that are expressions over an input Value.

type FlatExpression

type FlatExpression interface {
	// Repr returns a string representation that can be used to
	// identify this expression (e.g., "stream:col+3") and used as
	// a dictionary key for finding duplicate expressions.
	Repr() string

	// Columns returns a list of rowValues used in this expression.
	Columns() []rowValue

	// Volatility returns the volatility of an expression.
	Volatility() VolatilityType

	// ContainsWildcard returns whether this expression contains
	// a wildcard symbol.
	ContainsWildcard() bool
}

FlatExpression represents an expression that can be completely evaluated on a single row and results in an unnamed value. In particular, it cannot contain/represent a call to an aggregate function.

func ParserExprToFlatExpr

func ParserExprToFlatExpr(e parser.Expression, reg udf.FunctionRegistry) (FlatExpression, error)

ParserExprToFlatExpr converts an expression obtained by the BQL parser to a FlatExpression, i.e., there are only expressions contained that can be evaluated on one single row and return an (unnamed) value. In particular, this fails for Expressions containing aggregate functions.

func ParserExprToMaybeAggregate

func ParserExprToMaybeAggregate(e parser.Expression, aggIdx int, reg udf.FunctionRegistry) (FlatExpression, map[string]FlatExpression, error)

ParserExprToMaybeAggregate converts an expression obtained by the BQL parser into a data structure where the aggregate and the non-aggregate parts are separated.

type LogicalPlan

type LogicalPlan struct {
	GroupingStmt        bool
	EmitterType         parser.Emitter
	EmitterLimit        int64
	EmitterSampling     float64
	EmitterSamplingType parser.EmitterSamplingType
	Projections         []aliasedExpression
	parser.WindowedFromAST
	Filter    FlatExpression
	GroupList []FlatExpression
	parser.HavingAST
}

LogicalPlan represents a parsed and analyzed version of a SELECT statement. A LogicalPlan as returned by `Analyze` should not contain logical errors such as "... must appear in GROUP BY clause" etc.

func Analyze

Analyze checks the given SELECT statement for logical errors (references to unknown tables etc.) and creates a LogicalPlan that is internally consistent.

func (*LogicalPlan) LogicalOptimize

func (lp *LogicalPlan) LogicalOptimize() (*LogicalPlan, error)

LogicalOptimize does nothing at the moment. In the future, logical optimizations (evaluation of foldable terms etc.) can be added here.

func (*LogicalPlan) MakePhysicalPlan

func (lp *LogicalPlan) MakePhysicalPlan(reg udf.FunctionRegistry) (PhysicalPlan, error)

MakePhysicalPlan creates a physical execution plan that is able to deal with the statement under consideration.

type PhysicalPlan

type PhysicalPlan interface {
	// Process must be called whenever a new tuple arrives in
	// the input stream. It will return a list of data.Map
	// items where each of these items is to be emitted as
	// a tuple. It is the caller's task to create those tuples
	// and set appropriate meta information such as timestamps.
	//
	// Process must NOT modify any field of the input tuple when its
	// core.TFShared flag is set. To modify the tuple, create a shallow copy of
	// it. Moreover, when Tuple.Data field is cached in the plan,
	// core.TFSharedData flag of the input tuple must be set and the plan must
	// not modify the Data.
	//
	// NB. Process is not thread-safe, i.e., it must be called in
	// a single-threaded context.
	Process(input *core.Tuple) ([]data.Map, error)
}

PhysicalPlan is a physical interface that is capable of computing the data that needs to be emitted into an output stream when a new tuple arrives in the input stream.

func NewDefaultSelectExecutionPlan

func NewDefaultSelectExecutionPlan(lp *LogicalPlan, reg udf.FunctionRegistry) (PhysicalPlan, error)

NewDefaultSelectExecutionPlan creates a plan that follows the theoretical processing model. It does not support aggregration.

After each tuple arrives,

  • compute the contents of the current window using the specified window size/type,
  • perform a SELECT query on that data,
  • compute the data that need to be emitted by comparison with the previous run's results.

func NewFilterPlan

func NewFilterPlan(lp *LogicalPlan, reg udf.FunctionRegistry) (PhysicalPlan, error)

NewFilterPlan creates a fast and simple plan for the case where the BQL statement has an Rstream emitter, a [RANGE 1 TUPLES] and (maybe) a WHERE clause (no GROUP BY/aggregate functions). In that case we can perform the check with less memory and faster than the default plan.

func NewGroupbyExecutionPlan

func NewGroupbyExecutionPlan(lp *LogicalPlan, reg udf.FunctionRegistry) (PhysicalPlan, error)

NewGroupbyExecutionPlan builds a plan that follows the theoretical processing model. It supports only statements that use aggregation.

After each tuple arrives,

  • compute the contents of the current window using the specified window size/type,
  • perform a SELECT query on that data,
  • compute the data that need to be emitted by comparison with the previous run's results.

type VolatilityType

type VolatilityType int

VolatilityType describes the volatility of an expression as per the PostgreSQL classification.

func (VolatilityType) String

func (v VolatilityType) String() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL