optimizer

Documentation

Index

• func FieldsOf(e dag.Expr) (field.List, bool)
• func InferDemandSeqOut(seq dag.Seq) map[dag.Op]demand.Demand
• func IsCountByString(o dag.Op) (string, bool)
• func IsSum(o dag.Op) (string, bool)
• type Optimizer
  • func New(ctx context.Context, source *data.Source) *Optimizer
  • func (o *Optimizer) Optimize(seq dag.Seq) (dag.Seq, error)
  • func (o *Optimizer) OptimizeDeleter(seq dag.Seq, replicas int) (dag.Seq, error)
  • func (o *Optimizer) Parallelize(seq dag.Seq, n int) (dag.Seq, error)
  • func (o *Optimizer) SortKeys(seq dag.Seq) ([]order.SortKey, error)
  • func (o *Optimizer) Vectorize(seq dag.Seq) (dag.Seq, error)

Functions

func FieldsOf

func FieldsOf(e dag.Expr) (field.List, bool)

func InferDemandSeqOut

func InferDemandSeqOut(seq dag.Seq) map[dag.Op]demand.Demand

Returns a map from op to the demand on the output of that op.

func IsCountByString

func IsCountByString(o dag.Op) (string, bool)

IsCountByString returns whether o represents "count() by <top-level field>" along with the field name.

func IsSum

func IsSum(o dag.Op) (string, bool)

IsSum return whether o represents "sum(<top-level field>)" along with the field name.

Types

type Optimizer

type Optimizer struct {
	// contains filtered or unexported fields
}

func New

func New(ctx context.Context, source *data.Source) *Optimizer

func (*Optimizer) Optimize

func (o *Optimizer) Optimize(seq dag.Seq) (dag.Seq, error)

Optimize transforms the DAG by attempting to lift stateless operators from the downstream sequence into the trunk of each data source in the From operator at the entry point of the DAG. Once these paths are lifted, it also attempts to move any candidate filtering operations into the source's pushdown predicate. This should be called before ParallelizeScan(). TBD: we need to do pushdown for search/cut to optimize columnar extraction.

func (*Optimizer) OptimizeDeleter

func (o *Optimizer) OptimizeDeleter(seq dag.Seq, replicas int) (dag.Seq, error)

func (*Optimizer) Parallelize

func (o *Optimizer) Parallelize(seq dag.Seq, n int) (dag.Seq, error)

Parallelize tries to parallelize the DAG by splitting each source path as much as possible of the sequence into n parallel branches.

func (*Optimizer) SortKeys

func (o *Optimizer) SortKeys(seq dag.Seq) ([]order.SortKey, error)

func (*Optimizer) Vectorize

func (o *Optimizer) Vectorize(seq dag.Seq) (dag.Seq, error)