metricsview

package
v0.50.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 21, 2024 License: Apache-2.0 Imports: 20 Imported by: 0

README

runtime/metricsview/

This package contains the core logic for Rill's metrics layer (semantic layer). Namely, that includes validating metrics views and generating SQL queries for querying dimensions and measures defined in a metrics view.

Some other code files relevant to metrics views that are not found in this package:

  • proto/rill/runtime/v1/resources.proto: defines the spec for a metrics view
  • runtime/compilers/rillv1/parse_metrics_view.go: contains the logic for parsing a metrics view spec from a YAML file

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ExpressionToString 

func ExpressionToString(e *Expression) (string, error)

Types

type AST 

type AST struct {
	// Root of the AST
	Root *SelectNode
	// List of CTEs to add to the query
	CTEs []*SelectNode
	// contains filtered or unexported fields
}

AST is the abstract syntax tree for a metrics SQL query.

func NewAST 

func NewAST(mv *runtimev1.MetricsViewSpec, sec *runtime.ResolvedSecurity, qry *Query, dialect drivers.Dialect) (*AST, error)

NewAST builds a new SQL AST based on a metrics query.

Dynamic time ranges in the qry must be resolved to static start/end timestamps before calling this function. This is due to NewAST not being able (or intended) to resolve external time anchors such as watermarks.

The qry's PivotOn must be empty. Pivot queries must be rewritten/handled upstream of NewAST.

func (*AST) SQL 

func (a *AST) SQL() (string, []any, error)

SQL builds a SQL query from the AST. It returns the query and query arguments to be passed to the database driver.

type Condition 

type Condition struct {
	Operator    Operator      `mapstructure:"op"`
	Expressions []*Expression `mapstructure:"exprs"`
}

type Dimension 

type Dimension struct {
	Name    string            `mapstructure:"name"`
	Compute *DimensionCompute `mapstructure:"compute"`
}

type DimensionCompute 

type DimensionCompute struct {
	TimeFloor *DimensionComputeTimeFloor `mapstructure:"time_floor"`
}

type DimensionComputeTimeFloor 

type DimensionComputeTimeFloor struct {
	Dimension string    `mapstructure:"dimension"`
	Grain     TimeGrain `mapstructure:"grain"`
}

type Executor 

type Executor struct {
	// contains filtered or unexported fields
}

Executor is capable of executing queries and other operations against a metrics view.

func NewExecutor 

func NewExecutor(ctx context.Context, rt *runtime.Runtime, instanceID string, mv *runtimev1.MetricsViewSpec, sec *runtime.ResolvedSecurity, priority int) (*Executor, error)

NewExecutor creates a new Executor for the provided metrics view.

func (*Executor) Cacheable added in v0.48.0 

func (e *Executor) Cacheable(qry *Query) bool

Cacheable returns whether the result of running the given query is cacheable.

func (*Executor) Close 

func (e *Executor) Close()

Close releases the resources held by the Executor.

func (*Executor) Export 

func (e *Executor) Export(ctx context.Context, qry *Query, executionTime *time.Time, format drivers.FileFormat) (string, error)

Export executes and exports the provided query against the metrics view. It returns a path to a temporary file containing the export. The caller is responsible for cleaning up the file.

func (*Executor) Query 

func (e *Executor) Query(ctx context.Context, qry *Query, executionTime *time.Time) (*drivers.Result, error)

Query executes the provided query against the metrics view.

func (*Executor) Schema 

func (e *Executor) Schema(ctx context.Context) (*runtimev1.StructType, error)

Schema returns a schema for the metrics view's dimensions and measures.

func (*Executor) Search added in v0.49.0 

func (e *Executor) Search(ctx context.Context, qry *SearchQuery, executionTime *time.Time) ([]SearchResult, error)

SearchQuery executes the provided query against the metrics view.

func (*Executor) ValidateMetricsView 

func (e *Executor) ValidateMetricsView(ctx context.Context) error

ValidateMetricsView validates the dimensions and measures in the executor's metrics view.

func (*Executor) ValidateQuery 

func (e *Executor) ValidateQuery(qry *Query) error

ValidateQuery validates the provided query against the executor's metrics view.

func (*Executor) Watermark 

func (e *Executor) Watermark(ctx context.Context) (time.Time, error)

Watermark returns the current watermark of the metrics view. If the watermark resolves to null, it defaults to the current time.

type ExprNode 

type ExprNode struct {
	Expr string
	Args []any
}

ExprNode represents an expression for a WHERE clause.

type Expression 

type Expression struct {
	Name      string     `mapstructure:"name"`
	Value     any        `mapstructure:"val"`
	Condition *Condition `mapstructure:"cond"`
	Subquery  *Subquery  `mapstructure:"subquery"`
}

func NewExpressionFromProto 

func NewExpressionFromProto(expr *runtimev1.Expression) *Expression

type FieldNode 

type FieldNode struct {
	Name       string
	Label      string
	Expr       string
	AutoUnnest bool
}

FieldNode represents a column in a SELECT clause. It also carries metadata related to the dimension/measure it was derived from. The Name must always match a the name of a dimension/measure in the metrics view or a computed field specified in the request. This means that if two columns in different places in the AST have the same Name, they're guaranteed to resolve to the same value.

type JoinType added in v0.47.1 

type JoinType string

JoinType represents types of SQL joins. 

const (
	JoinTypeUnspecified JoinType = ""
	JoinTypeFull        JoinType = "FULL OUTER"
	JoinTypeLeft        JoinType = "LEFT OUTER"
	JoinTypeRight       JoinType = "RIGHT OUTER"
)

type Measure 

type Measure struct {
	Name    string          `mapstructure:"name"`
	Compute *MeasureCompute `mapstructure:"compute"`
}

type MeasureCompute 

type MeasureCompute struct {
	Count           bool                           `mapstructure:"count"`
	CountDistinct   *MeasureComputeCountDistinct   `mapstructure:"count_distinct"`
	ComparisonValue *MeasureComputeComparisonValue `mapstructure:"comparison_value"`
	ComparisonDelta *MeasureComputeComparisonDelta `mapstructure:"comparison_delta"`
	ComparisonRatio *MeasureComputeComparisonRatio `mapstructure:"comparison_ratio"`
	PercentOfTotal  *MeasureComputePercentOfTotal  `mapstructure:"percent_of_total"`
	URI             *MeasureComputeURI             `mapstructure:"uri"`
}

func (*MeasureCompute) Validate 

func (m *MeasureCompute) Validate() error

type MeasureComputeComparisonDelta 

type MeasureComputeComparisonDelta struct {
	Measure string `mapstructure:"measure"`
}

type MeasureComputeComparisonRatio 

type MeasureComputeComparisonRatio struct {
	Measure string `mapstructure:"measure"`
}

type MeasureComputeComparisonValue 

type MeasureComputeComparisonValue struct {
	Measure string `mapstructure:"measure"`
}

type MeasureComputeCountDistinct 

type MeasureComputeCountDistinct struct {
	Dimension string `mapstructure:"dimension"`
}

type MeasureComputePercentOfTotal added in v0.49.0 

type MeasureComputePercentOfTotal struct {
	Measure string   `mapstructure:"measure"`
	Total   *float64 `mapstructure:"total"`
}

type MeasureComputeURI added in v0.49.0 

type MeasureComputeURI struct {
	Dimension string `mapstructure:"dimension"`
}

type Operator 

type Operator string
const (
	OperatorUnspecified Operator = ""
	OperatorEq          Operator = "eq"
	OperatorNeq         Operator = "neq"
	OperatorLt          Operator = "lt"
	OperatorLte         Operator = "lte"
	OperatorGt          Operator = "gt"
	OperatorGte         Operator = "gte"
	OperatorIn          Operator = "in"
	OperatorNin         Operator = "nin"
	OperatorIlike       Operator = "ilike"
	OperatorNilike      Operator = "nilike"
	OperatorOr          Operator = "or"
	OperatorAnd         Operator = "and"
)

func (Operator) Valid 

func (o Operator) Valid() bool

type OrderFieldNode 

type OrderFieldNode struct {
	Name string
	Desc bool
}

OrderFieldNode represents a field in an ORDER BY clause.

type Query 

type Query struct {
	MetricsView         string      `mapstructure:"metrics_view"`
	Dimensions          []Dimension `mapstructure:"dimensions"`
	Measures            []Measure   `mapstructure:"measures"`
	PivotOn             []string    `mapstructure:"pivot_on"`
	Spine               *Spine      `mapstructure:"spine"`
	Sort                []Sort      `mapstructure:"sort"`
	TimeRange           *TimeRange  `mapstructure:"time_range"`
	ComparisonTimeRange *TimeRange  `mapstructure:"comparison_time_range"`
	Where               *Expression `mapstructure:"where"`
	Having              *Expression `mapstructure:"having"`
	Limit               *int64      `mapstructure:"limit"`
	Offset              *int64      `mapstructure:"offset"`
	TimeZone            string      `mapstructure:"time_zone"`
	Label               bool        `mapstructure:"label"`
}

type SearchQuery added in v0.49.0 

type SearchQuery struct {
	MetricsView string      `mapstructure:"metrics_view"`
	Dimensions  []string    `mapstructure:"dimensions"`
	Search      string      `mapstructure:"search"`
	Where       *Expression `mapstructure:"where"`
	Having      *Expression `mapstructure:"having"`
	TimeRange   *TimeRange  `mapstructure:"time_range"`
	Limit       *int64      `mapstructure:"limit"`
}

type SearchResult added in v0.49.0 

type SearchResult struct {
	Dimension string
	Value     any
}

type SelectNode 

type SelectNode struct {
	Alias                string           // Alias for the node used by outer SELECTs to reference it.
	IsCTE                bool             // Whether this node is a Common Table Expression
	DimFields            []FieldNode      // Dimensions fields to select
	MeasureFields        []FieldNode      // Measure fields to select
	FromTable            *string          // Underlying table expression to select from (if set, FromSelect must not be set)
	FromSelect           *SelectNode      // Sub-select to select from (if set, FromTable must not be set)
	SpineSelect          *SelectNode      // Sub-select that returns a spine of dimensions. Currently it will be right-joined onto FromSelect.
	LeftJoinSelects      []*SelectNode    // Sub-selects to left join onto FromSelect, to enable "per-dimension" measures
	JoinComparisonSelect *SelectNode      // Sub-select to join onto FromSelect for comparison measures
	JoinComparisonType   JoinType         // Type of join to use for JoinComparisonSelect
	Unnests              []string         // Unnest expressions to add in the FROM clause
	Group                bool             // Whether the SELECT is grouped. If yes, it will group by all DimFields.
	Where                *ExprNode        // Expression for the WHERE clause
	TimeWhere            *ExprNode        // Expression for the time range to add to the WHERE clause
	Having               *ExprNode        // Expression for the HAVING clause. If HAVING is not allowed in the current context, it will added as a WHERE in a wrapping SELECT.
	OrderBy              []OrderFieldNode // Fields to order by
	Limit                *int64           // Limit for the query
	Offset               *int64           // Offset for the query
}

SelectNode represents a query that computes measures by dimensions. The from/join clauses are not all compatible. The allowed combinations are:

  • FromTable
  • FromSelect and optionally SpineSelect and/or LeftJoinSelects
  • FromSelect and optionally JoinComparisonSelect (for comparison CTE based optimization, this combination is used, both should be set and one of them will be used as CTE)

type Sort 

type Sort struct {
	Name string `mapstructure:"name"`
	Desc bool   `mapstructure:"desc"`
}

type Spine 

type Spine struct {
	Where     *WhereSpine `mapstructure:"where"`
	TimeRange *TimeSpine  `mapstructure:"time"`
}

type Subquery 

type Subquery struct {
	Dimension Dimension   `mapstructure:"dimension"`
	Measures  []Measure   `mapstructure:"measures"`
	Where     *Expression `mapstructure:"where"`
	Having    *Expression `mapstructure:"having"`
}

type TimeGrain 

type TimeGrain string
const (
	TimeGrainUnspecified TimeGrain = ""
	TimeGrainMillisecond TimeGrain = "millisecond"
	TimeGrainSecond      TimeGrain = "second"
	TimeGrainMinute      TimeGrain = "minute"
	TimeGrainHour        TimeGrain = "hour"
	TimeGrainDay         TimeGrain = "day"
	TimeGrainWeek        TimeGrain = "week"
	TimeGrainMonth       TimeGrain = "month"
	TimeGrainQuarter     TimeGrain = "quarter"
	TimeGrainYear        TimeGrain = "year"
)

func TimeGrainFromProto 

func TimeGrainFromProto(t runtimev1.TimeGrain) TimeGrain

func (TimeGrain) ToProto 

func (t TimeGrain) ToProto() runtimev1.TimeGrain

func (TimeGrain) ToTimeutil 

func (t TimeGrain) ToTimeutil() timeutil.TimeGrain

func (TimeGrain) Valid 

func (t TimeGrain) Valid() bool

type TimeRange 

type TimeRange struct {
	Start        time.Time `mapstructure:"start"`
	End          time.Time `mapstructure:"end"`
	IsoDuration  string    `mapstructure:"iso_duration"`
	IsoOffset    string    `mapstructure:"iso_offset"`
	RoundToGrain TimeGrain `mapstructure:"round_to_grain"`
}

func (*TimeRange) IsZero 

func (tr *TimeRange) IsZero() bool

type TimeSpine 

type TimeSpine struct {
	Start time.Time `mapstructure:"start"`
	End   time.Time `mapstructure:"end"`
	Grain TimeGrain `mapstructure:"grain"`
}

type WhereSpine 

type WhereSpine struct {
	Expression *Expression `mapstructure:"expr"`
}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.