ast

package
v0.32.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2021 License: BSD-3-Clause Imports: 6 Imported by: 1

Documentation

Overview

Package ast declares the types used to represent syntax trees for Zed queries.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func UnpackJSON 

func UnpackJSON(buf []byte) (interface{}, error)

Types

type Agg 

type Agg struct {
	Kind  string `json:"kind" unpack:""`
	Name  string `json:"name"`
	Expr  Expr   `json:"expr"`
	Where Expr   `json:"where"`
}

An Agg is an AST node that represents a aggregate function. The Name field indicates the aggregation method while the Expr field indicates an expression applied to the incoming records that is operated upon by them aggregate function. If Expr isn't present, then the aggregator doesn't act upon a function of the record, e.g., count() counts up records without looking into them.

func (*Agg) ExprAST 

func (*Agg) ExprAST()

type ArrayExpr 

type ArrayExpr struct {
	Kind  string `json:"kind" unpack:""`
	Exprs []Expr `json:"exprs"`
}

func (*ArrayExpr) ExprAST 

func (*ArrayExpr) ExprAST()

type Assignment 

type Assignment struct {
	Kind string `json:"kind" unpack:""`
	LHS  Expr   `json:"lhs"`
	RHS  Expr   `json:"rhs"`
}

func NewAggAssignment 

func NewAggAssignment(kind string, lval field.Path, arg field.Path) Assignment

func (*Assignment) ExprAST 

func (*Assignment) ExprAST()

type BinaryExpr 

type BinaryExpr struct {
	Kind string `json:"kind" unpack:""`
	Op   string `json:"op"`
	LHS  Expr   `json:"lhs"`
	RHS  Expr   `json:"rhs"`
}

A BinaryExpr is any expression of the form "lhs kind rhs" including arithmetic (+, -, *, /), logical operators (and, or), comparisons (=, !=, <, <=, >, >=), index operatons (on arrays, sets, and records) with kind "[" and a dot expression (".") (on records).

func (*BinaryExpr) ExprAST 

func (*BinaryExpr) ExprAST()

type Call 

type Call struct {
	Kind string `json:"kind" unpack:""`
	Name string `json:"name"`
	Args []Expr `json:"args"`
}

A Call represents different things dependending on its context. As a proc, it is either a group-by with no group-by keys and no duration, or a filter with a function that is boolean valued. This is determined by the compiler rather than the syntax tree based on the specific functions and aggregators that are defined at compile time. In expression context, a function call has the standard semantics where it takes one or more arguments and returns a result.

func (*Call) ExprAST 

func (*Call) ExprAST()

func (*Call) ProcAST 

func (*Call) ProcAST()

type Case 

type Case struct {
	Expr Expr `json:"expr"`
	Proc Proc `json:"proc"`
}

type Cast 

type Cast struct {
	Kind string      `json:"kind" unpack:""`
	Expr Expr        `json:"expr"`
	Type astzed.Type `json:"type"`
}

func (*Cast) ExprAST 

func (*Cast) ExprAST()

type Conditional 

type Conditional struct {
	Kind string `json:"kind" unpack:""`
	Cond Expr   `json:"cond"`
	Then Expr   `json:"then"`
	Else Expr   `json:"else"`
}

func (*Conditional) ExprAST 

func (*Conditional) ExprAST()

type Const 

type Const struct {
	Kind string `json:"kind" unpack:""`
	Name string `json:"name"`
	Expr Expr   `json:"expr"`
}

XXX This is a quick and dirty way to get constants into Zed. They are smuggled in as fake procs. When we refactor this AST into a parser AST proper and a separate kernel DSL, we will clean this up.

func (*Const) ProcAST 

func (*Const) ProcAST()

type Cut 

type Cut struct {
	Kind string       `json:"kind" unpack:""`
	Args []Assignment `json:"args"`
}

A Cut proc represents a proc that removes fields from each input record where each removed field matches one of the named fields sending each such modified record to its output in the order received.

func (*Cut) ProcAST 

func (*Cut) ProcAST()

type Drop 

type Drop struct {
	Kind string `json:"kind" unpack:""`
	Args []Expr `json:"args"`
}

A Drop proc represents a proc that removes fields from each input record.

func (*Drop) ProcAST 

func (*Drop) ProcAST()

type EntryExpr 

type EntryExpr struct {
	Key   Expr `json:"key"`
	Value Expr `json:"value"`
}

type Explode 

type Explode struct {
	Kind string      `json:"kind" unpack:""`
	Args []Expr      `json:"args"`
	Type astzed.Type `json:"type"`
	As   Expr        `json:"as"`
}

func (*Explode) ProcAST 

func (*Explode) ProcAST()

type Expr 

type Expr interface {
	ExprAST()
}

func NewDotExpr 

func NewDotExpr(f field.Path) Expr

func UnpackMapAsExpr 

func UnpackMapAsExpr(m interface{}) (Expr, error)

type FieldCutter 

type FieldCutter struct {
	Kind  string     `json:"kind" unpack:""`
	Field field.Path `json:"field"`
	Out   field.Path `json:"out"`
}

func (*FieldCutter) ProcAST 

func (*FieldCutter) ProcAST()

type FieldExpr 

type FieldExpr struct {
	Name  string `json:"name"`
	Value Expr   `json:"value"`
}

type File 

type File struct {
	Kind   string  `json:"kind" unpack:""`
	Path   string  `json:"path"`
	Format string  `json:"format"`
	Layout *Layout `json:"layout"`
}

func (*File) Source 

func (*File) Source()

type Filter 

type Filter struct {
	Kind string `json:"kind" unpack:""`
	Expr Expr   `json:"expr"`
}

A Filter proc represents a proc that discards all records that do not match the indicfated filter and forwards all that match to its output.

func FilterToProc 

func FilterToProc(e Expr) *Filter

func (*Filter) ProcAST 

func (*Filter) ProcAST()

type From 

type From struct {
	Kind   string  `json:"kind" unpack:""`
	Trunks []Trunk `json:"trunks"`
}

A proc is a node in the flowgraph that takes records in, processes them, and produces records as output.

func (*From) ProcAST 

func (*From) ProcAST()

type Fuse 

type Fuse struct {
	Kind string `json:"kind" unpack:""`
}

A Fuse proc represents a proc that turns a zng stream into a dataframe.

func (*Fuse) ProcAST 

func (*Fuse) ProcAST()

type HTTP 

type HTTP struct {
	Kind   string  `json:"kind" unpack:""`
	URL    string  `json:"url"`
	Format string  `json:"format"`
	Layout *Layout `json:"layout"`
}

func (*HTTP) Source 

func (*HTTP) Source()
type Head struct {
	Kind  string `json:"kind" unpack:""`
	Count int    `json:"count"`
}

A Head proc represents a proc that forwards the indicated number of records then terminates.

func (*Head) ProcAST 

func (*Head) ProcAST()

type ID 

type ID struct {
	Kind string `json:"kind" unpack:""`
	Name string `json:"name"`
}

func (*ID) ExprAST 

func (*ID) ExprAST()

type Join 

type Join struct {
	Kind     string       `json:"kind" unpack:""`
	Style    string       `json:"style"`
	LeftKey  Expr         `json:"left_key"`
	RightKey Expr         `json:"right_key"`
	Args     []Assignment `json:"args"`
}

A Join proc represents a proc that joins two zng streams.

func (*Join) ProcAST 

func (*Join) ProcAST()

type Layout 

type Layout struct {
	Kind  string `json:"kind" unpack:""`
	Keys  []Expr `json:"keys"`
	Order string `json:"order"`
}

func NewLayout 

func NewLayout(layout order.Layout) *Layout

type MapExpr 

type MapExpr struct {
	Kind    string      `json:"kind" unpack:""`
	Entries []EntryExpr `json:"entries"`
}

func (*MapExpr) ExprAST 

func (*MapExpr) ExprAST()

type Method 

type Method struct {
	Name string `json:"name"`
	Args []Expr `json:"args"`
}

type OpAssignment 

type OpAssignment struct {
	Kind        string       `json:"kind" unpack:""`
	Assignments []Assignment `json:"assignments"`
}

An OpAssignment proc is a list of assignments whose parent proc is unknown: It could be a Summarize or Put proc. This will be determined in the semantic phase.

func (*OpAssignment) ProcAST 

func (*OpAssignment) ProcAST()

type Parallel 

type Parallel struct {
	Kind string `json:"kind" unpack:""`
	// If non-zero, MergeBy contains the field name on
	// which the branches of this parallel proc should be
	// merged in the order indicated by MergeReverse.
	// XXX merge_by should be a list of expressions
	MergeBy      field.Path `json:"merge_by,omitempty"`
	MergeReverse bool       `json:"merge_reverse,omitempty"`
	Procs        []Proc     `json:"procs"`
}

A Parallel proc represents a set of procs that each get a stream of records from their parent.

func (*Parallel) ProcAST 

func (*Parallel) ProcAST()

type Pass 

type Pass struct {
	Kind string `json:"kind" unpack:""`
}

A Pass proc represents a passthrough proc that mirrors incoming Pull()s on its parent and returns the result.

func (*Pass) ProcAST 

func (*Pass) ProcAST()

func (*Pass) Source 

func (*Pass) Source()

type Pick 

type Pick struct {
	Kind string       `json:"kind" unpack:""`
	Args []Assignment `json:"args"`
}

A Pick proc is like a Cut but skips records that do not match all of the field expressions.

func (*Pick) ProcAST 

func (*Pick) ProcAST()

type Pool 

type Pool struct {
	Kind      string   `json:"kind" unpack:""`
	Spec      PoolSpec `json:"spec"`
	At        string   `json:"at"`
	Range     *Range   `json:"range"`
	ScanOrder string   `json:"scan_order"` // asc, desc, or unknown
}

func (*Pool) Source 

func (*Pool) Source()

type PoolSpec 

type PoolSpec struct {
	Pool   string `json:"pool"`
	Commit string `json:"commit"`
	Meta   string `json:"meta"`
}

type Proc 

type Proc interface {
	ProcAST()
}

Proc is the interface implemented by all AST processor nodes.

func Copy 

func Copy(in Proc) Proc

func UnpackJSONAsProc 

func UnpackJSONAsProc(buf []byte) (Proc, error)

UnpackJSONAsProc transforms a JSON representation of a proc into an ast.Proc.

func UnpackMapAsProc 

func UnpackMapAsProc(m interface{}) (Proc, error)

type Put 

type Put struct {
	Kind string       `json:"kind" unpack:""`
	Args []Assignment `json:"args"`
}

A proc is a node in the flowgraph that takes records in, processes them, and produces records as output.

func (*Put) ProcAST 

func (*Put) ProcAST()

type Range 

type Range struct {
	Kind  string `json:"kind" unpack:""`
	Lower Expr   `json:"lower"`
	Upper Expr   `json:"upper"`
}

type RecordExpr 

type RecordExpr struct {
	Kind   string      `json:"kind" unpack:""`
	Fields []FieldExpr `json:"fields"`
}

func (*RecordExpr) ExprAST 

func (*RecordExpr) ExprAST()

type RegexpMatch 

type RegexpMatch struct {
	Kind    string `json:"kind" unpack:""`
	Pattern string `json:"pattern"`
	Expr    Expr   `json:"expr"`
}

func (*RegexpMatch) ExprAST 

func (*RegexpMatch) ExprAST()

type RegexpSearch 

type RegexpSearch struct {
	Kind    string `json:"kind" unpack:""`
	Pattern string `json:"pattern"`
}

func (*RegexpSearch) ExprAST 

func (*RegexpSearch) ExprAST()

type Rename 

type Rename struct {
	Kind string       `json:"kind" unpack:""`
	Args []Assignment `json:"args"`
}

A Rename proc represents a proc that renames fields.

func (*Rename) ProcAST 

func (*Rename) ProcAST()

type Root 

type Root struct {
	Kind string `json:"kind" unpack:""`
}

Root refers to the outer record being operated upon. Field accesses typically begin with the LHS of a "." expression set to a Root.

func (*Root) ExprAST 

func (*Root) ExprAST()

type SQLExpr 

type SQLExpr struct {
	Kind    string       `json:"kind" unpack:""`
	Select  []Assignment `json:"select"`
	From    *SQLFrom     `json:"from"`
	Joins   []SQLJoin    `json:"joins"`
	Where   Expr         `json:"where"`
	GroupBy []Expr       `json:"group_by"`
	Having  Expr         `json:"having"`
	OrderBy *SQLOrderBy  `json:"order_by"`
	Limit   int          `json:"limit"`
}

A SQLExpr can be a proc, an expression inside of a SQL FROM clause, or an expression used as a Zed value generator. Currenly, the "select" keyword collides with the select() generator function (it can be parsed unambiguosly because of the parens but this is not user friendly so we need a new name for select()... see issue #2133). TBD: from alias, "in" over tuples, WITH sub-queries, multi-table FROM implying a JOIN, aliases for tables in FROM and JOIN.

func (*SQLExpr) ExprAST 

func (*SQLExpr) ExprAST()

func (*SQLExpr) ProcAST 

func (*SQLExpr) ProcAST()

type SQLFrom 

type SQLFrom struct {
	Table Expr `json:"table"`
	Alias Expr `json:"alias"`
}

type SQLJoin 

type SQLJoin struct {
	Table    Expr   `json:"table"`
	Style    string `json:"style"`
	LeftKey  Expr   `json:"left_key"`
	RightKey Expr   `json:"right_key"`
	Alias    Expr   `json:"alias"`
}

type SQLOrderBy 

type SQLOrderBy struct {
	Kind  string      `json:"kind" unpack:""`
	Keys  []Expr      `json:"keys"`
	Order order.Which `json:"order"`
}
type Search struct {
	Kind  string           `json:"kind" unpack:""`
	Text  string           `json:"text"`
	Value astzed.Primitive `json:"value"` //XXX search should be extended to complex types
}

func (*Search) ExprAST 

func (*Search) ExprAST()

type SelectExpr 

type SelectExpr struct {
	Kind      string `json:"kind" unpack:""`
	Selectors []Expr `json:"selectors"`
	Methods   []Call `json:"methods"`
}

func (*SelectExpr) ExprAST 

func (*SelectExpr) ExprAST()

type SeqExpr 

type SeqExpr struct {
	Kind      string   `json:"kind" unpack:""`
	Name      string   `json:"name"`
	Selectors []Expr   `json:"selectors"`
	Methods   []Method `json:"methods"`
}

func (*SeqExpr) ExprAST 

func (*SeqExpr) ExprAST()

type Sequential 

type Sequential struct {
	Kind  string `json:"kind" unpack:""`
	Procs []Proc `json:"procs"`
}

A Sequential proc represents a set of procs that receive a stream of records from their parent into the first proc and each subsequent proc processes the output records from the previous proc.

func (*Sequential) Prepend 

func (seq *Sequential) Prepend(front Proc)

func (*Sequential) ProcAST 

func (*Sequential) ProcAST()

type SetExpr 

type SetExpr struct {
	Kind  string `json:"kind" unpack:""`
	Exprs []Expr `json:"exprs"`
}

func (*SetExpr) ExprAST 

func (*SetExpr) ExprAST()

type Shape 

type Shape struct {
	Kind string `json:"kind" unpack:""`
}

A proc is a node in the flowgraph that takes records in, processes them, and produces records as output.

func (*Shape) ProcAST 

func (*Shape) ProcAST()

type Sort 

type Sort struct {
	Kind       string      `json:"kind" unpack:""`
	Args       []Expr      `json:"args"`
	Order      order.Which `json:"order"`
	NullsFirst bool        `json:"nullsfirst"`
}

A Sort proc represents a proc that sorts records.

func (*Sort) ProcAST 

func (*Sort) ProcAST()

type Source 

type Source interface {
	Source()
}

type Summarize 

type Summarize struct {
	Kind     string            `json:"kind" unpack:""`
	Duration *astzed.Primitive `json:"duration"`
	Limit    int               `json:"limit"`
	Keys     []Assignment      `json:"keys"`
	Aggs     []Assignment      `json:"aggs"`
}

A Summarize proc represents a proc that consumes all the records in its input, partitions the records into groups based on the values of the fields specified in the keys field (where the first key is the primary grouping key), and applies aggregators (if any) to each group. If the Duration field is non-zero, then the groups are further partioned by time into bins of the duration. In this case, the primary grouping key is ts. The InputSortDir field indicates that input is sorted (with direction indicated by the sign of the field) in the primary grouping key. In this case, the proc outputs the aggregation results from each key as they complete so that large inputs are processed and streamed efficiently. The Limit field specifies the number of different groups that can be aggregated over. When absent, the runtime defaults to an appropriate value. If PartialsOut is true, the proc will produce partial aggregation output result; likewise, if PartialsIn is true, the proc will expect partial results as input.

func (*Summarize) ProcAST 

func (*Summarize) ProcAST()

type Switch 

type Switch struct {
	Kind  string `json:"kind" unpack:""`
	Expr  Expr   `json:"expr"`
	Cases []Case `json:"cases"`
}

A Switch proc represents a set of procs that each get a stream of records from their parent.

func (*Switch) ProcAST 

func (*Switch) ProcAST()

type Tail 

type Tail struct {
	Kind  string `json:"kind" unpack:""`
	Count int    `json:"count"`
}

A Tail proc represents a proc that reads all its records from its input transmits the final number of records indicated by the count.

func (*Tail) ProcAST 

func (*Tail) ProcAST()

type Top 

type Top struct {
	Kind  string `json:"kind" unpack:""`
	Limit int    `json:"limit"`
	Args  []Expr `json:"args"`
	Flush bool   `json:"flush"`
}

A Top proc is similar to a Sort with a few key differences: - It only sorts in descending order. - It utilizes a MaxHeap, immediately discarding records that are not in the top N of the sort. - It has an option (Flush) to sort and emit on every batch.

func (*Top) ProcAST 

func (*Top) ProcAST()

type Trunk 

type Trunk struct {
	Kind   string      `json:"kind" unpack:""`
	Source Source      `json:"source"`
	Seq    *Sequential `json:"seq"`
}

type TypeProc 

type TypeProc struct {
	Kind string      `json:"kind" unpack:""`
	Name string      `json:"name"`
	Type astzed.Type `json:"type"`
}

A proc is a node in the flowgraph that takes records in, processes them, and produces records as output.

func (*TypeProc) ProcAST 

func (*TypeProc) ProcAST()

type TypeSplitter 

type TypeSplitter struct {
	Kind     string     `json:"kind" unpack:""`
	Key      field.Path `json:"key"`
	TypeName string     `json:"type_name"`
}

func (*TypeSplitter) ProcAST 

func (t *TypeSplitter) ProcAST()

type UnaryExpr 

type UnaryExpr struct {
	Kind    string `json:"kind" unpack:""`
	Op      string `json:"op"`
	Operand Expr   `json:"operand"`
}

func (*UnaryExpr) ExprAST 

func (*UnaryExpr) ExprAST()

type Uniq 

type Uniq struct {
	Kind  string `json:"kind" unpack:""`
	Cflag bool   `json:"cflag"`
}

A Uniq proc represents a proc that discards any record that matches the previous record transmitted. The Cflag causes the output records to contain a new field called count that contains the number of matched records in that set, similar to the unix shell command uniq.

func (*Uniq) ProcAST 

func (*Uniq) ProcAST()

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.