types

package
v0.0.0-...-307fc59 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 8, 2024 License: Apache-2.0 Imports: 11 Imported by: 37

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsNullObject 

func IsNullObject(a interface{}) bool

func RecoverVQL 

func RecoverVQL(scope Scope)

Utilities to do with scope.

func ToString 

func ToString(ctx context.Context,
	scope Scope, x interface{}) string

Try very hard to convert to a string

Types

type AggregatorCtx 

type AggregatorCtx interface {
	// Modify the context under lock. If there is no existing value,
	// old_value will be nil and pres will be false. You can use this
	// to read the old value as well by just returning it.
	Modify(name string, modifier func(old_value Any, pres bool) Any) Any
}

type Any 

type Any interface{}

A Generic object which may be returned in a row from a plugin.

type Explainer 

type Explainer interface {
	// Emitted when beginning to explain new query
	StartQuery(select_ast_node interface{})

	// Report each row extracted from a plugin
	PluginOutput(plugin_ast_node interface{}, row Row)

	// The final row emitted from SELECT (after filtering)
	SelectOutput(row Row)

	// Report when args are parsed into a plugin/function
	ParseArgs(args *ordereddict.Dict, result interface{}, err error)

	RejectRow(where_ast_node interface{})

	// A general purpose log
	Log(message string)
}

An explainer can be installed into the scope and will be used to provide tracing information about the query as it executes.

type FunctionCopier 

type FunctionCopier interface {
	Copy() FunctionInterface
}

type FunctionInfo 

type FunctionInfo struct {
	Name    string
	Doc     string
	ArgType string

	// This is true for functions which operate on aggregates
	// (i.e. group by). For any columns which contains such a
	// function, vfilter will first run the group by clause then
	// re-evaluate the function on the aggregate column.
	IsAggregate bool

	// A version of this plugin. VQL queries can target certain
	// versions of this function if needed.
	Version int

	// Arbitrary metadata attched to the function info
	Metadata *ordereddict.Dict
}

Describe functions.

type FunctionInterface 

type FunctionInterface interface {
	Call(ctx context.Context, scope Scope, args *ordereddict.Dict) Any
	Info(scope Scope, type_map *TypeMap) *FunctionInfo
}

type GroupbyActor 

type GroupbyActor interface {
	// Just receive new rows. Return EOF when no more rows exist. Returns
	// 1. The next trasformed row to group.
	// 2. The original untransformed row (that can from the plugin).
	// 3. The group by bin index
	// 4. The scope over which the query is materialized.
	// 5. An error (Usually EOF if the stream is exhausted).
	//
	// Note that rows returned here are Lazy and are not
	// materialized. The grouper will materialize the row after
	// installing the correct bin context in the scope.
	GetNextRow(ctx context.Context, scope Scope) (LazyRow, Row, string, Scope, error)

	// Transform a raw row with the column selectors
	Transform(ctx context.Context, scope Scope, row Row) (LazyRow, func())

	// Materialize the row on the scope provided in the previous
	// call. The scope should contains the correct bin context
	// over which aggregate functions will be evaluated.
	MaterializeRow(ctx context.Context, row Row, scope Scope) *ordereddict.Dict
}

The GroupbyActor is passed to the grouper by the caller. The Grouper will then use it to create the result set. It is a way of delegating just the functionality required by the grouper to the query without exposing the internals of the query engine to the grouper.

type Grouper 

type Grouper interface {
	Group(ctx context.Context, scope Scope, actor GroupbyActor) <-chan Row
}

A grouper receives rows and groups them into groups. Callers must provide a valid actor. Results are not sorted but the order is stable.

type LazyAny 

type LazyAny interface{}

A Special type which can be used as a plugin parameter. This is used to prevent any kind of reducing or wrapping of the arg and leaves the caller to handle all aspects. This is mostly used in if() where it is critical to not evaluate unused branches in any circumstance.

type LazyExpr 

type LazyExpr interface {
	// Reduce with the scope captured at point of definition.
	Reduce(ctx context.Context) Any

	// Reduce with a new scope.
	ReduceWithScope(ctx context.Context, scope Scope) Any
}

A LazyExpr has a reduce method that allows it to be materialized.

type LazyRow 

type LazyRow interface {
	// Add a lazy evaluator to the column.
	AddColumn(name string, getter func(ctx context.Context, scope Scope) Any) LazyRow

	// Check if a row has a column name without evaluating it
	Has(name string) bool

	// Materialize the value at a column
	Get(name string) (Any, bool)

	// Return all the columns
	Columns() []string
}

A Lazy row holds column values without evaluating them. We call the act of evaluating a column, we materialize the column into a proper type.

type MarshalItem 

type MarshalItem struct {
	Type    string          `json:"type"`
	Comment string          `json:"comment,omitempty"`
	Data    json.RawMessage `json:"data"`
}

type Marshaler 

type Marshaler interface {
	Marshal(scope Scope) (*MarshalItem, error)
}

A type that implements the marshaller interface is able to convert itself into a MarshalItem

type Materializer 

type Materializer interface {
	Materialize(ctx context.Context, scope Scope) Any
}

type Memberer 

type Memberer interface {
	Members() []string
}

When types implement a lazy interface we need to know all their columns. The Memberer interface allows the type to tell us all its members. This is a convenience to having to implement the GetMembers() protocol.

type Null 

type Null struct{}

A real type which encodes to JSON NULL. Using go's nil is dangerous because it forces constant checking for nil pointer dereference. It is safer to just return this value when VQL needs to return NULL.

func (Null) MarshalJSON 

func (self Null) MarshalJSON() ([]byte, error)

func (Null) String 

func (self Null) String() string

type PluginGeneratorInterface 

type PluginGeneratorInterface interface {
	Call(ctx context.Context, scope Scope, args *ordereddict.Dict) <-chan Row
	Info(scope Scope, type_map *TypeMap) *PluginInfo
}

type PluginInfo 

type PluginInfo struct {
	// The name of the plugin.
	Name string

	// A helpful description about the plugin.
	Doc string

	ArgType string

	// A version of this plugin. VQL queries can target certain
	// versions of this plugin if needed.
	Version int

	// Arbitrary metadata attched to the plugin info
	Metadata *ordereddict.Dict
}

Describes the specific plugin.

type Row 

type Row interface{}

Plugins may return anything as long as there is a valid Associative() protocol handler. VFilter will simply call scope.Associative(row, column) to retrieve the cell value for each column. Note that VFilter will use reflection to implement the DefaultAssociative{} protocol - this means that plugins may just return any struct with exported methods and fields and it will be supported automatically.

func Materialize 

func Materialize(ctx context.Context, scope Scope, stored_query StoredQuery) []Row

Materialize a stored query into a set of rows.

type Scope 

type Scope interface {

	// Duplicate the scope to a completely new scope - this is a
	// deep copy not a subscope!  Very rarely used.
	NewScope() Scope

	// Copy the scope and create a subscope child.
	Copy() Scope

	// The scope context is a global k/v store. It is inherited into
	// subscopes so should be used to store global data. It is not
	// accessible from VQL itself.
	GetContext(name string) (Any, bool)
	SetContext(name string, value Any)

	// DEPRECATED: This should not really be used as it trashes
	// everything in the context. It is only used when making an
	// entirely new scope.
	ClearContext()

	// The aggregator context is used by aggregator functions to store
	// context within a query block.
	GetAggregatorCtx() AggregatorCtx

	// Creates a new AggregatorCtx on this scope. This is used by VQL
	// when entering a new semantic scope where aggregators need to be
	// reset.
	SetAggregatorCtx(ctx AggregatorCtx)

	// Extract debug string about the current scope state.
	PrintVars() string

	// Scope manages the protocols
	Bool(a Any) bool
	Eq(a Any, b Any) bool
	Lt(a Any, b Any) bool
	Gt(a Any, b Any) bool
	Add(a Any, b Any) Any
	Sub(a Any, b Any) Any
	Mul(a Any, b Any) Any
	Div(a Any, b Any) Any
	Membership(a Any, b Any) bool
	Associative(a Any, b Any) (Any, bool)
	GetMembers(a Any) []string
	Materialize(ctx context.Context,
		name string, query StoredQuery) StoredQuery

	Match(a Any, b Any) bool
	Iterate(ctx context.Context, a Any) <-chan Row

	// The scope's top level variable. Scopes search backward
	// through their parents to resolve names from these vars.
	AppendVars(row Row) Scope
	Resolve(field string) (interface{}, bool)

	// Program a custom sorter
	SetSorter(sorter Sorter)
	SetGrouper(grouper Grouper)
	SetMaterializer(materializer ScopeMaterializer)
	SetExplainer(explainer Explainer)

	// Start explaining this scope and its children
	EnableExplain()
	Explainer() Explainer

	// We can program the scope's protocols
	AddProtocolImpl(implementations ...Any) Scope
	AppendFunctions(functions ...FunctionInterface) Scope
	AppendPlugins(plugins ...PluginGeneratorInterface) Scope

	// Logging and performance monitoring.
	SetLogger(logger *log.Logger)
	SetTracer(logger *log.Logger)
	GetLogger() *log.Logger
	GetStats() *Stats

	// Log levels
	Log(format string, a ...interface{})
	Error(format string, a ...interface{})
	Warn(format string, a ...interface{})
	Debug(format string, a ...interface{})
	Trace(format string, a ...interface{})

	// Introspection
	GetFunction(name string) (FunctionInterface, bool)
	GetPlugin(name string) (PluginGeneratorInterface, bool)
	GetSimilarPlugins(name string) []string
	Describe(type_map *TypeMap) *ScopeInformation
	CheckForOverflow() bool

	// Charge an op to the throttler.
	ChargeOp()
	SetThrottler(t Throttler)

	// Destructors are called when the scope is Close(). If the
	// scope is already closed adding the destructor may fail.
	AddDestructor(fn func()) error
	IsClosed() bool
	Close()
}

A scope is passed inside the evaluation context. Although this is an interface, there is currently only a single implementation (scope.Scope). The interface exposes the public methods.

type ScopeInformation 

type ScopeInformation struct {
	Plugins   []*PluginInfo
	Functions []*FunctionInfo
}

type ScopeMaterializer 

type ScopeMaterializer interface {
	Materialize(ctx context.Context, scope Scope,
		name string, query StoredQuery) StoredQuery
}

A ScopeMaterializer handles VQL Let Materialize operators (<=). The returned object will be added to the scope and can be accessed in subsequent queries. This allows users of vfilter the ability to customize the implementation of materialized queries.

type Sorter 

type Sorter interface {
	Sort(ctx context.Context,
		scope Scope,
		input <-chan Row,
		key string,
		desc bool) <-chan Row
}

A Sorter is a pluggable way for VQL to sort an incoming set of rows.

type Stats 

type Stats struct {
	// contains filtered or unexported fields
}

A lightweight struct for accumulating general stats.

func (*Stats) IncFunctionsCalled 

func (self *Stats) IncFunctionsCalled()

func (*Stats) IncPluginsCalled 

func (self *Stats) IncPluginsCalled()

func (*Stats) IncProtocolSearch 

func (self *Stats) IncProtocolSearch(i int)

func (*Stats) IncRowsScanned 

func (self *Stats) IncRowsScanned()

func (*Stats) IncScopeCopy 

func (self *Stats) IncScopeCopy()

func (*Stats) Snapshot 

func (self *Stats) Snapshot() *ordereddict.Dict

type StoredExpression 

type StoredExpression interface {
	Reduce(ctx context.Context, scope Scope) Any
}

type StoredQuery 

type StoredQuery interface {
	Eval(ctx context.Context, scope Scope) <-chan Row
}

A plugin like object which takes no arguments but may be inserted into the scope to select from it.

type StringProtocol 

type StringProtocol interface {
	ToString(scope Scope) string
}

type Throttler 

type Throttler interface {
	// This is called frequently through the code, it may sleep in
	// order to delay query execution.
	ChargeOp()
	Close()
}

type TypeDescriber 

type TypeDescriber interface {
	DescribeType() string
}

type TypeDescription 

type TypeDescription struct {
	Fields *ordereddict.Dict
}

Describe a type. This is meant for human consumption so it does not need to be so accurate. Fields is a map between the Associative member and the type that is supposed to be returned. Note that Velocifilter automatically calls accessor methods so they look like standard exported fields.

type TypeMap 

type TypeMap struct {
	// contains filtered or unexported fields
}

Map between type name and its description.

func NewTypeMap 

func NewTypeMap() *TypeMap

func (*TypeMap) AddType 

func (self *TypeMap) AddType(scope Scope, a Any) string

Introspect the type of the parameter. Add type descriptor to the type map and return the type name.

func (*TypeMap) Get 

func (self *TypeMap) Get(scope Scope, name string) (*TypeDescription, bool)

type TypeReference 

type TypeReference struct {
	Target   string
	Repeated bool
	Tag      string
}

This describes what type is returned when we reference this field from the TypeDescription.

type Unmarshaller 

type Unmarshaller interface {
	Unmarshal(unmarshaller Unmarshaller,
		scope Scope, item *MarshalItem) (interface{}, error)
}

An unmarshaller for a custom type. Note: You must register this unmarshaller with marshal/Unmarshaller like this:

unmarshaller := marshal.NewUnmarshaller() unmarshaller.Handlers["Scope"] = vfilter.ScopeUnmarshaller{ignoreVars} unmarshaller.Handlers["Replay"] = vfilter.ReplayUnmarshaller{}

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.