ottl

package module
v0.72.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 22, 2023 License: Apache-2.0 Imports: 12 Imported by: 25

README

OpenTelemetry Transformation Language

The OpenTelemetry Transformation Language is a language for transforming open telemetry data based on the OpenTelemetry Collector Processing Exploration.

This package reads in OTTL statements and converts them to invokable Booleans and functions based on the OTTL's grammar.

The OTTL is signal agnostic; it is not aware of the type of telemetry on which it will operate. Instead, the Booleans and functions returned by the package must be passed a TransformContext, which provide access to the signal's telemetry. Telemetry data can be accessed and updated through Getters and Setters.

Grammar

The OTTL grammar includes Invocations, Values and Boolean Expressions.

Invocations

Invocations represent a function call that transform the underlying telemetry payload. Invocations are made up of 2 parts:

  • a string identifier. The string identifier must start with a lowercase letter.
  • zero or more Values (comma separated) surrounded by parentheses (()).

The OTTL does not define any function implementations. Users must supply a map between string identifiers and the actual function implementation. The OTTL will use this map and reflection to generate Invocations, that can then be invoked by the user. See ottlfuncs for pre-made, usable functions.

Example Invocations

  • route()
  • set(field, 1)
Invocation parameters

The OTTL will use reflection to determine parameter types when parsing an invocation within a statement.

When developing functions that represent invocations, the following types are supported for single parameter values:

  • Setter
  • GetSetter
  • Getter
  • Enum
  • string
  • float64
  • int64
  • bool

For slice parameters, the following types are supported:

  • string
  • float64
  • int64
  • uint8. Byte slice literals are parsed as byte slices by the OTTL.
  • Getter
Values

Values are passed as input to an Invocation or are used in a Boolean Expression. Values can take the form of:

Paths

A Path Value is a reference to a telemetry field. Paths are made up of lowercase identifiers, dots (.), and square brackets combined with a string key (["key"]). The interpretation of a Path is NOT implemented by the OTTL. Instead, the user must provide a PathExpressionParser that the OTTL can use to interpret paths. As a result, how the Path parts are used is up to the user. However, it is recommended, that the parts be used like so:

  • Identifiers are used to map to a telemetry field.
  • Dots (.) are used to separate nested fields.
  • Square brackets and keys (["key"]) are used to access values within maps.

When accessing a map's value, if the given key does not exist, nil will be returned. This can be used to check for the presence of a key within a map within a Boolean Expression.

Example Paths

  • name
  • value_double
  • resource.name
  • resource.attributes["key"]
Lists

A List Value comprises a sequence of Values. Currently, list can only be created by the grammar to be used in functions or conditions; the grammar does not provide an accessor to individual list entries.

Example List Values:

  • []
  • [1]
  • ["1", "2", "3"]
  • ["a", attributes["key"], Concat(["a", "b"], "-")]
Literals

Literals are literal interpretations of the Value into a Go value. Accepted literals are:

  • Strings. Strings are represented as literals by surrounding the string in double quotes ("").
  • Ints. Ints are represented by any digit, optionally prepended by plus (+) or minus (-). Internally the OTTL represents all ints as int64
  • Floats. Floats are represented by digits separated by a dot (.), optionally prepended by plus (+) or minus (-). The leading digit is optional. Internally the OTTL represents all Floats as float64.
  • Bools. Bools are represented by the exact strings true and false.
  • Nil. Nil is represented by the exact string nil.
  • Byte slices. Byte slices are represented via a hex string prefaced with 0x

Example Literals

  • "a string"
  • 1, -1
  • 1.5, -.5
  • true, false
  • nil,
  • 0x0001
Enums

Enums are uppercase identifiers that get interpreted during parsing and converted to an int64. The interpretation of an Enum is NOT implemented by the OTTL. Instead, the user must provide a EnumParser that the OTTL can use to interpret the Enum. The EnumParser returns an int64 instead of a function, which means that the Enum's numeric value is retrieved during parsing instead of during execution.

Within the grammar Enums are always used as int64. As a result, the Enum's symbol can be used as if it is an Int value.

When defining a function that will be used as an Invocation by the OTTL, if the function needs to take an Enum then the function must use the Enum type for that argument, not an int64.

Converters

Converters are special functions that convert data to a new format before being passed to an Invocation or Boolean Expression. Like Invocations, Converters are made up of 2 parts:

  • a string identifier. The string identifier must start with an uppercase letter.
  • zero or more Values (comma separated) surrounded by parentheses (()).

The OTTL does not define any converter implementations. Users must include converters in the same map that invocations are supplied. The OTTL will use this map and reflection to generate Converters that can then be invoked by the user. See ottlfuncs for pre-made, usable Converters.

Example Converters

  • Int()
  • IsMatch(field, ".*")
Math Expressions

Math Expressions represent arithmetic calculations. They support +, -, *, and /, along with () for grouping.

Math Expressions currently only support int64 and float64. Math Expressions support Paths and Invocations that return supported types. Note that * and / take precedence over + and -. Operations that share the same level of precedence will be executed in the order that they appear in the Math Expression. Math Expressions can be grouped with parentheses to override evaluation precedence. Math Expressions that mix int64 and float64 will result in an error. It is up to the function using the Math Expression to determine what to do with that error and the default return value of nil. Division by zero is gracefully handled with an error, but other arithmetic operations that would result in a panic will still result in a panic. Division of integers results in an integer and follows Go's rules for division of integers.

Since Math Expressions support Path and Invocation, they are evaluated during data processing. As a result, in order for a function to be able to accept an Math Expressions as a parameter it must use a Getter.

Example Math Expressions

  • 1 + 1
  • end_time_unix_nano - end_time_unix_nano
  • sum([1, 2, 3, 4]) + (10 / 1) - 1
Boolean Expressions

Boolean Expressions allow a decision to be made about whether an Invocation should be called. Boolean Expressions are optional. When used, the parsed statement will include a Condition, which can be used to evaluate the result of the statement's Boolean Expression. Boolean Expressions always evaluate to a boolean value (true or false).

Boolean Expressions consist of the literal string where followed by one or more Booleans (see below). Booleans can be joined with the literal strings and and or. Booleans can be negated with the literal string not. Note that not has the highest precedence and and Boolean Expressions have higher precedence than or. Boolean Expressions can be grouped with parentheses to override evaluation precedence.

Booleans

Booleans can be either:

  • A literal boolean value (true or false).
  • A Comparison, made up of a left Value, an operator, and a right Value. See Values for details on what a Value can be.

Operators determine how the two Values are compared.

The valid operators are:

  • Equal (==). Tests if the left and right Values are equal (see the Comparison Rules below).
  • Not Equal (!=). Tests if the left and right Values are not equal.
  • Less Than (<). Tests if left is less than right.
  • Greater Than (>). Tests if left is greater than right.
  • Less Than or Equal To (<=). Tests if left is less than or equal to right.
  • Greater Than or Equal to (>=). Tests if left is greater than or equal to right.

Booleans can be negated with the not keyword such as

  • not true
  • not name == "foo"
  • not (IsMatch(name, "http_.*") == true and kind > 0)
Comparison Rules

The table below describes what happens when two Values are compared. Value types are provided by the user of OTTL. All of the value types supported by OTTL are listed in this table.

If numeric values are of different types, they are compared as float64.

For numeric values and strings, the comparison rules are those implemented by Go. Numeric values are done with signed comparisons. For binary values, false is considered to be less than true.

For values that are not one of the basic primitive types, the only valid comparisons are Equal and Not Equal, which are implemented using Go's standard == and != operators.

A not equal notation in the table below means that the "!=" operator returns true, but any other operator returns false. Note that a nil byte array is considered equivalent to nil.

base type bool int64 float64 string Bytes nil
bool normal, T>F not equal not equal not equal not equal not equal
int64 not equal compared as largest compared as float64 not equal not equal not equal
float64 not equal compared as float64 compared as largest not equal not equal not equal
string not equal not equal not equal normal (compared as Go strings) not equal not equal
Bytes not equal not equal not equal not equal byte-for-byte comparison []byte(nil) == nil
nil not equal not equal not equal not equal []byte(nil) == nil true for equality only

Examples:

  • name == "a name"
  • 1 < 2
  • attributes["custom-attr"] != nil
  • IsMatch(resource.attributes["host.name"], "pod-*") == true

Accessing signal telemetry

Access to signal telemetry is provided to OTTL functions through a TransformContext that is created by the user and passed during statement evaluation. To allow functions to operate on the TransformContext, the OTTL provides Getter, Setter, and GetSetter interfaces.

Getters and Setters

Getters allow for reading the following types of data. See the respective section of each Value type for how they are interpreted.

It is possible to update the Value in a telemetry field using a Setter. For read and write access, the GetSetter interface extends both interfaces.

Logging inside a OTTL function

To emit logs inside a OTTL function, add a parameter of type component.TelemetrySettings to the function signature. The OTTL will then inject the TelemetrySettings that were passed to NewParser into the function. TelemetrySettings can be used to emit logs.

Examples

These examples contain a SQL-like declarative language. Applied statements interact with only one signal, but statements can be declared across multiple signals. Functions used in examples are indicative of what could be useful, but are not implemented by the OTTL itself.

Remove a forbidden attribute
traces:
  delete(attributes["http.request.header.authorization"])
metrics:
  delete(attributes["http.request.header.authorization"])
logs:
  delete(attributes["http.request.header.authorization"])
Remove all attributes except for some
traces:
  keep_keys(attributes, ["http.method", "http.status_code"])
metrics:
  keep_keys(attributes, ["http.method", "http.status_code"])
logs:
  keep_keys(attributes, ["http.method", "http.status_code"])
Reduce cardinality of an attribute
traces:
  replace_match(attributes["http.target"], "/user/*/list/*", "/user/{userId}/list/{listId}")
Reduce cardinality of a span name
traces:
  replace_match(name, "GET /user/*/list/*", "GET /user/{userId}/list/{listId}")
Reduce cardinality of any matching attribute
traces:
  replace_all_matches(attributes, "/user/*/list/*", "/user/{userId}/list/{listId}")
Decrease the size of the telemetry payload
traces:
  delete(resource.attributes["process.command_line"])
metrics:
  delete(resource.attributes["process.command_line"])
logs:
  delete(resource.attributes["process.command_line"])
Attach information from resource into telemetry
metrics:
  set(attributes["k8s_pod"], resource.attributes["k8s.pod.name"])
Decorate error spans with additional information
traces:
  set(attributes["whose_fault"], "theirs") where attributes["http.status"] == 400 or attributes["http.status"] == 404
  set(attributes["whose_fault"], "ours") where attributes["http.status"] == 500
Update a spans ID
logs:
  set(span_id, SpanID(0x0000000000000000))
traces:
  set(span_id, SpanID(0x0000000000000000))
Convert metric name to snake case
metrics:
  set(metric.name, ConvertCase(metric.name, "snake"))
Check if an attribute exists
traces:
  set(attributes["test-passed"], true) where attributes["target-attribute"] != nil

Documentation

Index

Constants

View Source 
const (
	EQ compareOp = iota
	NE
	LT
	LTE
	GTE
	GT
)

These are the allowed values of a compareOp

View Source 
const (
	ADD mathOp = iota
	SUB
	MULT
	DIV
)

Variables

This section is empty.

Functions

This section is empty.

Types

type BoolExpr added in v0.64.0 

type BoolExpr[K any] struct {
	// contains filtered or unexported fields
}

func (BoolExpr[K]) Eval added in v0.64.0 

func (e BoolExpr[K]) Eval(ctx context.Context, tCtx K) (bool, error)

type Enum 

type Enum int64

type EnumParser 

type EnumParser func(*EnumSymbol) (*Enum, error)

type EnumSymbol 

type EnumSymbol string

type ErrorMode added in v0.71.0 

type ErrorMode string
const (
	IgnoreError    ErrorMode = "ignore"
	PropagateError ErrorMode = "propagate"
)

func (*ErrorMode) UnmarshalText added in v0.72.0 

func (e *ErrorMode) UnmarshalText(text []byte) error

type Expr added in v0.64.0 

type Expr[K any] struct {
	// contains filtered or unexported fields
}

func (Expr[K]) Eval added in v0.64.0 

func (e Expr[K]) Eval(ctx context.Context, tCtx K) (interface{}, error)

type ExprFunc 

type ExprFunc[K any] func(ctx context.Context, tCtx K) (interface{}, error)

type Field 

type Field struct {
	Name   string  `parser:"@Lowercase"`
	MapKey *string `parser:"( '[' @String ']' )?"`
}

Field is an item within a Path.

type GetSetter 

type GetSetter[K any] interface {
	Getter[K]
	Setter[K]
}

type Getter 

type Getter[K any] interface {
	Get(ctx context.Context, tCtx K) (interface{}, error)
}

type IntGetter added in v0.72.0 

type IntGetter[K any] interface {
	Get(ctx context.Context, tCtx K) (int64, error)
}

type Option added in v0.70.0 

type Option[K any] func(*Parser[K])

func WithEnumParser added in v0.70.0 

func WithEnumParser[K any](parser EnumParser) Option[K]

type Parser 

type Parser[K any] struct {
	// contains filtered or unexported fields
}

func NewParser 

func NewParser[K any](
	functions map[string]interface{},
	pathParser PathExpressionParser[K],
	settings component.TelemetrySettings,
	options ...Option[K],
) (Parser[K], error)

func (*Parser[K]) ParseStatement added in v0.72.0 

func (p *Parser[K]) ParseStatement(statement string) (*Statement[K], error)

func (*Parser[K]) ParseStatements 

func (p *Parser[K]) ParseStatements(statements []string) ([]*Statement[K], error)

type Path 

type Path struct {
	Fields []Field `parser:"@@ ( '.' @@ )*"`
}

Path represents a telemetry path mathExpression.

type PathExpressionParser 

type PathExpressionParser[K any] func(*Path) (GetSetter[K], error)

type Setter 

type Setter[K any] interface {
	Set(ctx context.Context, tCtx K, val interface{}) error
}

type StandardGetSetter 

type StandardGetSetter[K any] struct {
	Getter func(ctx context.Context, tCtx K) (interface{}, error)
	Setter func(ctx context.Context, tCtx K, val interface{}) error
}

func (StandardGetSetter[K]) Get 

func (path StandardGetSetter[K]) Get(ctx context.Context, tCtx K) (interface{}, error)

func (StandardGetSetter[K]) Set 

func (path StandardGetSetter[K]) Set(ctx context.Context, tCtx K, val interface{}) error

type StandardTypeGetter added in v0.72.0 

type StandardTypeGetter[K any, T any] struct {
	Getter func(ctx context.Context, tCtx K) (interface{}, error)
}

func (StandardTypeGetter[K, T]) Get added in v0.72.0 

func (g StandardTypeGetter[K, T]) Get(ctx context.Context, tCtx K) (T, error)

type Statement 

type Statement[K any] struct {
	// contains filtered or unexported fields
}

Statement holds a top level Statement for processing telemetry data. A Statement is a combination of a function invocation and the boolean expression to match telemetry for invoking the function.

func (*Statement[K]) Execute added in v0.62.0 

func (s *Statement[K]) Execute(ctx context.Context, tCtx K) (any, bool, error)

Execute is a function that will execute the statement's function if the statement's condition is met. Returns true if the function was run, returns false otherwise. If the statement contains no condition, the function will run and true will be returned. In addition, the functions return value is always returned.

type Statements added in v0.71.0 

type Statements[K any] struct {
	// contains filtered or unexported fields
}

Statements represents a list of statements that will be executed sequentially for a TransformContext.

func NewStatements added in v0.72.0 

func NewStatements[K any](statements []*Statement[K], telemetrySettings component.TelemetrySettings, options ...StatementsOption[K]) Statements[K]

func (*Statements[K]) Execute added in v0.71.0 

func (s *Statements[K]) Execute(ctx context.Context, tCtx K) error

Execute is a function that will execute all the statements in the Statements list.

type StatementsOption added in v0.72.0 

type StatementsOption[K any] func(*Statements[K])

func WithErrorMode added in v0.72.0 

func WithErrorMode[K any](errorMode ErrorMode) StatementsOption[K]

type StringGetter added in v0.72.0 

type StringGetter[K any] interface {
	Get(ctx context.Context, tCtx K) (string, error)
}

Source Files

View all Source files

Directories

Path Synopsis
contexts
internal/ottlcommon
ottldatapoint
ottllog
ottlmetric
ottlresource
ottlscope
ottlspan
ottlspanevent

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.