starlark

package
v1.15.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 31, 2020 License: MIT Imports: 10 Imported by: 1

README

Starlark Processor

The starlark processor calls a Starlark function for each matched metric, allowing for custom programmatic metric processing.

The Starlark language is a dialect of Python, and will be familiar to those who have experience with the Python language. However, there are major differences. Existing Python code is unlikely to work unmodified. The execution environment is sandboxed, and it is not possible to do I/O operations such as reading from files or sockets.

The Starlark specification has details about the syntax and available functions.

Telegraf minimum version: Telegraf 1.15.0

Configuration
[[processors.starlark]]
  ## The Starlark source can be set as a string in this configuration file, or
  ## by referencing a file containing the script.  Only one source or script
  ## should be set at once.

  ## Source of the Starlark script.
  source = '''
def apply(metric):
	return metric
'''

  ## File containing a Starlark script.
  # script = "/usr/local/bin/myscript.star"
Usage

The Starlark code should contain a function called apply that takes a metric as its single argument. The function will be called with each metric, and can return None, a single metric, or a list of metrics.

def apply(metric):
	return metric

For a list of available types and functions that can be used in the code, see the Starlark specification.

In addition to these, the following InfluxDB-specific types and functions are exposed to the script.

  • Metric(name): Create a new metric with the given measurement name. The metric will have no tags or fields and defaults to the current time.

  • name: The name is a string containing the metric measurement name.

  • tags: A dict-like object containing the metric's tags.

  • fields: A dict-like object containing the metric's fields. The values may be of type int, float, string, or bool.

  • time: The timestamp of the metric as an integer in nanoseconds since the Unix epoch.

  • deepcopy(metric): Make a copy of an existing metric.

Python Differences

While Starlark is similar to Python, there are important differences to note:

  • Starlark has limited support for error handling and no exceptions. If an error occurs the script will immediately end and Telegraf will drop the metric. Check the Telegraf logfile for details about the error.

  • It is not possible to import other packages and the Python standard library is not available.

  • It is not possible to open files or sockets.

  • These common keywords are not supported in the Starlark grammar:

    as             finally        nonlocal
    assert         from           raise
    class          global         try
    del            import         with
    except         is             yield
    
Common Questions

How can I drop/delete a metric?

If you don't return the metric it will be deleted. Usually this means the function should return None.

How should I make a copy of a metric?

Use deepcopy(metric) to create a copy of the metric.

How can I return multiple metrics?

You can return a list of metrics:

def apply(metric):
    m2 = deepcopy(metric)
    return [metric, m2]

What happens to a tracking metric if an error occurs in the script?

The metric is marked as undelivered.

How do I create a new metric?

Use the Metric(name) function and set at least one field.

What is the fastest way to iterate over tags/fields?

The fastest way to iterate is to use a for-loop on the tags or fields attribute:

def apply(metric):
    for k in metric.tags:
        pass
    return metric

When you use this form, it is not possible to modify the tags inside the loop, if this is needed you should use one of the .keys(), .values(), or .items() methods:

def apply(metric):
    for k, v in metric.tags.items():
        pass
    return metric

How can I save values across multiple calls to the script?

Telegraf freezes the global scope, which prevents it from being modified. Attempting to modify the global scope will fail with an error.

Examples

Open a PR to add any other useful Starlark examples.

Documentation

Index

Constants

This section is empty.

Variables

View Source
var FieldDictMethods = map[string]builtinMethod{
	"clear":      dict_clear,
	"get":        dict_get,
	"items":      dict_items,
	"keys":       dict_keys,
	"pop":        dict_pop,
	"popitem":    dict_popitem,
	"setdefault": dict_setdefault,
	"update":     dict_update,
	"values":     dict_values,
}
View Source
var TagDictMethods = map[string]builtinMethod{
	"clear":      dict_clear,
	"get":        dict_get,
	"items":      dict_items,
	"keys":       dict_keys,
	"pop":        dict_pop,
	"popitem":    dict_popitem,
	"setdefault": dict_setdefault,
	"update":     dict_update,
	"values":     dict_values,
}

Functions

This section is empty.

Types

type FieldDict

type FieldDict struct {
	*Metric
}

FieldDict is a starlark.Value for the metric fields. It is heavily based on the starlark.Dict.

func (FieldDict) Attr

func (d FieldDict) Attr(name string) (starlark.Value, error)

Attr implements the starlark.HasAttrs interface.

func (FieldDict) AttrNames

func (d FieldDict) AttrNames() []string

AttrNames implements the starlark.HasAttrs interface.

func (FieldDict) Clear

func (d FieldDict) Clear() error

func (FieldDict) Delete

func (d FieldDict) Delete(k starlark.Value) (v starlark.Value, found bool, err error)

func (FieldDict) Freeze

func (d FieldDict) Freeze()

func (FieldDict) Get

func (d FieldDict) Get(key starlark.Value) (v starlark.Value, found bool, err error)

Get implements the starlark.Mapping interface.

func (FieldDict) Hash

func (d FieldDict) Hash() (uint32, error)

func (FieldDict) Items

func (d FieldDict) Items() []starlark.Tuple

Items implements the starlark.IterableMapping interface.

func (FieldDict) Iterate

func (d FieldDict) Iterate() starlark.Iterator

Items implements the starlark.Mapping interface.

func (FieldDict) PopItem

func (d FieldDict) PopItem() (v starlark.Value, err error)

func (FieldDict) SetKey

func (d FieldDict) SetKey(k, v starlark.Value) error

SetKey implements the starlark.HasSetKey interface to support map update using x[k]=v syntax, like a dictionary.

func (FieldDict) String

func (d FieldDict) String() string

func (FieldDict) Truth

func (d FieldDict) Truth() starlark.Bool

func (FieldDict) Type

func (d FieldDict) Type() string

type FieldIterator

type FieldIterator struct {
	*Metric
	// contains filtered or unexported fields
}

func (*FieldIterator) Done

func (i *FieldIterator) Done()

Done implements the starlark.Iterator interface.

func (*FieldIterator) Next

func (i *FieldIterator) Next(p *starlark.Value) bool

Next implements the starlark.Iterator interface.

type Metric

type Metric struct {
	// contains filtered or unexported fields
}

func (*Metric) Attr

func (m *Metric) Attr(name string) (starlark.Value, error)

Attr implements the starlark.HasAttrs interface.

func (*Metric) AttrNames

func (m *Metric) AttrNames() []string

AttrNames implements the starlark.HasAttrs interface.

func (*Metric) Fields

func (m *Metric) Fields() FieldDict

func (*Metric) Freeze

func (m *Metric) Freeze()

func (*Metric) Hash

func (m *Metric) Hash() (uint32, error)

func (*Metric) Name

func (m *Metric) Name() starlark.String

func (*Metric) SetField

func (m *Metric) SetField(name string, value starlark.Value) error

SetField implements the starlark.HasSetField interface.

func (*Metric) SetName

func (m *Metric) SetName(value starlark.Value) error

func (*Metric) SetTime

func (m *Metric) SetTime(value starlark.Value) error

func (*Metric) String

func (m *Metric) String() string

String returns the starlark representation of the Metric.

The String function is called by both the repr() and str() functions, and so it behaves more like the repr function would in Python.

func (*Metric) Tags

func (m *Metric) Tags() TagDict

func (*Metric) Time

func (m *Metric) Time() starlark.Int

func (*Metric) Truth

func (m *Metric) Truth() starlark.Bool

func (*Metric) Type

func (m *Metric) Type() string

func (*Metric) Unwrap

func (m *Metric) Unwrap() telegraf.Metric

Unwrap removes the telegraf.Metric from the startlark.Metric.

func (*Metric) Wrap

func (m *Metric) Wrap(metric telegraf.Metric)

Wrap updates the starlark.Metric to wrap a new telegraf.Metric.

type Starlark

type Starlark struct {
	Source string `toml:"source"`
	Script string `toml:"script"`

	Log telegraf.Logger `toml:"-"`
	// contains filtered or unexported fields
}

func (*Starlark) Add

func (s *Starlark) Add(metric telegraf.Metric, acc telegraf.Accumulator) error

func (*Starlark) Description

func (s *Starlark) Description() string

func (*Starlark) Init

func (s *Starlark) Init() error

func (*Starlark) SampleConfig

func (s *Starlark) SampleConfig() string

func (*Starlark) Start

func (s *Starlark) Start(acc telegraf.Accumulator) error

func (*Starlark) Stop

func (s *Starlark) Stop() error

type TagDict

type TagDict struct {
	*Metric
}

TagDict is a starlark.Value for the metric tags. It is heavily based on the starlark.Dict.

func (TagDict) Attr

func (d TagDict) Attr(name string) (starlark.Value, error)

Attr implements the starlark.HasAttrs interface.

func (TagDict) AttrNames

func (d TagDict) AttrNames() []string

AttrNames implements the starlark.HasAttrs interface.

func (TagDict) Clear

func (d TagDict) Clear() error

func (TagDict) Delete

func (d TagDict) Delete(k starlark.Value) (v starlark.Value, found bool, err error)

func (TagDict) Freeze

func (d TagDict) Freeze()

func (TagDict) Get

func (d TagDict) Get(key starlark.Value) (v starlark.Value, found bool, err error)

Get implements the starlark.Mapping interface.

func (TagDict) Hash

func (d TagDict) Hash() (uint32, error)

func (TagDict) Items

func (d TagDict) Items() []starlark.Tuple

Items implements the starlark.IterableMapping interface.

func (TagDict) Iterate

func (d TagDict) Iterate() starlark.Iterator

Items implements the starlark.Mapping interface.

func (TagDict) PopItem

func (d TagDict) PopItem() (v starlark.Value, err error)

func (TagDict) SetKey

func (d TagDict) SetKey(k, v starlark.Value) error

SetKey implements the starlark.HasSetKey interface to support map update using x[k]=v syntax, like a dictionary.

func (TagDict) String

func (d TagDict) String() string

func (TagDict) Truth

func (d TagDict) Truth() starlark.Bool

func (TagDict) Type

func (d TagDict) Type() string

type TagIterator

type TagIterator struct {
	*Metric
	// contains filtered or unexported fields
}

func (*TagIterator) Done

func (i *TagIterator) Done()

Done implements the starlark.Iterator interface.

func (*TagIterator) Next

func (i *TagIterator) Next(p *starlark.Value) bool

Next implements the starlark.Iterator interface.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL