telegraf

README

Telegraf

Telegraf is an agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.

• Offers a comprehensive suite of over 300 plugins, covering a wide range of functionalities including system monitoring, cloud services, and message passing
• Enables the integration of user-defined code to collect, transform, and transmit data efficiently
• Compiles into a standalone static binary without any external dependencies, ensuring a streamlined deployment process
• Utilizes TOML for configuration, providing a user-friendly and unambiguous setup experience
• Developed with contributions from a diverse community of over 1,200 contributors

Users can choose plugins from a wide range of topics, including but not limited to:

• Devices: OPC UA, Modbus
• Logs: File, Tail, Directory Monitor
• Messaging: AMQP, Kafka, MQTT
• Monitoring: OpenTelemetry, Prometheus
• Networking: Cisco TelemetryMDT, gNMI
• System monitoring: CPU, Memory, Disk, Network, SMART, Docker, Nvidia SMI, etc.
• Universal: Exec, HTTP, HTTP Listener, SNMP, SQL
• Windows: Event Log, Management Instrumentation, Performance Counters

🔨 Installation

For binary builds, Docker images, RPM & DEB packages, and other builds of Telegraf, please see the install guide.

See the releases documentation for details on versioning and when releases are made.

💻 Usage

Users define a TOML configuration with the plugins and settings they wish to use, then pass that configuration to Telegraf. The Telegraf agent then collects data from inputs at each interval and sends data to outputs at each flush interval.

For a basic walkthrough see quick start.

📖 Documentation

For a full list of documentation including tutorials, reference and other material, start with the /docs directory.

Additionally, each plugin has its own README that includes details about how to configure, use, and sometimes debug or troubleshoot. Look under the /plugins directory for specific plugins.

Here are some commonly used documents:

• Changelog
• Configuration
• FAQ
• Releases
• Security

❤️ Contribute

We love our community of over 1,200 contributors! Many of the plugins included in Telegraf were originally contributed by community members. Check out our contributing guide if you are interested in helping out. Also, join us on our Community Slack or Community Forums if you have questions or comments for our engineering teams.

If you are completely new to Telegraf and InfluxDB, you can also enroll for free at InfluxDB university to take courses to learn more.

ℹ️ Support

Please use the Community Slack or Community Forums if you have questions or comments for our engineering teams. GitHub issues are limited to actual issues and feature requests only.

📜 License

Documentation

Index

• type Accumulator
• type AggregatingOutput
• type Aggregator
• type DeliveryInfo
• type DeprecationInfo
• type Field
• type Initializer
• type Input
• type LogLevel
  • func LogLevelFromString(name string) LogLevel
  • func (e LogLevel) Includes(level LogLevel) bool
  • func (e LogLevel) Indicator() string
  • func (e LogLevel) String() string
• type Logger
• type Metric
• type Output
• type Parser
• type ParserFunc
• type ParserFuncPlugin
• type ParserPlugin
• type PluginDescriber
• type PluginWithID
• type Processor
• type ResolveFunc
• type SecretStore
• type Serializer
• type SerializerFunc
• type SerializerFuncPlugin
• type SerializerPlugin
• type ServiceInput
• type StatefulPlugin
• type StreamingProcessor
• type Tag
• type TemplateMetric
• type TrackingAccumulator
• type TrackingData
• type TrackingID
• type TrackingMetric
• type UnwrappableMetric
• type ValueType

Types

type Accumulator

type Accumulator interface {
	// AddFields adds a metric to the accumulator with the given measurement
	// name, fields, and tags (and timestamp). If a timestamp is not provided,
	// then the accumulator sets it to "now".
	AddFields(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddGauge is the same as AddFields, but will add the metric as a "Gauge" type
	AddGauge(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddCounter is the same as AddFields, but will add the metric as a "Counter" type
	AddCounter(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddSummary is the same as AddFields, but will add the metric as a "Summary" type
	AddSummary(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddHistogram is the same as AddFields, but will add the metric as a "Histogram" type
	AddHistogram(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddMetric adds a metric to the accumulator.
	AddMetric(Metric)

	// SetPrecision sets the timestamp rounding precision. All metrics
	// added to the accumulator will have their timestamp rounded to the
	// nearest multiple of precision.
	SetPrecision(precision time.Duration)

	// Report an error.
	AddError(err error)

	// Upgrade to a TrackingAccumulator with space for maxTracked
	// metrics/batches.
	WithTracking(maxTracked int) TrackingAccumulator
}

Accumulator allows adding metrics to the processing flow.

type AggregatingOutput

type AggregatingOutput interface {
	Output

	// Add the metric to the aggregator
	Add(in Metric)
	// Push returns the aggregated metrics and is called every flush interval.
	Push() []Metric
	// Reset signals that the aggregator period is completed.
	Reset()
}

AggregatingOutput adds aggregating functionality to an Output. May be used if the Output only accepts a fixed set of aggregations over a time period. These functions may be called concurrently to the Write function.

type Aggregator

type Aggregator interface {
	PluginDescriber

	// Add the metric to the aggregator.
	Add(in Metric)

	// Push pushes the current aggregates to the accumulator.
	Push(acc Accumulator)

	// Reset resets the aggregators caches and aggregates.
	Reset()
}

Aggregator is an interface for implementing an Aggregator plugin. the RunningAggregator wraps this interface and guarantees that Add, Push, and Reset can not be called concurrently, so locking is not required when implementing an Aggregator plugin.

type DeliveryInfo

type DeliveryInfo interface {
	// ID is the TrackingID
	ID() TrackingID

	// Delivered returns true if the metric was processed successfully.
	Delivered() bool
}

DeliveryInfo provides the results of a delivered metric group.

type DeprecationInfo

type DeprecationInfo struct {
	// Since specifies the version since when the plugin is deprecated
	Since string
	// RemovalIn optionally specifies the version when the plugin is scheduled for removal
	RemovalIn string
	// Notice for the user on suggested replacements etc.
	Notice string
}

DeprecationInfo contains information for marking a plugin deprecated.

type Field

type Field struct {
	Key   string
	Value interface{}
}

Field represents a single field key and value.

type Initializer

type Initializer interface {
	// Init performs one time setup of the plugin and returns an error if the
	// configuration is invalid.
	Init() error
}

Initializer is an interface that all plugin types: Inputs, Outputs, Processors, and Aggregators can optionally implement to initialize the plugin.

type Input

type Input interface {
	PluginDescriber

	// Gather takes in an accumulator and adds the metrics that the Input
	// gathers. This is called every agent.interval
	Gather(Accumulator) error
}

type LogLevel

type LogLevel int

LogLevel denotes the level for logging

const (
	// None means nothing is logged
	None LogLevel = iota
	// Error will log error messages
	Error
	// Warn will log error messages and warnings
	Warn
	// Info will log error messages, warnings and information messages
	Info
	// Debug will log all of the above and debugging messages issued by plugins
	Debug
	// Trace will log all of the above and trace messages issued by plugins
	Trace
)

func LogLevelFromString

func LogLevelFromString(name string) LogLevel

func (LogLevel) Includes

func (e LogLevel) Includes(level LogLevel) bool

func (LogLevel) Indicator

func (e LogLevel) Indicator() string

func (LogLevel) String

func (e LogLevel) String() string

type Logger

type Logger interface {
	// Level returns the configured log-level of the logger
	Level() LogLevel

	// AddAttribute allows to add a key-value attribute to the logging output
	AddAttribute(key string, value interface{})

	// Errorf logs an error message, patterned after log.Printf.
	Errorf(format string, args ...interface{})
	// Error logs an error message, patterned after log.Print.
	Error(args ...interface{})
	// Warnf logs a warning message, patterned after log.Printf.
	Warnf(format string, args ...interface{})
	// Warn logs a warning message, patterned after log.Print.
	Warn(args ...interface{})
	// Infof logs an information message, patterned after log.Printf.
	Infof(format string, args ...interface{})
	// Info logs an information message, patterned after log.Print.
	Info(args ...interface{})
	// Debugf logs a debug message, patterned after log.Printf.
	Debugf(format string, args ...interface{})
	// Debug logs a debug message, patterned after log.Print.
	Debug(args ...interface{})
	// Tracef logs a trace message, patterned after log.Printf.
	Tracef(format string, args ...interface{})
	// Trace logs a trace message, patterned after log.Print.
	Trace(args ...interface{})
}

Logger defines an plugin-related interface for logging.

type Metric

type Metric interface {
	// Name is the primary identifier for the Metric and corresponds to the
	// measurement in the InfluxDB data model.
	Name() string

	// Tags returns the tags as a map.  This method is deprecated, use TagList instead.
	Tags() map[string]string

	// TagList returns the tags as a slice ordered by the tag key in lexical
	// bytewise ascending order.  The returned value should not be modified,
	// use the AddTag or RemoveTag methods instead.
	TagList() []*Tag

	// Fields returns the fields as a map.  This method is deprecated, use FieldList instead.
	Fields() map[string]interface{}

	// FieldList returns the fields as a slice in an undefined order.  The
	// returned value should not be modified, use the AddField or RemoveField
	// methods instead.
	FieldList() []*Field

	// Time returns the timestamp of the metric.
	Time() time.Time

	// Type returns a general type for the entire metric that describes how you
	// might interpret, aggregate the values. Used by prometheus and statsd.
	Type() ValueType

	// SetName sets the metric name.
	SetName(name string)

	// AddPrefix adds a string to the front of the metric name.  It is
	// equivalent to m.SetName(prefix + m.Name()).
	//
	// This method is deprecated, use SetName instead.
	AddPrefix(prefix string)

	// AddSuffix appends a string to the back of the metric name.  It is
	// equivalent to m.SetName(m.Name() + suffix).
	//
	// This method is deprecated, use SetName instead.
	AddSuffix(suffix string)

	// GetTag returns the value of a tag and a boolean to indicate if it was set.
	GetTag(key string) (string, bool)

	// HasTag returns true if the tag is set on the Metric.
	HasTag(key string) bool

	// AddTag sets the tag on the Metric.  If the Metric already has the tag
	// set then the current value is replaced.
	AddTag(key, value string)

	// RemoveTag removes the tag if it is set.
	RemoveTag(key string)

	// GetField returns the value of a field and a boolean to indicate if it was set.
	GetField(key string) (interface{}, bool)

	// HasField returns true if the field is set on the Metric.
	HasField(key string) bool

	// AddField sets the field on the Metric.  If the Metric already has the field
	// set then the current value is replaced.
	AddField(key string, value interface{})

	// RemoveField removes the tag if it is set.
	RemoveField(key string)

	// SetTime sets the timestamp of the Metric.
	SetTime(t time.Time)

	// SetType sets the value-type of the Metric.
	SetType(t ValueType)

	// HashID returns an unique identifier for the series.
	HashID() uint64

	// Copy returns a deep copy of the Metric.
	Copy() Metric

	// Accept marks the metric as processed successfully and written to an
	// output.
	Accept()

	// Reject marks the metric as processed unsuccessfully.
	Reject()

	// Drop marks the metric as processed successfully without being written
	// to any output.
	Drop()
}

Metric is the type of data that is processed by Telegraf. Input plugins, and to a lesser degree, Processor and Aggregator plugins create new Metrics and Output plugins write them.

type Output

type Output interface {
	PluginDescriber

	// Connect to the Output; connect is only called once when the plugin starts
	Connect() error
	// Close any connections to the Output. Close is called once when the output
	// is shutting down. Close will not be called until all writes have finished,
	// and Write() will not be called once Close() has been, so locking is not
	// necessary.
	Close() error
	// Write takes in group of points to be written to the Output
	Write(metrics []Metric) error
}

type Parser

type Parser interface {
	// Parse takes a byte buffer separated by newlines
	// ie, `cpu.usage.idle 90\ncpu.usage.busy 10`
	// and parses it into telegraf metrics
	//
	// Must be thread-safe.
	Parse(buf []byte) ([]Metric, error)

	// ParseLine takes a single string metric
	// ie, "cpu.usage.idle 90"
	// and parses it into a telegraf metric.
	//
	// Must be thread-safe.
	ParseLine(line string) (Metric, error)

	// SetDefaultTags tells the parser to add all of the given tags
	// to each parsed metric.
	// NOTE: do _not_ modify the map after you've passed it here!!
	SetDefaultTags(tags map[string]string)
}

Parser is an interface defining functions that a parser plugin must satisfy.

type ParserFunc

type ParserFunc func() (Parser, error)

ParserFunc is a function to create a new instance of a parser

type ParserFuncPlugin

type ParserFuncPlugin interface {
	// GetParser returns a new parser.
	SetParserFunc(fn ParserFunc)
}

ParserFuncPlugin is an interface for plugins that are able to parse arbitrary data formats.

type ParserPlugin

type ParserPlugin interface {
	// SetParser sets the parser function for the interface
	SetParser(parser Parser)
}

ParserPlugin is an interface for plugins that are able to parse arbitrary data formats.

type PluginDescriber

type PluginDescriber interface {
	// SampleConfig returns the default configuration of the Plugin
	SampleConfig() string
}

PluginDescriber contains the functions all plugins must implement to describe themselves to Telegraf. Note that all plugins may define a logger that is not part of the interface, but will receive an injected logger if it's set. eg: Log telegraf.Logger `toml:"-"`

type PluginWithID

type PluginWithID interface {
	// ID returns the ID of the plugin instance. This function has to be
	// callable directly after the plugin's Init() function if there is any!
	ID() string
}

PluginWithID allows a plugin to overwrite its identifier of the plugin instance by a user specified value. By default the ID is generated using the plugin's configuration.

type Processor

type Processor interface {
	PluginDescriber

	// Apply the filter to the given metric.
	Apply(in ...Metric) []Metric
}

Processor is a processor plugin interface for defining new inline processors. these are extremely efficient and should be used over StreamingProcessor if you do not need asynchronous metric writes.

type ResolveFunc

type ResolveFunc func() ([]byte, bool, error)

ResolveFunc is a function to resolve the secret. The returned flag indicates if the resolver is static (false), i.e. the secret will not change over time, or dynamic (true) to handle secrets that change over time (e.g. TOTP).

type SecretStore

type SecretStore interface {
	Initializer
	PluginDescriber

	// Get searches for the given key and return the secret
	Get(key string) ([]byte, error)

	// Set sets the given secret for the given key
	Set(key, value string) error

	// List lists all known secret keys
	List() ([]string, error)

	// GetResolver returns a function to resolve the given key.
	GetResolver(key string) (ResolveFunc, error)
}

SecretStore is an interface defining functions that a secret-store plugin must satisfy.

type Serializer

type Serializer interface {
	// Serialize takes a single telegraf metric and turns it into a byte buffer.
	// separate metrics should be separated by a newline, and there should be
	// a newline at the end of the buffer.
	//
	// New plugins should use SerializeBatch instead to allow for non-line
	// delimited metrics.
	Serialize(metric Metric) ([]byte, error)

	// SerializeBatch takes an array of telegraf metric and serializes it into
	// a byte buffer.  This method is not required to be suitable for use with
	// line oriented framing.
	SerializeBatch(metrics []Metric) ([]byte, error)
}

Serializer is an interface defining functions that a serializer plugin must satisfy.

Implementations of this interface should be reentrant but are not required to be thread-safe.

type SerializerFunc

type SerializerFunc func() (Serializer, error)

SerializerFunc is a function to create a new instance of a serializer

type SerializerFuncPlugin

type SerializerFuncPlugin interface {
	// GetParser returns a new parser.
	SetSerializerFunc(fn SerializerFunc)
}

SerializerFuncPlugin is an interface for plugins that are able to serialize arbitrary data formats and require multiple instances of a parser.

type SerializerPlugin

type SerializerPlugin interface {
	// SetSerializer sets the serializer function for the interface.
	SetSerializer(serializer Serializer)
}

SerializerPlugin is an interface for plugins that are able to serialize telegraf metrics into arbitrary data formats.

type ServiceInput

type ServiceInput interface {
	Input

	// Start the ServiceInput.  The Accumulator may be retained and used until
	// Stop returns.
	Start(Accumulator) error

	// Stop stops the services and closes any necessary channels and connections.
	// Metrics should not be written out to the accumulator once stop returns, so
	// Stop() should stop reading and wait for any in-flight metrics to write out
	// to the accumulator before returning.
	Stop()
}

type StatefulPlugin

type StatefulPlugin interface {
	// GetState returns the current state of the plugin to persist
	// The returned state can be of any time as long as it can be
	// serialized to JSON. The best choice is a structure defined in
	// your plugin.
	// Note: This function has to be callable directly after the
	// plugin's Init() function if there is any!
	GetState() interface{}

	// SetState is called by the Persister once after loading and
	// initialization (after Init() function).
	SetState(state interface{}) error
}

StatefulPlugin contains the functions that plugins must implement to persist an internal state across Telegraf runs. Note that plugins may define a persister that is not part of the interface, but can be used to trigger state updates by the plugin if it exists in the plugin struct, eg: Persister telegraf.StatePersister `toml:"-"`

type StreamingProcessor

type StreamingProcessor interface {
	PluginDescriber

	// Start is called once when the plugin starts; it is only called once per
	// plugin instance, and never in parallel.
	// Start should return once it is ready to receive metrics.
	// The passed in accumulator is the same as the one passed to Add(), so you
	// can choose to save it in the plugin, or use the one received from Add().
	Start(acc Accumulator) error

	// Add is called for each metric to be processed. The Add() function does not
	// need to wait for the metric to be processed before returning, and it may
	// be acceptable to let background goroutine(s) handle the processing if you
	// have slow processing you need to do in parallel.
	// Keep in mind Add() should not spawn unbounded goroutines, so you may need
	// to use a semaphore or pool of workers (eg: reverse_dns plugin does this)
	// Metrics you don't want to pass downstream should have metric.Drop() called,
	// rather than simply omitting the acc.AddMetric() call
	Add(metric Metric, acc Accumulator) error

	// Stop gives you an opportunity to gracefully shut down the processor.
	// Once Stop() is called, Add() will not be called any more. If you are using
	// goroutines, you should wait for any in-progress metrics to be processed
	// before returning from Stop().
	// When stop returns, you should no longer be writing metrics to the
	// accumulator.
	Stop()
}

StreamingProcessor is a processor that can take in a stream of messages

type Tag

type Tag struct {
	Key   string
	Value string
}

Tag represents a single tag key and value.

type TemplateMetric

type TemplateMetric interface {
	Name() string
	Field(key string) interface{}
	Fields() map[string]interface{}
	Tag(key string) string
	Tags() map[string]string
	Time() time.Time
	String() string
}

TemplateMetric is an interface to use in templates (e.g text/template) to generate complex strings from metric properties e.g. '{{.Name}}-{{.Tag "foo"}}-{{.Field "bar"}}'

type TrackingAccumulator

type TrackingAccumulator interface {
	Accumulator

	// Add the Metric and arrange for tracking feedback after processing.
	AddTrackingMetric(m Metric) TrackingID

	// Add a group of Metrics and arrange for a signal when the group has been
	// processed.
	AddTrackingMetricGroup(group []Metric) TrackingID

	// Delivered returns a channel that will contain the tracking results.
	Delivered() <-chan DeliveryInfo
}

TrackingAccumulator is an Accumulator that provides a signal when the metric has been fully processed. Sending more metrics than the accumulator has been allocated for without reading status from the Accepted or Rejected channels is an error.

type TrackingData

type TrackingData interface {
	// ID is the TrackingID
	ID() TrackingID

	// RefCount is the number of tracking metrics still persistent and referencing this tracking ID
	RefCount() int32
}

type TrackingID

type TrackingID uint64

TrackingID uniquely identifies a tracked metric group

type TrackingMetric

type TrackingMetric interface {
	// TrackingID returns the ID used for tracking the metric
	TrackingID() TrackingID
	TrackingData() TrackingData
	UnwrappableMetric
}

type UnwrappableMetric

type UnwrappableMetric interface {
	// Unwrap allows to access the underlying raw metric if an implementation
	// wraps it in the first place.
	Unwrap() Metric
}

type ValueType

type ValueType int

ValueType is an enumeration of metric types that represent a simple value.

const (
	Counter ValueType
	Gauge
	Untyped
	Summary
	Histogram
)

Possible values for the ValueType enum.