telegraf

README

Telegraf

Telegraf is an agent written in Go for collecting, processing, aggregating, and writing metrics.

Design goals are to have a minimal memory footprint with a plugin system so that developers in the community can easily add support for collecting metrics from local or remote services.

Telegraf is plugin-driven and has the concept of 4 distinct plugins:

1. Input Plugins collect metrics from the system, services, or 3rd party APIs
2. Processor Plugins transform, decorate, and/or filter metrics
3. Aggregator Plugins create aggregate metrics (e.g. mean, min, max, quantiles, etc.)
4. Output Plugins write metrics to various destinations

For more information on Processor and Aggregator plugins please read this.

New plugins are designed to be easy to contribute, we'll eagerly accept pull requests and will manage the set of plugins that Telegraf supports.

Contributing

There are many ways to contribute:

• Fix and report bugs
• Improve documentation
• Review code and feature proposals
• Answer questions on github and on the Community Site
• Contribute plugins

Installation:

You can download the binaries directly from the downloads page or from the releases section.

Ansible Role:

Ansible role: https://github.com/rossmcdonald/telegraf

From Source:

Telegraf requires golang version 1.8+, the Makefile requires GNU make.

Dependencies are managed with gdm, which is installed by the Makefile if you don't have it already.

1. Install Go
2. Setup your GOPATH
3. Run go get -d github.com/influxdata/telegraf
4. Run cd $GOPATH/src/github.com/influxdata/telegraf
5. Run make

Nightly Builds

These builds are generated from the master branch:

• telegraf_nightly_amd64.deb
• telegraf_nightly_arm64.deb
• telegraf-nightly.arm64.rpm
• telegraf_nightly_armel.deb
• telegraf-nightly.armel.rpm
• telegraf_nightly_armhf.deb
• telegraf-nightly.armv6hl.rpm
• telegraf-nightly_freebsd_amd64.tar.gz
• telegraf-nightly_freebsd_i386.tar.gz
• telegraf_nightly_i386.deb
• telegraf-nightly.i386.rpm
• telegraf-nightly_linux_amd64.tar.gz
• telegraf-nightly_linux_arm64.tar.gz
• telegraf-nightly_linux_armel.tar.gz
• telegraf-nightly_linux_armhf.tar.gz
• telegraf-nightly_linux_i386.tar.gz
• telegraf-nightly_linux_s390x.tar.gz
• telegraf_nightly_s390x.deb
• telegraf-nightly.s390x.rpm
• telegraf-nightly_windows_amd64.zip
• telegraf-nightly_windows_i386.zip
• telegraf-nightly.x86_64.rpm
• telegraf-static-nightly_linux_amd64.tar.gz

How to use it:

See usage with:

./telegraf --help

Generate a telegraf config file:

./telegraf config > telegraf.conf

Generate config with only cpu input & influxdb output plugins defined:

./telegraf --input-filter cpu --output-filter influxdb config

Run a single telegraf collection, outputing metrics to stdout:

./telegraf --config telegraf.conf --test

Run telegraf with all plugins defined in config file:

./telegraf --config telegraf.conf

Run telegraf, enabling the cpu & memory input, and influxdb output plugins:

./telegraf --config telegraf.conf --input-filter cpu:mem --output-filter influxdb

Configuration

See the configuration guide for a rundown of the more advanced configuration options.

Input Plugins

• aerospike
• amqp_consumer (rabbitmq)
• apache
• aws cloudwatch
• bcache
• bond
• cassandra
• ceph
• cgroup
• chrony
• consul
• conntrack
• couchbase
• couchdb
• DC/OS
• disque
• dmcache
• dns query time
• docker
• dovecot
• elasticsearch
• exec (generic executable plugin, support JSON, influx, graphite and nagios)
• fail2ban
• filestat
• fluentd
• graylog
• haproxy
• hddtemp
• http_response
• httpjson (generic JSON-emitting http service plugin)
• internal
• influxdb
• interrupts
• ipmi_sensor
• iptables
• jolokia (deprecated, use jolokia2)
• jolokia2
• kapacitor
• kubernetes
• leofs
• lustre2
• mailchimp
• memcached
• mesos
• minecraft
• mongodb
• mysql
• net_response
• nginx
• nginx_plus
• nsq
• nstat
• ntpq
• openldap
• opensmtpd
• pf
• phpfpm
• phusion passenger
• ping
• postfix
• postgresql_extensible
• postgresql
• powerdns
• procstat
• prometheus (can be used for Caddy server)
• puppetagent
• rabbitmq
• raindrops
• redis
• rethinkdb
• riak
• salesforce
• sensors
• smart
• snmp
• snmp_legacy
• solr
• sql server (microsoft)
• teamspeak
• tomcat
• twemproxy
• unbound
• varnish
• zfs
• zookeeper
• win_perf_counters (windows performance counters)
• win_services
• sysstat
• system
  • cpu
  • mem
  • net
  • netstat
  • disk
  • diskio
  • swap
  • processes
  • kernel (/proc/stat)
  • kernel (/proc/vmstat)
  • linux_sysctl_fs (/proc/sys/fs)

Telegraf can also collect metrics via the following service plugins:

• http_listener
• kafka_consumer
• mqtt_consumer
• nats_consumer
• nsq_consumer
• logparser
• statsd
• socket_listener
• tail
• tcp_listener
• udp_listener
• webhooks
  • filestack
  • github
  • mandrill
  • papertrail
  • particle
  • rollbar
• zipkin

Telegraf is able to parse the following input data formats into metrics, these formats may be used with input plugins supporting the data_format option:

• InfluxDB Line Protocol
• JSON
• Graphite
• Value
• Nagios
• Collectd

Processor Plugins

• printer

Aggregator Plugins

• basicstats
• minmax
• histogram

Output Plugins

• influxdb
• amon
• amqp (rabbitmq)
• aws kinesis
• aws cloudwatch
• cratedb
• datadog
• discard
• elasticsearch
• file
• graphite
• graylog
• instrumental
• kafka
• librato
• mqtt
• nats
• nsq
• opentsdb
• prometheus
• riemann
• riemann_legacy
• socket_writer
• tcp
• udp
• wavefront

Documentation

Index

• type Accumulator
• type Aggregator
• type Input
• type Metric
• type Output
• type Processor
• type ServiceInput
• type ServiceOutput
• type ValueType

Types

type Accumulator

type Accumulator interface {
	// AddFields adds a metric to the accumulator with the given measurement
	// name, fields, and tags (and timestamp). If a timestamp is not provided,
	// then the accumulator sets it to "now".
	// Create a point with a value, decorating it with tags
	// NOTE: tags is expected to be owned by the caller, don't mutate
	// it after passing to Add.
	AddFields(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddGauge is the same as AddFields, but will add the metric as a "Gauge" type
	AddGauge(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddCounter is the same as AddFields, but will add the metric as a "Counter" type
	AddCounter(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddSummary is the same as AddFields, but will add the metric as a "Summary" type
	AddSummary(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	// AddHistogram is the same as AddFields, but will add the metric as a "Histogram" type
	AddHistogram(measurement string,
		fields map[string]interface{},
		tags map[string]string,
		t ...time.Time)

	SetPrecision(precision, interval time.Duration)

	AddError(err error)
}

Accumulator is an interface for "accumulating" metrics from plugin(s). The metrics are sent down a channel shared between all plugins.

type Aggregator

type Aggregator interface {
	// SampleConfig returns the default configuration of the Input.
	SampleConfig() string

	// Description returns a one-sentence description on the Input.
	Description() string

	// Add the metric to the aggregator.
	Add(in Metric)

	// Push pushes the current aggregates to the accumulator.
	Push(acc Accumulator)

	// Reset resets the aggregators caches and aggregates.
	Reset()
}

Aggregator is an interface for implementing an Aggregator plugin. the RunningAggregator wraps this interface and guarantees that Add, Push, and Reset can not be called concurrently, so locking is not required when implementing an Aggregator plugin.

type Input

type Input interface {
	// SampleConfig returns the default configuration of the Input
	SampleConfig() string

	// Description returns a one-sentence description on the Input
	Description() string

	// Gather takes in an accumulator and adds the metrics that the Input
	// gathers. This is called every "interval"
	Gather(Accumulator) error
}

type Metric

type Metric interface {
	// Serialize serializes the metric into a line-protocol byte buffer,
	// including a newline at the end.
	Serialize() []byte
	// same as Serialize, but avoids an allocation.
	// returns number of bytes copied into dst.
	SerializeTo(dst []byte) int
	// String is the same as Serialize, but returns a string.
	String() string
	// Copy deep-copies the metric.
	Copy() Metric
	// Split will attempt to return multiple metrics with the same timestamp
	// whose string representations are no longer than maxSize.
	// Metrics with a single field may exceed the requested size.
	Split(maxSize int) []Metric

	// Tag functions
	HasTag(key string) bool
	AddTag(key, value string)
	RemoveTag(key string)

	// Field functions
	HasField(key string) bool
	AddField(key string, value interface{})
	RemoveField(key string) error

	// Name functions
	SetName(name string)
	SetPrefix(prefix string)
	SetSuffix(suffix string)

	// Getting data structure functions
	Name() string
	Tags() map[string]string
	Fields() map[string]interface{}
	Time() time.Time
	UnixNano() int64
	Type() ValueType
	Len() int // returns the length of the serialized metric, including newline
	HashID() uint64

	// aggregator things:
	SetAggregate(bool)
	IsAggregate() bool
}

type Output

type Output interface {
	// Connect to the Output
	Connect() error
	// Close any connections to the Output
	Close() error
	// Description returns a one-sentence description on the Output
	Description() string
	// SampleConfig returns the default configuration of the Output
	SampleConfig() string
	// Write takes in group of points to be written to the Output
	Write(metrics []Metric) error
}

type Processor

type Processor interface {
	// SampleConfig returns the default configuration of the Input
	SampleConfig() string

	// Description returns a one-sentence description on the Input
	Description() string

	// Apply the filter to the given metric
	Apply(in ...Metric) []Metric
}

type ServiceInput

type ServiceInput interface {
	// SampleConfig returns the default configuration of the Input
	SampleConfig() string

	// Description returns a one-sentence description on the Input
	Description() string

	// Gather takes in an accumulator and adds the metrics that the Input
	// gathers. This is called every "interval"
	Gather(Accumulator) error

	// Start starts the ServiceInput's service, whatever that may be
	Start(Accumulator) error

	// Stop stops the services and closes any necessary channels and connections
	Stop()
}

type ServiceOutput

type ServiceOutput interface {
	// Connect to the Output
	Connect() error
	// Close any connections to the Output
	Close() error
	// Description returns a one-sentence description on the Output
	Description() string
	// SampleConfig returns the default configuration of the Output
	SampleConfig() string
	// Write takes in group of points to be written to the Output
	Write(metrics []Metric) error
	// Start the "service" that will provide an Output
	Start() error
	// Stop the "service" that will provide an Output
	Stop()
}

type ValueType

type ValueType int

ValueType is an enumeration of metric types that represent a simple value.

const (
	Counter ValueType
	Gauge
	Untyped
	Summary
	Histogram
)

Possible values for the ValueType enum.