opentsdb

README

OpenTSDB Input

InfluxDB supports both the telnet and HTTP OpenTSDB protocol. This means that InfluxDB can act as a drop-in replacement for your OpenTSDB system.

Configuration

The OpenTSDB inputs allow the binding address, target database, and target retention policy within that database, to be set. If the database does not exist, it will be created automatically when the input is initialized. If you also decide to configure retention policy (without configuration the input will use the auto-created default retention policy), both the database and retention policy must already exist.

The write-consistency-level can also be set. If any write operations do not meet the configured consistency guarantees, an error will occur and the data will not be indexed. The default consistency-level is ONE.

The OpenTSDB input also performs internal batching of the points it receives, as batched writes to the database are more efficient. The default batch size is 1000, pending batch factor is 5, with a batch timeout of 1 second. This means the input will write batches of maximum size 1000, but if a batch has not reached 1000 points within 1 second of the first point being added to a batch, it will emit that batch regardless of size. The pending batch factor controls how many batches can be in memory at once, allowing the input to transmit a batch, while still building other batches.

Documentation

Overview

Package opentsdb provides a service for InfluxDB to ingest data via the opentsdb protocol.

Index

• Constants
• type Config
  • func NewConfig() Config
  • func (c *Config) WithDefaults() *Config
• type Configs
  • func (c Configs) Diagnostics() (*diagnostics.Diagnostics, error)
  • func (c Configs) Enabled() bool
• type Handler
  • func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)
• type Service
  • func NewService(c Config) (*Service, error)
  • func (s *Service) Addr() net.Addr
  • func (s *Service) Close() error
  • func (s *Service) Closed() bool
  • func (s *Service) Open() error
  • func (s *Service) Statistics(tags map[string]string) []models.Statistic
  • func (s *Service) WithLogger(log *zap.Logger)
• type Statistics

Constants

const (
	// DefaultBindAddress is the default address that the service binds to.
	DefaultBindAddress = ":4242"

	// DefaultDatabase is the default database used for writes.
	DefaultDatabase = "opentsdb"

	// DefaultRetentionPolicy is the default retention policy used for writes.
	DefaultRetentionPolicy = ""

	// DefaultConsistencyLevel is the default write consistency level.
	DefaultConsistencyLevel = "one"

	// DefaultBatchSize is the default OpenTSDB batch size.
	DefaultBatchSize = 1000

	// DefaultBatchTimeout is the default OpenTSDB batch timeout.
	DefaultBatchTimeout = time.Second

	// DefaultBatchPending is the default number of batches that can be in the queue.
	DefaultBatchPending = 5

	// DefaultCertificate is the default location of the certificate used when TLS is enabled.
	DefaultCertificate = "/etc/ssl/influxdb.pem"
)

Types

type Config

type Config struct {
	Enabled          bool          `toml:"enabled"`
	BindAddress      string        `toml:"bind-address"`
	Database         string        `toml:"database"`
	RetentionPolicy  string        `toml:"retention-policy"`
	ConsistencyLevel string        `toml:"consistency-level"`
	TLSEnabled       bool          `toml:"tls-enabled"`
	Certificate      string        `toml:"certificate"`
	BatchSize        int           `toml:"batch-size"`
	BatchPending     int           `toml:"batch-pending"`
	BatchTimeout     toml.Duration `toml:"batch-timeout"`
	LogPointErrors   bool          `toml:"log-point-errors"`
	TLS              *tls.Config   `toml:"-"`
}

Config represents the configuration of the OpenTSDB service.

func NewConfig

func NewConfig() Config

NewConfig returns a new config for the service.

func (*Config) WithDefaults

func (c *Config) WithDefaults() *Config

WithDefaults takes the given config and returns a new config with any required default values set.

type Configs

type Configs []Config

Configs wraps a slice of Config to aggregate diagnostics.

func (Configs) Diagnostics

func (c Configs) Diagnostics() (*diagnostics.Diagnostics, error)

Diagnostics returns one set of diagnostics for all of the Configs.

func (Configs) Enabled

func (c Configs) Enabled() bool

Enabled returns true if any underlying Config is Enabled.

type Handler

type Handler struct {
	Database        string
	RetentionPolicy string

	PointsWriter interface {
		WritePointsPrivileged(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, points []models.Point) error
	}

	Logger *zap.Logger
	// contains filtered or unexported fields
}

Handler is an http.Handler for the OpenTSDB service.

func (*Handler) ServeHTTP

func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP handles an HTTP request of the OpenTSDB REST API.

type Service

type Service struct {
	BindAddress     string
	Database        string
	RetentionPolicy string

	PointsWriter interface {
		WritePointsPrivileged(database, retentionPolicy string, consistencyLevel models.ConsistencyLevel, points []models.Point) error
	}
	MetaClient interface {
		CreateDatabase(name string) (*meta.DatabaseInfo, error)
	}

	LogPointErrors bool
	Logger         *zap.Logger
	// contains filtered or unexported fields
}

Service manages the listener and handler for an HTTP endpoint.

func NewService

func NewService(c Config) (*Service, error)

NewService returns a new instance of Service.

func (*Service) Addr

func (s *Service) Addr() net.Addr

Addr returns the listener's address. Returns nil if listener is closed.

func (*Service) Close

func (s *Service) Close() error

Close closes the openTSDB service.

func (*Service) Closed

func (s *Service) Closed() bool

Closed returns true if the service is currently closed.

func (*Service) Open

func (s *Service) Open() error

Open starts the service.

func (*Service) Statistics

func (s *Service) Statistics(tags map[string]string) []models.Statistic

Statistics returns statistics for periodic monitoring.

func (*Service) WithLogger

func (s *Service) WithLogger(log *zap.Logger)

WithLogger sets the logger for the service.

type Statistics

type Statistics struct {
	HTTPConnectionsHandled   int64
	ActiveTelnetConnections  int64
	HandledTelnetConnections int64
	TelnetPointsReceived     int64
	TelnetBytesReceived      int64
	TelnetReadError          int64
	TelnetBadLine            int64
	TelnetBadTime            int64
	TelnetBadTag             int64
	TelnetBadFloat           int64
	BatchesTransmitted       int64
	PointsTransmitted        int64
	BatchesTransmitFail      int64
	ActiveConnections        int64
	HandledConnections       int64
	InvalidDroppedPoints     int64
}

Statistics maintains statistics for the subscriber service.