README ¶
The collectd Input
The collectd input allows InfluxDB to accept data transmitted in collectd native format. This data is transmitted over UDP.
A note on UDP/IP OS Buffer sizes
If you're running Linux or FreeBSD, please adjust your OS UDP buffer size limit, see here for more details.
Configuration
Each collectd input allows the binding address, target database, and target retention policy to be set. If the database does not exist, it will be created automatically when the input is initialized. If the retention policy is not configured, then the default retention policy for the database is used. However if the retention policy is set, the retention policy must be explicitly created. The input will not automatically create it.
Each collectd input also performs internal batching of the points it receives, as batched writes to the database are more efficient. The default batch size is 1000, pending batch factor is 5, with a batch timeout of 1 second. This means the input will write batches of maximum size 1000, but if a batch has not reached 1000 points within 1 second of the first point being added to a batch, it will emit that batch regardless of size. The pending batch factor controls how many batches can be in memory at once, allowing the input to transmit a batch, while still building other batches.
Multi-value plugins can be handled two ways. Setting parse-multivalue-plugin to "split" will parse and store the multi-value plugin data (e.g., df free:5000,used:1000) into separate measurements (e.g., (df_free, value=5000) (df_used, value=1000)), while "join" will parse and store the multi-value plugin as a single multi-value measurement (e.g., (df, free=5000,used=1000)). "split" is the default behavior for backward compatability with previous versions of influxdb.
The path to the collectd types database file may also be set.
Large UDP packets
Please note that UDP packets larger than the standard size of 1452 are dropped at the time of ingestion. Be sure to set MaxPacketSize
to 1452 in the collectd configuration.
Config Example
[[collectd]]
enabled = true
bind-address = ":25826" # the bind address
database = "collectd" # Name of the database that will be written to
retention-policy = ""
batch-size = 5000 # will flush if this many points get buffered
batch-pending = 10 # number of batches that may be pending in memory
batch-timeout = "10s"
read-buffer = 0 # UDP read buffer size, 0 means to use OS default
typesdb = "/usr/share/collectd/types.db"
security-level = "none" # "none", "sign", or "encrypt"
auth-file = "/etc/collectd/auth_file"
parse-multivalue-plugin = "split" # "split" or "join"
Documentation ¶
Overview ¶
Package collectd provides a service for InfluxDB to ingest data via the collectd protocol.
Index ¶
- Constants
- func TypesDBFile(path string) (typesdb *api.TypesDB, err error)
- type Config
- type Configs
- type Service
- func (s *Service) Addr() net.Addr
- func (s *Service) Close() error
- func (s *Service) Open() error
- func (s *Service) SetTypes(types string) (err error)
- func (s *Service) Statistics(tags map[string]string) []models.Statistic
- func (s *Service) UnmarshalValueList(vl *api.ValueList) []models.Point
- func (s *Service) UnmarshalValueListPacked(vl *api.ValueList) []models.Point
- func (s *Service) WithLogger(log zap.Logger)
- type Statistics
Constants ¶
const ( // DefaultBindAddress is the default port to bind to. DefaultBindAddress = ":25826" // DefaultDatabase is the default DB to write to. DefaultDatabase = "collectd" // DefaultRetentionPolicy is the default retention policy of the writes. DefaultRetentionPolicy = "" // DefaultBatchSize is the default write batch size. DefaultBatchSize = 5000 // DefaultBatchPending is the default number of pending write batches. DefaultBatchPending = 10 // DefaultBatchDuration is the default batch timeout duration. DefaultBatchDuration = toml.Duration(10 * time.Second) // DefaultTypesDB is the default location of the collectd types db file. DefaultTypesDB = "/usr/share/collectd/types.db" // DefaultReadBuffer is the default buffer size for the UDP listener. // Sets the size of the operating system's receive buffer associated with // the UDP traffic. Keep in mind that the OS must be able // to handle the number set here or the UDP listener will error and exit. // // DefaultReadBuffer = 0 means to use the OS default, which is usually too // small for high UDP performance. // // Increasing OS buffer limits: // Linux: sudo sysctl -w net.core.rmem_max=<read-buffer> // BSD/Darwin: sudo sysctl -w kern.ipc.maxsockbuf=<read-buffer> DefaultReadBuffer = 0 // DefaultSecurityLevel is the default security level. DefaultSecurityLevel = "none" // DefaultAuthFile is the default location of the user/password file. DefaultAuthFile = "/etc/collectd/auth_file" // DefaultParseMultiValuePlugin is "split", defaulting to version <1.2 where plugin values were split into separate rows DefaultParseMultiValuePlugin = "split" )
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct { Enabled bool `toml:"enabled"` BindAddress string `toml:"bind-address"` Database string `toml:"database"` RetentionPolicy string `toml:"retention-policy"` BatchSize int `toml:"batch-size"` BatchPending int `toml:"batch-pending"` BatchDuration toml.Duration `toml:"batch-timeout"` ReadBuffer int `toml:"read-buffer"` TypesDB string `toml:"typesdb"` SecurityLevel string `toml:"security-level"` AuthFile string `toml:"auth-file"` ParseMultiValuePlugin string `toml:"parse-multivalue-plugin"` }
Config represents a configuration for the collectd service.
func (*Config) WithDefaults ¶ added in v0.13.0
WithDefaults takes the given config and returns a new config with any required default values set.
type Configs ¶ added in v1.3.0
type Configs []Config
Configs wraps a slice of Config to aggregate diagnostics.
func (Configs) Diagnostics ¶ added in v1.3.0
func (c Configs) Diagnostics() (*diagnostics.Diagnostics, error)
Diagnostics returns one set of diagnostics for all of the Configs.
type Service ¶
type Service struct { Config *Config MetaClient metaClient PointsWriter pointsWriter Logger zap.Logger // contains filtered or unexported fields }
Service represents a UDP server which receives metrics in collectd's binary protocol and stores them in InfluxDB.
func NewService ¶
NewService returns a new instance of the collectd service.
func (*Service) Statistics ¶ added in v1.0.0
Statistics returns statistics for periodic monitoring.
func (*Service) UnmarshalValueList ¶ added in v1.2.0
UnmarshalValueList translates a ValueList into InfluxDB data points.
func (*Service) UnmarshalValueListPacked ¶ added in v1.4.0
UnmarshalValueListPacked is an alternative to the original UnmarshalValueList. The difference is that the original provided measurements like (PLUGIN_DSNAME, ["value",xxx]) while this one will provide measurements like (PLUGIN, {["DSNAME",xxx]}). This effectively joins collectd data that should go together, such as: (df, {["used",1000],["free",2500]}).
func (*Service) WithLogger ¶ added in v1.2.0
WithLogger sets the service's logger.