config

package
v3.8.1+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 1, 2018 License: Apache-2.0 Imports: 25 Imported by: 0

Documentation

Overview

Package config contains configuration structures and related helper logic for all agent components.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// EnvReplacer replaces . and - with _
	EnvReplacer = strings.NewReplacer(".", "_", "-", "_")
)

Functions

func CallConfigure 

func CallConfigure(instance, conf interface{}) error

CallConfigure will call the Configure method on an observer or monitor with a `conf` object, typed to the correct type. This allows monitors/observers to set the type of the config object to their own config and not have to worry about casting or converting.

func DecodeExtraConfig 

func DecodeExtraConfig(in CustomConfigurable, out interface{}, strict bool) error

DecodeExtraConfig will pull out the OtherConfig values from both ObserverConfig and MonitorConfig and decode them to a struct that is provided in the `out` arg. Whether all fields have to be in 'out' is determined by the 'strict' flag. Any errors decoding will cause `out` to be nil.

func DecodeExtraConfigStrict 

func DecodeExtraConfigStrict(in CustomConfigurable, out interface{}) error

DecodeExtraConfigStrict will pull out any config values from 'in' and put them on the 'out' struct, returning an error if anything in 'in' isn't in 'out'.

func FillInConfigTemplate 

func FillInConfigTemplate(embeddedFieldName string, configTemplate interface{}, conf CustomConfigurable) error

FillInConfigTemplate takes a config template value that a monitor/observer provided and fills it in dynamically from the provided conf

func LoadConfig 

func LoadConfig(ctx context.Context, configPath string) (<-chan *Config, error)

LoadConfig handles loading the main config file and recursively rendering any dynamic values in the config. If watchInterval is 0, the config will be loaded once and sent to the returned channel, after which the channel will be closed. Otherwise, the returned channel will remain open and will be sent any config updates.

func ToString 

func ToString(conf interface{}) string

ToString converts a config struct to a pseudo-yaml text outut. If a struct field has the 'neverLog' tag, its value will be replaced by asterisks, or completely omitted if the tag value is 'omit'.

func ValidateCustomConfig 

func ValidateCustomConfig(conf interface{}) error

ValidateCustomConfig for module-specific config ahead of time for a specific module configuration. This way, the Configure method of modules will be guaranteed to receive valid configuration. The module-specific configuration struct must implement the Validate method that returns a bool.

Types

type CollectdConfig 

type CollectdConfig struct {
	// If you won't be using any collectd monitors, this can be set to true to
	// prevent collectd from pre-initializing
	DisableCollectd bool `yaml:"disableCollectd" default:"false"`
	// How many read intervals before abandoning a metric. Doesn't affect much
	// in normal usage.
	// See [Timeout](https://collectd.org/documentation/manpages/collectd.conf.5.shtml#timeout_iterations).
	Timeout int `yaml:"timeout" default:"40"`
	// Number of threads dedicated to executing read callbacks. See
	// [ReadThreads](https://collectd.org/documentation/manpages/collectd.conf.5.shtml#readthreads_num)
	ReadThreads int `yaml:"readThreads" default:"5"`
	// Number of threads dedicated to writing value lists to write callbacks.
	// This should be much less than readThreads because writing is batched in
	// the write_http plugin that writes back to the agent.
	// See [WriteThreads](https://collectd.org/documentation/manpages/collectd.conf.5.shtml#writethreads_num).
	WriteThreads int `yaml:"writeThreads" default:"2"`
	// The maximum numbers of values in the queue to be written back to the
	// agent from collectd.  Since the values are written to a local socket
	// that the agent exposes, there should be almost no queuing and the
	// default should be more than sufficient. See
	// [WriteQueueLimitHigh](https://collectd.org/documentation/manpages/collectd.conf.5.shtml#writequeuelimithigh_highnum)
	WriteQueueLimitHigh int `yaml:"writeQueueLimitHigh" default:"500000"`
	// The lowest number of values in the collectd queue before which metrics
	// begin being randomly dropped.  See
	// [WriteQueueLimitLow](https://collectd.org/documentation/manpages/collectd.conf.5.shtml#writequeuelimitlow_lownum)
	WriteQueueLimitLow int `yaml:"writeQueueLimitLow" default:"400000"`
	// Collectd's log level -- info, notice, warning, or err
	LogLevel string `yaml:"logLevel" default:"notice"`
	// A default read interval for collectd plugins.  If zero or undefined,
	// will default to the global agent interval.  Some collectd python
	// monitors do not support overridding the interval at the monitor level,
	// but this setting will apply to them.
	IntervalSeconds int `yaml:"intervalSeconds" default:"0"`
	// The local IP address of the server that the agent exposes to which
	// collectd will send metrics.  This defaults to an arbitrary address in
	// the localhost subnet, but can be overridden if needed.
	WriteServerIPAddr string `yaml:"writeServerIPAddr" default:"127.9.8.7"`
	// The port of the agent's collectd metric sink server.  If set to zero
	// (the default) it will allow the OS to assign it a free port.
	WriteServerPort uint16 `yaml:"writeServerPort" default:"0"`
	// This is where the agent will write the collectd config files that it
	// manages.  If you have secrets in those files, consider setting this to a
	// path on a tmpfs mount.  The files in this directory should be considered
	// transient -- there is no value in editing them by hand.  If you want to
	// add your own collectd config, see the collectd/custom monitor.
	ConfigDir string `yaml:"configDir" default:"/var/run/signalfx-agent/collectd"`

	// The following are propagated from the top-level config
	BundleDir            string `yaml:"-"`
	HasGenericJMXMonitor bool   `yaml:"-"`
	// Assigned by manager, not by user
	InstanceName string `yaml:"-"`
	// A hack to allow custom collectd to easily specify a single monitorID via
	// query parameter
	WriteServerQuery string `yaml:"-"`
}

CollectdConfig high-level configurations

func (*CollectdConfig) ConfigFilePath 

func (cc *CollectdConfig) ConfigFilePath() string

ConfigFilePath returns the path where collectd should render its main config file.

func (*CollectdConfig) Hash 

func (cc *CollectdConfig) Hash() uint64

Hash calculates a unique hash value for this config struct

func (*CollectdConfig) InstanceConfigDir 

func (cc *CollectdConfig) InstanceConfigDir() string

InstanceConfigDir is the directory underneath the ConfigDir that is specific to this collectd instance.

func (*CollectdConfig) ManagedConfigDir 

func (cc *CollectdConfig) ManagedConfigDir() string

ManagedConfigDir returns the dir path where all monitor config should go.

func (*CollectdConfig) Validate 

func (cc *CollectdConfig) Validate() error

Validate the collectd specific config

func (*CollectdConfig) WriteServerURL 

func (cc *CollectdConfig) WriteServerURL() string

WriteServerURL is the local address served by the agent where collect should write datapoints

type Config 

type Config struct {
	// The access token for the org that should receive the metrics emitted by
	// the agent.
	SignalFxAccessToken string `yaml:"signalFxAccessToken" neverLog:"true" validate:"required"`
	// The URL of SignalFx ingest server.  Should be overridden if using the
	// Metric Proxy.  If you want to send trace spans to a different location,
	// set the `traceEndpointUrl` option.
	IngestURL string `yaml:"ingestUrl" default:"https://ingest.signalfx.com"`
	// The full URL (including path) to the trace ingest server.  If this is
	// not set, all trace spans will be sent to the `ingestUrl` configured
	// above.
	TraceEndpointURL string `yaml:"traceEndpointUrl"`
	// The SignalFx API base URL
	APIURL string `yaml:"apiUrl" default:"https://api.signalfx.com"`
	// The hostname that will be reported as the `host` dimension. If blank,
	// this will be auto-determined by the agent based on a reverse lookup of
	// the machine's IP address.
	Hostname string `yaml:"hostname"`
	// If true (the default), and the `hostname` option is not set, the
	// hostname will be determined by doing a reverse DNS query on the IP
	// address that is returned by querying for the bare hostname.  This is
	// useful in cases where the hostname reported by the kernel is a short
	// name.
	UseFullyQualifiedHost *bool `yaml:"useFullyQualifiedHost"`
	// Our standard agent model is to collect metrics for services running on
	// the same host as the agent.  Therefore, host-specific dimensions (e.g.
	// `host`, `AWSUniqueId`, etc) are automatically added to every datapoint
	// that is emitted from the agent by default.  Set this to true if you are
	// using the agent primarily to monitor things on other hosts.  You can set
	// this option at the monitor level as well.
	DisableHostDimensions bool `yaml:"disableHostDimensions" default:"false"`
	// How often to send metrics to SignalFx.  Monitors can override this
	// individually.
	IntervalSeconds int `yaml:"intervalSeconds" default:"10"`
	// Dimensions (key:value pairs) that will be added to every datapoint emitted by the agent.
	// To specify that all metrics should be high-resolution, add the dimension `sf_hires:1`
	GlobalDimensions map[string]string `yaml:"globalDimensions" default:"{}"`
	// Whether to send the machine-id dimension on all host-specific datapoints
	// generated by the agent.  This dimension is derived from the Linux
	// machine-id value.
	SendMachineID bool `yaml:"sendMachineID"`
	// A list of observers to use (see observer config)
	Observers []ObserverConfig `yaml:"observers" default:"[]" neverLog:"omit"`
	// A list of monitors to use (see monitor config)
	Monitors []MonitorConfig `yaml:"monitors" default:"[]" neverLog:"omit"`
	// Configuration of the datapoint/event writer
	Writer WriterConfig `yaml:"writer"`
	// Log configuration
	Logging LogConfig `yaml:"logging" default:"{}"`
	// Configuration of the managed collectd subprocess
	Collectd CollectdConfig `yaml:"collectd" default:"{}"`
	// A list of metric filters
	MetricsToExclude []MetricFilter `yaml:"metricsToExclude" default:"[]"`
	// A list of properties filters
	PropertiesToExclude []PropertyFilterConfig `yaml:"propertiesToExclude" default:"[]"`
	// (**NOT FUNCTIONAL**) Whether to enable the Python sub-agent ("neopy")
	// that can directly use DataDog and Collectd Python plugins.  This is not
	// the same as Collectd's Python plugin, which is always enabled.
	PythonEnabled bool `yaml:"pythonEnabled" default:"false"`

	DiagnosticsServerPath string `yaml:"-"`
	// The path where the agent will create a named pipe and serve diagnostic output (windows only)
	DiagnosticsServerNamedPipePath string `yaml:"diagnosticsNamedPipePath" default:"\\\\.\\pipe\\signalfx-agent-diagnostics" copyTo:"DiagnosticsServerPath,GOOS=windows"`
	// The path where the agent will create UNIX socket and serve diagnostic output (linux only)
	DiagnosticsSocketPath string `yaml:"diagnosticsSocketPath" default:"/var/run/signalfx-agent/diagnostics.sock" copyTo:"DiagnosticsServerPath,GOOS=!windows"`

	InternalMetricsServerPath string `yaml:"-"`
	// The path where the agent will create a named pipe that serves internal
	// metrics (used by the internal-metrics monitor) (windows only)
	InternalMetricsServerNamedPipePath string `` /* 138-byte string literal not displayed */
	// The path where the agent will create a socket that serves internal
	// metrics (used by the internal-metrics monitor) (linux only)
	InternalMetricsSocketPath string `` /* 137-byte string literal not displayed */

	// Enables Go pprof endpoint on port 6060 that serves profiling data for
	// development
	EnableProfiling bool `yaml:"profiling" default:"false"`
	// Path to the directory holding the agent dependencies.  This will
	// normally be derived automatically. Overrides the envvar
	// SIGNALFX_BUNDLE_DIR if set.
	BundleDir string `yaml:"bundleDir"`
	// This exists purely to give the user a place to put common yaml values to
	// reference in other parts of the config file.
	Scratch interface{} `yaml:"scratch" neverLog:"omit"`
	// Configuration of remote config stores
	Sources sources.SourceConfig `yaml:"configSources"`
}

Config is the top level config struct for configurations that are common to all platoforms

type CustomConfigurable 

type CustomConfigurable interface {
	ExtraConfig() map[string]interface{}
}

CustomConfigurable should be implemented by config structs that have the concept of generic other config that is initially deserialized into a map[string]interface{} to be later transformed to another form.

type LogConfig 

type LogConfig struct {
	// Valid levels include `debug`, `info`, `warn`, `error`.  Note that
	// `debug` logging may leak sensitive configuration (e.g. passwords) to the
	// agent output.
	Level string `yaml:"level" default:"info"`
}

LogConfig contains configuration related to logging

func (*LogConfig) LogrusLevel 

func (lc *LogConfig) LogrusLevel() *log.Level

LogrusLevel returns a logrus log level based on the configured level in LogConfig.

type MetricFilter 

type MetricFilter struct {
	// A map of dimension key/values to match against.  All key/values must
	// match a datapoint for it to be matched.
	Dimensions map[string]string `yaml:"dimensions" default:"{}"`
	// A list of metric names to match against, OR'd together
	MetricNames []string `yaml:"metricNames"`
	// A single metric name to match against
	MetricName string `yaml:"metricName"`
	// (**Only applicable for the top level filters**) Limits this scope of the
	// filter to datapoints from a specific monitor. If specified, any
	// datapoints not from this monitor type will never match against this
	// filter.
	MonitorType string `yaml:"monitorType"`
	// Negates the result of the match so that it matches all datapoints that
	// do NOT match the metric name and dimension values given. This does not
	// negate monitorType, if given.
	Negated bool `yaml:"negated"`
}

MetricFilter describes a set of subtractive filters applied to datapoints right before they are sent.

func (*MetricFilter) MakeFilter 

func (mf *MetricFilter) MakeFilter() (dpfilters.DatapointFilter, error)

MakeFilter returns an actual filter instance from the config

type MonitorConfig 

type MonitorConfig struct {
	// The type of the monitor
	Type string `yaml:"type" json:"type"`
	// The rule used to match up this configuration with a discovered endpoint.
	// If blank, the configuration will be run immediately when the agent is
	// started.  If multiple endpoints match this rule, multiple instances of
	// the monitor type will be created with the same configuration (except
	// different host/port).
	DiscoveryRule string `yaml:"discoveryRule" json:"discoveryRule"`
	// A set of extra dimensions (key:value pairs) to include on datapoints emitted by the
	// monitor(s) created from this configuration. To specify metrics from this
	// monitor should be high-resolution, add the dimension `sf_hires:1`
	ExtraDimensions map[string]string `yaml:"extraDimensions" json:"extraDimensions"`
	// The interval (in seconds) at which to emit datapoints from the
	// monitor(s) created by this configuration.  If not set (or set to 0), the
	// global agent intervalSeconds config option will be used instead.
	IntervalSeconds int `yaml:"intervalSeconds" json:"intervalSeconds"`
	// If one or more configurations have this set to true, only those
	// configurations will be considered -- useful for testing
	Solo bool `yaml:"solo" json:"solo"`
	// A list of metric filters
	MetricsToExclude []MetricFilter `yaml:"metricsToExclude" json:"metricsToExclude" default:"[]"`
	// Some monitors pull metrics from services not running on the same host
	// and should not get the host-specific dimensions set on them (e.g.
	// `host`, `AWSUniqueId`, etc).  Setting this to `true` causes those
	// dimensions to be omitted.  You can disable this globally with the
	// `disableHostDimensions` option on the top level of the config.
	DisableHostDimensions bool `yaml:"disableHostDimensions" json:"disableHostDimensions" default:"false"`
	// This can be set to true if you don't want to include the dimensions that
	// are specific to the endpoint that was discovered by an observer.  This
	// is useful when you have an endpoint whose identity is not particularly
	// important since it acts largely as a proxy or adapter for other metrics.
	DisableEndpointDimensions bool `yaml:"disableEndpointDimensions" json:"disableEndpointDimensions"`
	// OtherConfig is everything else that is custom to a particular monitor
	OtherConfig map[string]interface{} `yaml:",inline" neverLog:"omit"`
	// ValidationError is where a message concerning validation issues can go
	// so that diagnostics can output it.
	Hostname        string               `yaml:"-" json:"-"`
	BundleDir       string               `yaml:"-" json:"-"`
	ValidationError string               `yaml:"-" json:"-" hash:"ignore"`
	MonitorID       types.MonitorID      `yaml:"-" hash:"ignore"`
	Filter          *dpfilters.FilterSet `yaml:"-" json:"-" hash:"ignore"`
}

MonitorConfig is used to configure monitor instances. One instance of MonitorConfig may be used to configure multiple monitor instances. If a monitor's discovery rule does not match any discovered services, the monitor will not run.

func (*MonitorConfig) Equals 

func (mc *MonitorConfig) Equals(other *MonitorConfig) bool

Equals tests if two monitor configs are sufficiently equal to each other. Two monitors should only be equal if it doesn't make sense for two configurations to be active at the same time.

func (*MonitorConfig) ExtraConfig 

func (mc *MonitorConfig) ExtraConfig() map[string]interface{}

ExtraConfig returns generic config as a map

func (*MonitorConfig) HasAutoDiscovery 

func (mc *MonitorConfig) HasAutoDiscovery() bool

HasAutoDiscovery returns whether the monitor is static (i.e. doesn't rely on autodiscovered services and is manually configured) or dynamic.

func (*MonitorConfig) Hash 

func (mc *MonitorConfig) Hash() uint64

Hash calculates a unique hash value for this config struct

func (*MonitorConfig) MonitorConfigCore 

func (mc *MonitorConfig) MonitorConfigCore() *MonitorConfig

MonitorConfigCore provides a way of getting the MonitorConfig when embedded in a struct that is referenced through a more generic interface.

type MonitorCustomConfig 

type MonitorCustomConfig interface {
	MonitorConfigCore() *MonitorConfig
}

MonitorCustomConfig represents monitor-specific configuration that doesn't appear in the MonitorConfig struct.

type ObserverConfig 

type ObserverConfig struct {
	// The type of the observer
	Type        string                 `yaml:"type,omitempty"`
	OtherConfig map[string]interface{} `yaml:",inline" default:"{}"`
}

ObserverConfig holds the configuration for an observer

func (*ObserverConfig) ExtraConfig 

func (oc *ObserverConfig) ExtraConfig() map[string]interface{}

ExtraConfig returns generic config as a map

type PropertyFilterConfig 

type PropertyFilterConfig struct {
	// A single property name to match
	PropertyName *string `yaml:"propertyName" default:"*"`
	// A property value to match
	PropertyValue *string `yaml:"propertyValue" default:"*"`
	// A dimension name to match
	DimensionName *string `yaml:"dimensionName" default:"*"`
	// A dimension value to match
	DimensionValue *string `yaml:"dimensionValue" default:"*"`
}

PropertyFilterConfig describes a set of subtractive filters applied to properties used to create a PropertyFilter

func (*PropertyFilterConfig) MakePropertyFilter 

func (pfc *PropertyFilterConfig) MakePropertyFilter() (propfilters.DimPropsFilter, error)

MakePropertyFilter returns an actual filter instance from the config

type StoreConfig 

type StoreConfig struct {
	OtherConfig map[string]interface{} `yaml:",inline,omitempty" default:"{}"`
}

StoreConfig holds configuration related to config stores (e.g. filesystem, zookeeper, etc)

func (*StoreConfig) ExtraConfig 

func (sc *StoreConfig) ExtraConfig() map[string]interface{}

ExtraConfig returns generic config as a map

type Validatable 

type Validatable interface {
	Validate() error
}

Validatable should be implemented by config structs that want to provide validation when the config is loaded.

type WriterConfig 

type WriterConfig struct {
	// The maximum number of datapoints to include in a batch before sending the
	// batch to the ingest server.  Smaller batch sizes than this will be sent
	// if datapoints originate in smaller chunks.
	DatapointMaxBatchSize int `yaml:"datapointMaxBatchSize" default:"1000"`
	// The analogue of `datapointMaxBatchSize` for trace spans.
	TraceSpanMaxBatchSize int `yaml:"traceSpanMaxBatchSize" default:"1000"`
	// Deprecated: use `maxRequests` instead.
	DatapointMaxRequests int `yaml:"datapointMaxRequests"`
	// The maximum number of concurrent requests to make to a single ingest server
	// with datapoints/events/trace spans.  This number multipled by
	// `datapointMaxBatchSize` is more or less the maximum number of datapoints
	// that can be "in-flight" at any given time.
	MaxRequests int `yaml:"maxRequests" default:"10"`
	// The agent does not send events immediately upon a monitor generating
	// them, but buffers them and sends them in batches.  The lower this
	// number, the less delay for events to appear in SignalFx.
	EventSendIntervalSeconds int `yaml:"eventSendIntervalSeconds" default:"1"`
	// The analogue of `maxRequests` for dimension property requests.
	PropertiesMaxRequests uint `yaml:"propertiesMaxRequests" default:"10"`
	// Properties that are synced to SignalFx are cached to prevent duplicate
	// requests from being sent, causing unnecessary load on our backend.
	PropertiesHistorySize uint `yaml:"propertiesHistorySize" default:"1000"`
	// If the log level is set to `debug` and this is true, all datapoints
	// generated by the agent will be logged.
	LogDatapoints bool `yaml:"logDatapoints"`
	// The analogue of `logDatapoints` for events.
	LogEvents bool `yaml:"logEvents"`
	// The analogue of `logDatapoints` for trace spans.
	LogTraceSpans bool `yaml:"logTraceSpans"`
	// Whether to send host correlation metrics to correlation traced services
	// with the underlying host
	SendTraceHostCorrelationMetrics *bool `yaml:"sendTraceHostCorrelationMetrics" default:"false"`
	// How long to wait after a trace span's service name is last seen to
	// continue sending the correlation datapoints for that service.  This
	// should be a duration string that is accepted by
	// https://golang.org/pkg/time/#ParseDuration.  This option is irrelvant if
	// `sendTraceHostCorrelationMetrics` is false.
	StaleServiceTimeout time.Duration `yaml:"staleServiceTimeout" default:"5m"`
	// How frequently to send host correlation metrics that are generated from
	// the service name seen in trace spans sent through or by the agent.  This
	// should be a duration string that is accepted by
	// https://golang.org/pkg/time/#ParseDuration.  This option is irrelvant if
	// `sendTraceHostCorrelationMetrics` is false.
	TraceHostCorrelationMetricsInterval time.Duration `yaml:"traceHostCorrelationMetricsInterval" default:"1m"`
	// The following are propagated from elsewhere
	HostIDDims          map[string]string      `yaml:"-"`
	IngestURL           *url.URL               `yaml:"-"`
	APIURL              *url.URL               `yaml:"-"`
	TraceEndpointURL    *url.URL               `yaml:"-"`
	SignalFxAccessToken string                 `yaml:"-"`
	GlobalDimensions    map[string]string      `yaml:"-"`
	DatapointFilter     *dpfilters.FilterSet   `yaml:"-"`
	PropertyFilter      *propfilters.FilterSet `yaml:"-"`
}

WriterConfig holds configuration for the datapoint writer.

func (*WriterConfig) Hash 

func (wc *WriterConfig) Hash() uint64

Hash calculates a unique hash value for this config struct

Source Files

View all Source files

Directories

Path Synopsis
Package sources contains all of the config source logic.
 Package sources contains all of the config source logic.
consul
env
etcd2
file
types
zookeeper
Package zookeeper contains the logic for using Zookeeper as a config source.
 Package zookeeper contains the logic for using Zookeeper as a config source.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.