instance

package
v0.13.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 25, 2021 License: Apache-2.0 Imports: 40 Imported by: 0

Documentation

Overview

Package instance provides a mini Prometheus scraper and remote_writer.

Index

Constants

This section is empty.

Variables

View Source 
var (

	// DefaultBasicManagerConfig is the default config for the BasicManager.
	DefaultBasicManagerConfig = BasicManagerConfig{
		InstanceRestartBackoff: 5 * time.Second,
	}
)
View Source 
var (
	DefaultConfig = Config{
		HostFilter:           false,
		WALTruncateFrequency: 60 * time.Minute,
		MinWALTime:           5 * time.Minute,
		MaxWALTime:           4 * time.Hour,
		RemoteFlushDeadline:  1 * time.Minute,
		WriteStaleOnShutdown: false,
	}
)

Default configuration values

View Source 
var ErrNotReady = errors.New("Scrape manager not ready")
View Source 
var HostFilterLabelMatchers = []string{

	"__meta_consul_node",

	"__meta_dockerswarm_node_id",
	"__meta_dockerswarm_node_hostname",
	"__meta_dockerswarm_node_address",

	"__meta_kubernetes_pod_node_name",
	"__meta_kubernetes_node_name",

	"__host__",
}

HostFilterLabelMatchers are the set of labels that will be used to match against an incoming target.

Functions

func Hostname added in v0.4.0 

func Hostname() (string, error)

Hostname retrieves the hostname identifying the machine the process is running on. It will return the value of $HOSTNAME, if defined, and fall back to Go's os.Hostname.

func MarshalConfig 

func MarshalConfig(c *Config, scrubSecrets bool) ([]byte, error)

MarshalConfig marshals an instance config based on a provided content type.

func MarshalConfigToWriter 

func MarshalConfigToWriter(c *Config, w io.Writer, scrubSecrets bool) error

MarshalConfigToWriter marshals a config to an io.Writer.

func NewSigV4RoundTripper added in v0.10.0 

func NewSigV4RoundTripper(cfg SigV4Config, next http.RoundTripper) (http.RoundTripper, error)

NewSigV4RoundTripper returns a new http.RoundTripper that will sign requests using Amazon's Signature Verification V4 signing procedure. The request will then be handed off to the next RoundTripper provided by next. If next is nil, http.DefaultTransport will be used.

Credentials for signing are retrieving used the default AWS credential chain. If credentials could not be found, an error will be returned.

Types

type BasicManager added in v0.5.0 

type BasicManager struct {
	// contains filtered or unexported fields
}

BasicManager creates a new BasicManager, implementing the Manager interface. BasicManager will directly launch instances and perform no extra processing.

Other implementations of Manager usually wrap a BasicManager.

func NewBasicManager added in v0.5.0 

func NewBasicManager(cfg BasicManagerConfig, logger log.Logger, launch Factory, validate ConfigValidator) *BasicManager

NewBasicManager creates a new BasicManager. The launch function will be invoked any time a new Config is applied.

The lifecycle of any ManagedInstance returned by the launch function will be handled by the BasicManager. Instances will be automatically restarted if stopped, updated if the config changes, or removed when the Config is deleted.

The validate function will be called before creating an instance. If the config is not valid, launch will not be called. validate is optional.

func (*BasicManager) ApplyConfig added in v0.5.0 

func (m *BasicManager) ApplyConfig(c Config) error

ApplyConfig takes a Config and either starts a new managed instance or updates an existing managed instance. The value for Name in c is used to uniquely identify the Config and determine whether the Config has an existing associated managed instance.

func (*BasicManager) DeleteConfig added in v0.5.0 

func (m *BasicManager) DeleteConfig(name string) error

DeleteConfig removes a managed instance by its config name. Returns an error if there is no such managed instance with the given name.

func (*BasicManager) ListConfigs added in v0.5.0 

func (m *BasicManager) ListConfigs() map[string]Config

ListConfigs lists the current active configs managed by BasicManager.

func (*BasicManager) ListInstances added in v0.5.0 

func (m *BasicManager) ListInstances() map[string]ManagedInstance

ListInstances returns the current active instances managed by BasicManager.

func (*BasicManager) Stop added in v0.5.0 

func (m *BasicManager) Stop()

Stop stops the BasicManager and stops all active processes for configs.

type BasicManagerConfig added in v0.5.0 

type BasicManagerConfig struct {
	InstanceRestartBackoff time.Duration
}

BasicManagerConfig controls the operations of a BasicManager.

type Config 

type Config struct {
	Name                     string                 `yaml:"name" json:"name"`
	HostFilter               bool                   `yaml:"host_filter" json:"host_filter"`
	HostFilterRelabelConfigs []*relabel.Config      `yaml:"host_filter_relabel_configs,omitempty"`
	ScrapeConfigs            []*config.ScrapeConfig `yaml:"scrape_configs,omitempty" json:"scrape_configs,omitempty"`
	RemoteWrite              []*RemoteWriteConfig   `yaml:"remote_write,omitempty" json:"remote_write,omitempty"`

	// How frequently the WAL should be truncated.
	WALTruncateFrequency time.Duration `yaml:"wal_truncate_frequency,omitempty" json:"wal_truncate_frequency,omitempty"`

	// Minimum and maximum time series should exist in the WAL for.
	MinWALTime time.Duration `yaml:"min_wal_time,omitempty" json:"min_wal_time,omitempty"`
	MaxWALTime time.Duration `yaml:"max_wal_time,omitempty" json:"max_wal_time,omitempty"`

	RemoteFlushDeadline  time.Duration `yaml:"remote_flush_deadline,omitempty" json:"remote_flush_deadline,omitempty"`
	WriteStaleOnShutdown bool          `yaml:"write_stale_on_shutdown,omitempty" json:"write_stale_on_shutdown,omitempty"`
}

Config is a specific agent that runs within the overall Prometheus agent. It has its own set of scrape_configs and remote_write rules.

func UnmarshalConfig 

func UnmarshalConfig(r io.Reader) (*Config, error)

UnmarshalConfig unmarshals an instance config from a reader based on a provided content type.

func (*Config) ApplyDefaults 

func (c *Config) ApplyDefaults(global *config.GlobalConfig) error

ApplyDefaults applies default configurations to the configuration to all values that have not been changed to their non-zero value. ApplyDefaults also validates the config.

func (*Config) BaseRemoteWrite added in v0.10.0 

func (c *Config) BaseRemoteWrite() []*config.RemoteWriteConfig

BaseRemoteWrite returns the base remote write configs without the added fields.

func (Config) MarshalYAML 

func (c Config) MarshalYAML() (interface{}, error)

func (*Config) UnmarshalYAML 

func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error

type ConfigValidator added in v0.5.0 

type ConfigValidator func(c *Config) error

A ConfigValidator should validate a Config and return an error if there is a problem. ConfigValidator may mutate the Config.

type CountingManager added in v0.6.0 

type CountingManager struct {
	// contains filtered or unexported fields
}

CountingManager counts how many unique configs pass through a Manager. It may be distinct from the set of instances depending on available Managers.

CountingManager implements Manager.

func NewCountingManager added in v0.6.0 

func NewCountingManager(reg prometheus.Registerer, inner Manager) *CountingManager

NewCountingManager creates a new CountingManager. The Manager provided by inner will be wrapped by CountingManager and will handle requests to apply configs.

func (*CountingManager) ApplyConfig added in v0.6.0 

func (cm *CountingManager) ApplyConfig(c Config) error

ApplyConfig implements Manager.

func (*CountingManager) DeleteConfig added in v0.6.0 

func (cm *CountingManager) DeleteConfig(name string) error

DeleteConfig implements Manager.

func (*CountingManager) ListConfigs added in v0.6.0 

func (cm *CountingManager) ListConfigs() map[string]Config

ListConfigs implements Manager.

func (*CountingManager) ListInstances added in v0.6.0 

func (cm *CountingManager) ListInstances() map[string]ManagedInstance

ListInstances implements Manager.

func (*CountingManager) Stop added in v0.6.0 

func (cm *CountingManager) Stop()

Stop implements Manager.

type DiscoveredGroups 

type DiscoveredGroups = map[string][]*targetgroup.Group

DiscoveredGroups is a set of groups found via service discovery.

func FilterGroups 

func FilterGroups(in DiscoveredGroups, host string, configs []*relabel.Config) DiscoveredGroups

FilterGroups takes a set of DiscoveredGroups as input and filters out any Target that is not running on the host machine provided by host.

This is done by looking at HostFilterLabelMatchers and __address__.

If the discovered address is localhost or 127.0.0.1, the group is never filtered out.

type ErrInvalidUpdate added in v0.5.0 

type ErrInvalidUpdate struct {
	Inner error
}

ErrInvalidUpdate is returned whenever Update is called against an instance but an invalid field is changed between configs. If ErrInvalidUpdate is returned, the instance must be fully stopped and replaced with a new one with the new config.

func (ErrInvalidUpdate) As added in v0.5.0 

func (e ErrInvalidUpdate) As(err interface{}) bool

As will set the err object to ErrInvalidUpdate provided err is a pointer to ErrInvalidUpdate.

func (ErrInvalidUpdate) Error added in v0.5.0 

func (e ErrInvalidUpdate) Error() string

Error implements the error interface.

func (ErrInvalidUpdate) Is added in v0.5.0 

func (e ErrInvalidUpdate) Is(err error) bool

Is returns true if err is an ErrInvalidUpdate.

type Factory added in v0.5.0 

type Factory func(c Config) (ManagedInstance, error)

Factory should return an unstarted instance given some config.

type GroupChannel 

type GroupChannel = <-chan DiscoveredGroups

GroupChannel is a channel that provides discovered target groups.

type GroupManager added in v0.5.0 

type GroupManager struct {
	// contains filtered or unexported fields
}

A GroupManager wraps around another Manager and groups all incoming Configs into a smaller set of configs, causing less managed instances to be spawned.

Configs are grouped by all settings for a Config *except* scrape configs. Any difference found in any flag will cause a Config to be placed in another group. One exception to this rule is that remote_writes are compared unordered, but the sets of remote_writes should otherwise be identical.

GroupManagers drastically improve the performance of the Agent when a significant number of instances are spawned, as the overhead of each instance having its own service discovery, WAL, and remote_write can be significant.

The config names of instances within the group will be represented by that group's hash of settings.

func NewGroupManager added in v0.5.0 

func NewGroupManager(inner Manager) *GroupManager

NewGroupManager creates a new GroupManager for combining instances of the same "group."

func (*GroupManager) ApplyConfig added in v0.5.0 

func (m *GroupManager) ApplyConfig(c Config) error

ApplyConfig will determine the group of the Config before applying it to the group. If no group exists, one will be created. If a group already exists, the group will have its settings merged with the Config and will be updated.

func (*GroupManager) DeleteConfig added in v0.5.0 

func (m *GroupManager) DeleteConfig(name string) error

DeleteConfig will remove a Config from its associated group. If there are no more Configs within that group after this Config is deleted, the managed instance will be stopped. Otherwise, the managed instance will be updated with the new grouped Config that doesn't include the removed one.

func (*GroupManager) ListConfigs added in v0.5.0 

func (m *GroupManager) ListConfigs() map[string]Config

ListConfigs returns the UNGROUPED instance configs with their original settings. To see the grouped instances, call ListInstances instead.

func (*GroupManager) ListInstances added in v0.5.0 

func (m *GroupManager) ListInstances() map[string]ManagedInstance

ListInstances returns all currently grouped managed instances. The key will be the group's hash of shared settings.

func (*GroupManager) Stop added in v0.5.0 

func (m *GroupManager) Stop()

Stop stops the Manager and all of its managed instances.

type HostFilter 

type HostFilter struct {
	// contains filtered or unexported fields
}

HostFilter acts as a MITM between the discovery manager and the scrape manager, filtering out discovered targets that are not running on the same node as the agent itself.

func NewHostFilter 

func NewHostFilter(host string, relabels []*relabel.Config) *HostFilter

NewHostFilter creates a new HostFilter

func (*HostFilter) Run 

func (f *HostFilter) Run(syncCh GroupChannel)

func (*HostFilter) Stop 

func (f *HostFilter) Stop()

Stop stops the host filter from processing more target updates.

func (*HostFilter) SyncCh 

func (f *HostFilter) SyncCh() GroupChannel

SyncCh returns a read only channel used by all the clients to receive target updates.

type Instance 

type Instance struct {
	// contains filtered or unexported fields
}

Instance is an individual metrics collector and remote_writer.

func New 

func New(reg prometheus.Registerer, globalCfg config.GlobalConfig, cfg Config, walDir string, logger log.Logger) (*Instance, error)

New creates a new Instance with a directory for storing the WAL. The instance will not start until Run is called on the instance.

func (*Instance) Run 

func (i *Instance) Run(ctx context.Context) error

Run starts the instance, initializing Prometheus components, and will continue to run until an error happens during execution or the provided context is cancelled.

Run may be re-called after exiting, as components will be reinitialized each time Run is called.

func (*Instance) StorageDirectory added in v0.10.0 

func (i *Instance) StorageDirectory() string

func (*Instance) TargetsActive added in v0.5.0 

func (i *Instance) TargetsActive() map[string][]*scrape.Target

TargetsActive returns the set of active targets from the scrape manager. Returns nil if the scrape manager is not ready yet.

func (*Instance) Update added in v0.5.0 

func (i *Instance) Update(c Config) (err error)

Update accepts a new Config for the Instance and will dynamically update any running Prometheus components with the new values from Config. Update will return an ErrInvalidUpdate if the Update could not be applied.

type ManagedInstance added in v0.5.0 

type ManagedInstance interface {
	Run(ctx context.Context) error
	Update(c Config) error
	TargetsActive() map[string][]*scrape.Target
	StorageDirectory() string
}

ManagedInstance is implemented by Instance. It is defined as an interface for the sake of testing from Manager implementations.

type Manager added in v0.5.0 

type Manager interface {
	// ListInstances returns all currently managed instances running
	// within the Manager. The key will be the instance name from their config.
	ListInstances() map[string]ManagedInstance

	// ListConfigs returns the config objects associated with a managed
	// instance. The key will be the Name field from Config.
	ListConfigs() map[string]Config

	// ApplyConfig creates a new Config or updates an existing Config if
	// one with Config.Name already exists.
	ApplyConfig(Config) error

	// DeleteConfig deletes a given managed instance based on its Config.Name.
	DeleteConfig(name string) error

	// Stop stops the Manager and all managed instances.
	Stop()
}

Manager represents a set of methods for manipulating running instances at runtime.

type MetricValueCollector 

type MetricValueCollector struct {
	// contains filtered or unexported fields
}

MetricValueCollector wraps around a Gatherer and provides utilities for pulling metric values from a given metric name and label matchers.

This is used by the agent instances to find the most recent timestamp successfully remote_written to for purposes of safely truncating the WAL.

MetricValueCollector is only intended for use with Gauges and Counters.

func NewMetricValueCollector 

func NewMetricValueCollector(g prometheus.Gatherer, match string) *MetricValueCollector

NewMetricValueCollector creates a new MetricValueCollector.

func (*MetricValueCollector) GetValues 

func (vc *MetricValueCollector) GetValues(label string, labelValues ...string) ([]float64, error)

GetValues looks through all the tracked metrics and returns all values for metrics that match some key value pair.

type MockManager added in v0.5.0 

type MockManager struct {
	ListInstancesFunc func() map[string]ManagedInstance
	ListConfigsFunc   func() map[string]Config
	ApplyConfigFunc   func(Config) error
	DeleteConfigFunc  func(name string) error
	StopFunc          func()
}

MockManager exposes methods of the Manager interface as struct fields. Useful for tests.

func (MockManager) ApplyConfig added in v0.5.0 

func (m MockManager) ApplyConfig(c Config) error

ApplyConfig implements Manager.

func (MockManager) DeleteConfig added in v0.5.0 

func (m MockManager) DeleteConfig(name string) error

DeleteConfig implements Manager.

func (MockManager) ListConfigs added in v0.5.0 

func (m MockManager) ListConfigs() map[string]Config

ListConfigs implements Manager.

func (MockManager) ListInstances added in v0.5.0 

func (m MockManager) ListInstances() map[string]ManagedInstance

ListInstances implements Manager.

func (MockManager) Stop added in v0.5.0 

func (m MockManager) Stop()

Stop implements Manager.

type NoOpInstance added in v0.5.0 

type NoOpInstance struct{}

NoOpInstance implements the Instance interface in pkg/prom but does not do anything. Useful for tests.

func (NoOpInstance) Run added in v0.5.0 

func (NoOpInstance) Run(ctx context.Context) error

func (NoOpInstance) StorageDirectory added in v0.10.0 

func (NoOpInstance) StorageDirectory() string

func (NoOpInstance) TargetsActive added in v0.5.0 

func (NoOpInstance) TargetsActive() map[string][]*scrape.Target

func (NoOpInstance) Update added in v0.5.0 

func (NoOpInstance) Update(_ Config) error

type RemoteWriteConfig added in v0.10.0 

type RemoteWriteConfig struct {
	Base config.RemoteWriteConfig `yaml:",inline"`

	SigV4 SigV4Config `yaml:"sigv4,omitempty"`
}

RemoteWriteConfig extends the default RemoteWriteConfig with extra settings.

func (*RemoteWriteConfig) UnmarshalYAML added in v0.10.0 

func (c *RemoteWriteConfig) UnmarshalYAML(unmarshal func(interface{}) error) error

func (*RemoteWriteConfig) Validate added in v0.10.0 

func (c *RemoteWriteConfig) Validate() error

Validate validates the HTTPClientConfig along with the SigV4Config to ensure only one authentication mechanism is used.

type SigV4Config added in v0.10.0 

type SigV4Config struct {
	Enabled bool   `yaml:"enabled,omitempty"`
	Region  string `yaml:"region,omitempty"`
}

SigV4Config configures signing requests with SigV4.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.