instance

package
v0.14.0-rc.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 7, 2021 License: Apache-2.0 Imports: 36 Imported by: 0

Documentation

Overview

Package instance provides a mini Prometheus scraper and remote_writer.

Index

Constants

This section is empty.

Variables

View Source 
var (

	// DefaultBasicManagerConfig is the default config for the BasicManager.
	DefaultBasicManagerConfig = BasicManagerConfig{
		InstanceRestartBackoff: 5 * time.Second,
	}
)
View Source 
var (
	DefaultConfig = Config{
		HostFilter:           false,
		WALTruncateFrequency: 60 * time.Minute,
		MinWALTime:           5 * time.Minute,
		MaxWALTime:           4 * time.Hour,
		RemoteFlushDeadline:  1 * time.Minute,
		WriteStaleOnShutdown: false,
	}
)

Default configuration values

View Source 
var DefaultGlobalConfig = GlobalConfig{
	Prometheus: config.DefaultGlobalConfig,
}

DefaultGlobalConfig holds default global settings to be used across all instances.

View Source 
var ErrNotReady = errors.New("Scrape manager not ready")

ErrNotReady is returned when the scrape manager is used but has not been initialized yet.

View Source 
var HostFilterLabelMatchers = []string{

	"__meta_consul_node",

	"__meta_dockerswarm_node_id",
	"__meta_dockerswarm_node_hostname",
	"__meta_dockerswarm_node_address",

	"__meta_kubernetes_pod_node_name",
	"__meta_kubernetes_node_name",

	"__host__",
}

HostFilterLabelMatchers are the set of labels that will be used to match against an incoming target.

Functions

func Hostname added in v0.4.0 

func Hostname() (string, error)

Hostname retrieves the hostname identifying the machine the process is running on. It will return the value of $HOSTNAME, if defined, and fall back to Go's os.Hostname.

func MarshalConfig 

func MarshalConfig(c *Config, scrubSecrets bool) ([]byte, error)

MarshalConfig marshals an instance config based on a provided content type.

func MarshalConfigToWriter 

func MarshalConfigToWriter(c *Config, w io.Writer, scrubSecrets bool) error

MarshalConfigToWriter marshals a config to an io.Writer.

Types

type BasicManager added in v0.5.0 

type BasicManager struct {
	// contains filtered or unexported fields
}

BasicManager creates a new BasicManager, implementing the Manager interface. BasicManager will directly launch instances and perform no extra processing.

Other implementations of Manager usually wrap a BasicManager.

func NewBasicManager added in v0.5.0 

func NewBasicManager(cfg BasicManagerConfig, logger log.Logger, launch Factory) *BasicManager

NewBasicManager creates a new BasicManager. The launch function will be invoked any time a new Config is applied.

The lifecycle of any ManagedInstance returned by the launch function will be handled by the BasicManager. Instances will be automatically restarted if stopped, updated if the config changes, or removed when the Config is deleted.

func (*BasicManager) ApplyConfig added in v0.5.0 

func (m *BasicManager) ApplyConfig(c Config) error

ApplyConfig takes a Config and either starts a new managed instance or updates an existing managed instance. The value for Name in c is used to uniquely identify the Config and determine whether the Config has an existing associated managed instance.

func (*BasicManager) DeleteConfig added in v0.5.0 

func (m *BasicManager) DeleteConfig(name string) error

DeleteConfig removes a managed instance by its config name. Returns an error if there is no such managed instance with the given name.

func (*BasicManager) ListConfigs added in v0.5.0 

func (m *BasicManager) ListConfigs() map[string]Config

ListConfigs lists the current active configs managed by BasicManager.

func (*BasicManager) ListInstances added in v0.5.0 

func (m *BasicManager) ListInstances() map[string]ManagedInstance

ListInstances returns the current active instances managed by BasicManager.

func (*BasicManager) Stop added in v0.5.0 

func (m *BasicManager) Stop()

Stop stops the BasicManager and stops all active processes for configs.

func (*BasicManager) UpdateManagerConfig added in v0.14.0 

func (m *BasicManager) UpdateManagerConfig(c BasicManagerConfig)

UpdateManagerConfig updates the BasicManagerConfig.

type BasicManagerConfig added in v0.5.0 

type BasicManagerConfig struct {
	InstanceRestartBackoff time.Duration
}

BasicManagerConfig controls the operations of a BasicManager.

type Config 

type Config struct {
	Name                     string                      `yaml:"name,omitempty"`
	HostFilter               bool                        `yaml:"host_filter,omitempty"`
	HostFilterRelabelConfigs []*relabel.Config           `yaml:"host_filter_relabel_configs,omitempty"`
	ScrapeConfigs            []*config.ScrapeConfig      `yaml:"scrape_configs,omitempty"`
	RemoteWrite              []*config.RemoteWriteConfig `yaml:"remote_write,omitempty"`

	// How frequently the WAL should be truncated.
	WALTruncateFrequency time.Duration `yaml:"wal_truncate_frequency,omitempty"`

	// Minimum and maximum time series should exist in the WAL for.
	MinWALTime time.Duration `yaml:"min_wal_time,omitempty"`
	MaxWALTime time.Duration `yaml:"max_wal_time,omitempty"`

	RemoteFlushDeadline  time.Duration `yaml:"remote_flush_deadline,omitempty"`
	WriteStaleOnShutdown bool          `yaml:"write_stale_on_shutdown,omitempty"`
}

Config is a specific agent that runs within the overall Prometheus agent. It has its own set of scrape_configs and remote_write rules.

func UnmarshalConfig 

func UnmarshalConfig(r io.Reader) (*Config, error)

UnmarshalConfig unmarshals an instance config from a reader based on a provided content type.

func (*Config) ApplyDefaults 

func (c *Config) ApplyDefaults(global *GlobalConfig) error

ApplyDefaults applies default configurations to the configuration to all values that have not been changed to their non-zero value. ApplyDefaults also validates the config.

func (Config) MarshalYAML 

func (c Config) MarshalYAML() (interface{}, error)

MarshalYAML implements yaml.Marshaler.

func (*Config) UnmarshalYAML 

func (c *Config) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML implements yaml.Unmarshaler.

type DiscoveredGroups 

type DiscoveredGroups = map[string][]*targetgroup.Group

DiscoveredGroups is a set of groups found via service discovery.

func FilterGroups 

func FilterGroups(in DiscoveredGroups, host string, configs []*relabel.Config) DiscoveredGroups

FilterGroups takes a set of DiscoveredGroups as input and filters out any Target that is not running on the host machine provided by host.

This is done by looking at HostFilterLabelMatchers and __address__.

If the discovered address is localhost or 127.0.0.1, the group is never filtered out.

type ErrInvalidUpdate added in v0.5.0 

type ErrInvalidUpdate struct {
	Inner error
}

ErrInvalidUpdate is returned whenever Update is called against an instance but an invalid field is changed between configs. If ErrInvalidUpdate is returned, the instance must be fully stopped and replaced with a new one with the new config.

func (ErrInvalidUpdate) As added in v0.5.0 

func (e ErrInvalidUpdate) As(err interface{}) bool

As will set the err object to ErrInvalidUpdate provided err is a pointer to ErrInvalidUpdate.

func (ErrInvalidUpdate) Error added in v0.5.0 

func (e ErrInvalidUpdate) Error() string

Error implements the error interface.

func (ErrInvalidUpdate) Is added in v0.5.0 

func (e ErrInvalidUpdate) Is(err error) bool

Is returns true if err is an ErrInvalidUpdate.

type Factory added in v0.5.0 

type Factory func(c Config) (ManagedInstance, error)

Factory should return an unstarted instance given some config.

type GlobalConfig added in v0.14.0 

type GlobalConfig struct {
	Prometheus  config.GlobalConfig         `yaml:",inline"`
	RemoteWrite []*config.RemoteWriteConfig `yaml:"remote_write,omitempty"`
}

GlobalConfig holds global settings that apply to all instances by default.

func (*GlobalConfig) UnmarshalYAML added in v0.14.0 

func (c *GlobalConfig) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML implements yaml.Unmarshaler.

type GroupChannel 

type GroupChannel = <-chan DiscoveredGroups

GroupChannel is a channel that provides discovered target groups.

type GroupManager added in v0.5.0 

type GroupManager struct {
	// contains filtered or unexported fields
}

A GroupManager wraps around another Manager and groups all incoming Configs into a smaller set of configs, causing less managed instances to be spawned.

Configs are grouped by all settings for a Config *except* scrape configs. Any difference found in any flag will cause a Config to be placed in another group. One exception to this rule is that remote_writes are compared unordered, but the sets of remote_writes should otherwise be identical.

GroupManagers drastically improve the performance of the Agent when a significant number of instances are spawned, as the overhead of each instance having its own service discovery, WAL, and remote_write can be significant.

The config names of instances within the group will be represented by that group's hash of settings.

func NewGroupManager added in v0.5.0 

func NewGroupManager(inner Manager) *GroupManager

NewGroupManager creates a new GroupManager for combining instances of the same "group."

func (*GroupManager) ApplyConfig added in v0.5.0 

func (m *GroupManager) ApplyConfig(c Config) error

ApplyConfig will determine the group of the Config before applying it to the group. If no group exists, one will be created. If a group already exists, the group will have its settings merged with the Config and will be updated.

func (*GroupManager) DeleteConfig added in v0.5.0 

func (m *GroupManager) DeleteConfig(name string) error

DeleteConfig will remove a Config from its associated group. If there are no more Configs within that group after this Config is deleted, the managed instance will be stopped. Otherwise, the managed instance will be updated with the new grouped Config that doesn't include the removed one.

func (*GroupManager) ListConfigs added in v0.5.0 

func (m *GroupManager) ListConfigs() map[string]Config

ListConfigs returns the UNGROUPED instance configs with their original settings. To see the grouped instances, call ListInstances instead.

func (*GroupManager) ListInstances added in v0.5.0 

func (m *GroupManager) ListInstances() map[string]ManagedInstance

ListInstances returns all currently grouped managed instances. The key will be the group's hash of shared settings.

func (*GroupManager) Stop added in v0.5.0 

func (m *GroupManager) Stop()

Stop stops the Manager and all of its managed instances.

type HostFilter 

type HostFilter struct {
	// contains filtered or unexported fields
}

HostFilter acts as a MITM between the discovery manager and the scrape manager, filtering out discovered targets that are not running on the same node as the agent itself.

func NewHostFilter 

func NewHostFilter(host string, relabels []*relabel.Config) *HostFilter

NewHostFilter creates a new HostFilter.

func (*HostFilter) Run 

func (f *HostFilter) Run(syncCh GroupChannel)

Run starts the HostFilter. It only exits when the HostFilter is stopped. Run will continually read from syncCh and filter groups discovered down to targets that are colocated on the same node as the one the HostFilter is running in.

func (*HostFilter) Stop 

func (f *HostFilter) Stop()

Stop stops the host filter from processing more target updates.

func (*HostFilter) SyncCh 

func (f *HostFilter) SyncCh() GroupChannel

SyncCh returns a read only channel used by all the clients to receive target updates.

type Instance 

type Instance struct {
	// contains filtered or unexported fields
}

Instance is an individual metrics collector and remote_writer.

func New 

func New(reg prometheus.Registerer, globalCfg GlobalConfig, cfg Config, walDir string, logger log.Logger) (*Instance, error)

New creates a new Instance with a directory for storing the WAL. The instance will not start until Run is called on the instance.

func (*Instance) Run 

func (i *Instance) Run(ctx context.Context) error

Run starts the instance, initializing Prometheus components, and will continue to run until an error happens during execution or the provided context is cancelled.

Run may be re-called after exiting, as components will be reinitialized each time Run is called.

func (*Instance) StorageDirectory added in v0.10.0 

func (i *Instance) StorageDirectory() string

StorageDirectory returns the directory where this Instance is writing series and samples to for the WAL.

func (*Instance) TargetsActive added in v0.5.0 

func (i *Instance) TargetsActive() map[string][]*scrape.Target

TargetsActive returns the set of active targets from the scrape manager. Returns nil if the scrape manager is not ready yet.

func (*Instance) Update added in v0.5.0 

func (i *Instance) Update(c Config) (err error)

Update accepts a new Config for the Instance and will dynamically update any running Prometheus components with the new values from Config. Update will return an ErrInvalidUpdate if the Update could not be applied.

type ManagedInstance added in v0.5.0 

type ManagedInstance interface {
	Run(ctx context.Context) error
	Update(c Config) error
	TargetsActive() map[string][]*scrape.Target
	StorageDirectory() string
}

ManagedInstance is implemented by Instance. It is defined as an interface for the sake of testing from Manager implementations.

type Manager added in v0.5.0 

type Manager interface {
	// ListInstances returns all currently managed instances running
	// within the Manager. The key will be the instance name from their config.
	ListInstances() map[string]ManagedInstance

	// ListConfigs returns the config objects associated with a managed
	// instance. The key will be the Name field from Config.
	ListConfigs() map[string]Config

	// ApplyConfig creates a new Config or updates an existing Config if
	// one with Config.Name already exists.
	ApplyConfig(Config) error

	// DeleteConfig deletes a given managed instance based on its Config.Name.
	DeleteConfig(name string) error

	// Stop stops the Manager and all managed instances.
	Stop()
}

Manager represents a set of methods for manipulating running instances at runtime.

type MetricValueCollector 

type MetricValueCollector struct {
	// contains filtered or unexported fields
}

MetricValueCollector wraps around a Gatherer and provides utilities for pulling metric values from a given metric name and label matchers.

This is used by the agent instances to find the most recent timestamp successfully remote_written to for purposes of safely truncating the WAL.

MetricValueCollector is only intended for use with Gauges and Counters.

func NewMetricValueCollector 

func NewMetricValueCollector(g prometheus.Gatherer, match string) *MetricValueCollector

NewMetricValueCollector creates a new MetricValueCollector.

func (*MetricValueCollector) GetValues 

func (vc *MetricValueCollector) GetValues(label string, labelValues ...string) ([]float64, error)

GetValues looks through all the tracked metrics and returns all values for metrics that match some key value pair.

type MockManager added in v0.5.0 

type MockManager struct {
	ListInstancesFunc func() map[string]ManagedInstance
	ListConfigsFunc   func() map[string]Config
	ApplyConfigFunc   func(Config) error
	DeleteConfigFunc  func(name string) error
	StopFunc          func()
}

MockManager exposes methods of the Manager interface as struct fields. Useful for tests.

func (MockManager) ApplyConfig added in v0.5.0 

func (m MockManager) ApplyConfig(c Config) error

ApplyConfig implements Manager.

func (MockManager) DeleteConfig added in v0.5.0 

func (m MockManager) DeleteConfig(name string) error

DeleteConfig implements Manager.

func (MockManager) ListConfigs added in v0.5.0 

func (m MockManager) ListConfigs() map[string]Config

ListConfigs implements Manager.

func (MockManager) ListInstances added in v0.5.0 

func (m MockManager) ListInstances() map[string]ManagedInstance

ListInstances implements Manager.

func (MockManager) Stop added in v0.5.0 

func (m MockManager) Stop()

Stop implements Manager.

type ModalManager added in v0.14.0 

type ModalManager struct {
	// contains filtered or unexported fields
}

ModalManager runs instances by either grouping them or running them fully separately.

func NewModalManager added in v0.14.0 

func NewModalManager(reg prometheus.Registerer, l log.Logger, next Manager, mode Mode) (*ModalManager, error)

NewModalManager creates a new ModalManager.

func (*ModalManager) ApplyConfig added in v0.14.0 

func (m *ModalManager) ApplyConfig(c Config) error

ApplyConfig implements Manager.

func (*ModalManager) DeleteConfig added in v0.14.0 

func (m *ModalManager) DeleteConfig(name string) error

DeleteConfig implements Manager.

func (*ModalManager) ListConfigs added in v0.14.0 

func (m *ModalManager) ListConfigs() map[string]Config

ListConfigs implements Manager.

func (*ModalManager) ListInstances added in v0.14.0 

func (m *ModalManager) ListInstances() map[string]ManagedInstance

ListInstances implements Manager.

func (*ModalManager) SetMode added in v0.14.0 

func (m *ModalManager) SetMode(newMode Mode) error

SetMode updates the mode ModalManager is running in. Changing the mode is an expensive operation; all underlying configs must be stopped and then reapplied.

func (*ModalManager) Stop added in v0.14.0 

func (m *ModalManager) Stop()

Stop implements Manager.

type Mode added in v0.14.0 

type Mode string

Mode controls how instances are created. 

var (
	ModeDistinct Mode = "distinct"
	ModeShared   Mode = "shared"

	DefaultMode = ModeShared
)

Types of instance modes

func (*Mode) UnmarshalYAML added in v0.14.0 

func (m *Mode) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML unmarshals a string to a Mode. Fails if the string is unrecognized.

type NoOpInstance added in v0.5.0 

type NoOpInstance struct{}

NoOpInstance implements the Instance interface in pkg/prom but does not do anything. Useful for tests.

func (NoOpInstance) Run added in v0.5.0 

func (NoOpInstance) Run(ctx context.Context) error

Run implements Instance.

func (NoOpInstance) StorageDirectory added in v0.10.0 

func (NoOpInstance) StorageDirectory() string

StorageDirectory implements Instance.

func (NoOpInstance) TargetsActive added in v0.5.0 

func (NoOpInstance) TargetsActive() map[string][]*scrape.Target

TargetsActive implements Instance.

func (NoOpInstance) Update added in v0.5.0 

func (NoOpInstance) Update(_ Config) error

Update implements Instance.

Source Files

View all Source files

Directories

Path Synopsis
Package configstore abstracts the concepts of where instance files get retrieved.
 Package configstore abstracts the concepts of where instance files get retrieved.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.