monitor

README

System Monitoring

System Monitoring means all statistical and diagnostic information made availabe to the user of InfluxDB system, about the system itself. Its purpose is to assist with troubleshooting and performance analysis.

Supported Commands

• SHOW STATS
• SHOW DIAGNOSTICS

If statistical information is also written to an InfluxDB system, the data will also be queryable by the InfluxQL query language.

Statistics vs. Diagnostics

A distinction between statistics and diagnostics is made for the purposes of monitoring. Generally a statistical quality is something that is being counted, and for which it makes sense to store for historical analysis. Diagnostic information is not necessarily numerical, and may not make sense to store.

An example of statistical information would be the number of points received over UDP, or the number of queries executed. Examples of diagnostic information would be a list of current Graphite TCP connections, the version of InfluxDB, or the uptime of the process.

Design and Implementation

A new module named monitor supports all statistics and diagnostic functionality. This includes:

• Allowing other modules to register statistics and diagnostics information, allowing it to be accessed on demand by the monitor module.
• Serving the statistics and diagnostic information to the user, in response to commands such as SHOW DIAGNOSTICS.
• Expose standard Go runtime information such as garbage collection statistics.
• Make all collected expvar data via HTTP, for collection by 3rd-party tools.
• Writing the statistical information to an InfluxDB system, for historical analysis. This may be the same system generating the statistical information, but it does not have to be. Information is written used the Line Protocol.

To register with monitor, a module must implement the following interface:

type Client interface {
    Statistics() (map[string]interface{}, error)
    Diagnostics() (map[string]interface{}, error)
}

The module then calls Register(name string, tags map[string]string, client Client). name is the Measurement name that will be associated with the statistics. tags will be the tags, though an empty map is acceptable. client is the module which implements the Client interface.

expvar

Statistical information is gathered by each package using expvar. Each package registers a map using its package name.

Due to the nature of expvar, statistical information is reset to its initial state when a server is restarted.

Configuration

The monitor module will allow the following configuration:

• Whether to write statistical and diagnostic information to an InfluxDB system. This is enabled by default.
• The name of the database to where this information should be written. Defaults to _internal. The information is written to the default retention policy for the given database.
• The name of the retention policy, along with full configuration control of the retention policy.
• The address and port of the InfluxDB system. This will default to the system generating the data.
• The rate at which this information should be written. The maximum rate will be once a second.

Documentation

Index

• Constants
• type Config
  • func NewConfig() Config
• type Diagnostic
  • func DiagnosticFromMap(m map[string]interface{}) *Diagnostic
  • func NewDiagnostic(columns []string) *Diagnostic
  • func (d *Diagnostic) AddRow(r []interface{})
• type DiagsClient
• type DiagsClientFunc
  • func (f DiagsClientFunc) Diagnostics() (*Diagnostic, error)
• type Monitor
  • func New(c Config) *Monitor
  • func (m *Monitor) Close()
  • func (m *Monitor) Diagnostics() (map[string]*Diagnostic, error)
  • func (m *Monitor) Open() error
  • func (m *Monitor) RegisterDiagnosticsClient(name string, client DiagsClient) error
  • func (m *Monitor) SetLogger(l *log.Logger)
  • func (m *Monitor) Statistics(tags map[string]string) ([]*statistic, error)
• type StatementExecutor
  • func (s *StatementExecutor) ExecuteStatement(stmt influxql.Statement) *influxql.Result

Constants

const (
	// DefaultStoreEnabled is whether the system writes gathered information in
	// an InfluxDB system for historical analysis.
	DefaultStoreEnabled = true

	// DefaultStoreDatabase is the name of the database where gathered information is written
	DefaultStoreDatabase = "_internal"

	// DefaultStoreRetentionPolicy is the name of the retention policy for monitor data.
	DefaultStoreRetentionPolicy = "monitor"

	// DefaultRetentionPolicyDuration is the duration the data is retained.
	DefaultStoreRetentionPolicyDuration = 168 * time.Hour

	// DefaultStoreReplicationFactor is the default replication factor for the data.
	DefaultStoreReplicationFactor = 1

	// DefaultStoreInterval is the period between storing gathered information.
	DefaultStoreInterval = 10 * time.Second
)

Types

type Config

type Config struct {
	StoreEnabled           bool          `toml:"store-enabled"`
	StoreDatabase          string        `toml:"store-database"`
	StoreRetentionPolicy   string        `toml:"store-retention-policy"`
	StoreRetentionDuration toml.Duration `toml:"store-retention-duration"`
	StoreReplicationFactor int           `toml:"store-replication-factor"`
	StoreInterval          toml.Duration `toml:"store-interval"`
}

Config represents the configuration for the monitor service.

func NewConfig

func NewConfig() Config

NewConfig returns an instance of Config with defaults.

type Diagnostic

type Diagnostic struct {
	Columns []string
	Rows    [][]interface{}
}

Diagnostic represents a table of diagnostic information. The first value is the name of the columns, the second is a slice of interface slices containing the values for each column, by row. This information is never written to an InfluxDB system and is display-only. An example showing, say, connections follows:

source_ip    source_port       dest_ip     dest_port
182.1.0.2    2890              127.0.0.1   38901
174.33.1.2   2924              127.0.0.1   38902

func DiagnosticFromMap

func DiagnosticFromMap(m map[string]interface{}) *Diagnostic

DiagnosticFromMap returns a Diagnostic from a map.

func NewDiagnostic

func NewDiagnostic(columns []string) *Diagnostic

func (*Diagnostic) AddRow

func (d *Diagnostic) AddRow(r []interface{})

type DiagsClient

type DiagsClient interface {
	Diagnostics() (*Diagnostic, error)
}

DiagsClient is the interface modules implement if they register diags with monitor.

type DiagsClientFunc

type DiagsClientFunc func() (*Diagnostic, error)

The DiagsClientFunc type is an adapter to allow the use of ordinary functions as Diagnostis clients.

func (DiagsClientFunc) Diagnostics

func (f DiagsClientFunc) Diagnostics() (*Diagnostic, error)

Diagnostics calls f().

type Monitor

type Monitor struct {
	// Build information for diagnostics.
	Version string
	Commit  string
	Branch  string

	MetaStore interface {
		ClusterID() (uint64, error)
		NodeID() uint64
		WaitForLeader(d time.Duration) error
		CreateDatabaseIfNotExists(name string) (*meta.DatabaseInfo, error)
		CreateRetentionPolicyIfNotExists(database string, rpi *meta.RetentionPolicyInfo) (*meta.RetentionPolicyInfo, error)
	}

	PointsWriter interface {
		WritePoints(p *cluster.WritePointsRequest) error
	}

	Logger *log.Logger
	// contains filtered or unexported fields
}

Monitor represents an instance of the monitor system.

func New

func New(c Config) *Monitor

New returns a new instance of the monitor system.

func (*Monitor) Close

func (m *Monitor) Close()

Close closes the monitor system.

func (*Monitor) Diagnostics

func (m *Monitor) Diagnostics() (map[string]*Diagnostic, error)

func (*Monitor) Open

func (m *Monitor) Open() error

Open opens the monitoring system, using the given clusterID, node ID, and hostname for identification purposem.

func (*Monitor) RegisterDiagnosticsClient

func (m *Monitor) RegisterDiagnosticsClient(name string, client DiagsClient) error

RegisterDiagnosticsClient registers a diagnostics client with the given name and tags.

func (*Monitor) SetLogger

func (m *Monitor) SetLogger(l *log.Logger)

SetLogger sets the internal logger to the logger passed in.

func (*Monitor) Statistics

func (m *Monitor) Statistics(tags map[string]string) ([]*statistic, error)

Statistics returns the combined statistics for all expvar data. The given tags are added to each of the returned statistics.

type StatementExecutor

type StatementExecutor struct {
	Monitor interface {
		Statistics(map[string]string) ([]*statistic, error)
		Diagnostics() (map[string]*Diagnostic, error)
	}
}

StatementExecutor translates InfluxQL queries to Monitor methods.

func (*StatementExecutor) ExecuteStatement

func (s *StatementExecutor) ExecuteStatement(stmt influxql.Statement) *influxql.Result

ExecuteStatement executes monitor-related query statements.