statuscheck

README

Concept

The statuscheck infrastructure plugin monitors the overall status of a CN-Infra based app by collecting and aggregating partial statuses of agents plugins. The status is exposed to external clients via ETCD - datasync and HTTP, as shown in the following diagram:

For more detailed description see the godoc (especially doc.go).

Overall Agent Status

The overall Agent Status is aggregated from all Plugins' Status (logical AND for each Plugin Status success/error).

The agent's current overall status can be retrieved from ETCD from the following key: /vnf-agent/<agent-label>/check/status

$ etcdctl get /vnf-agent/<agent-label>/check/status/v1/agent
/vnf-agent/<agent-label>/check/status/v1/agent
{"build_version":"e059fdfcd96565eb976a947b59ce56cfb7b1e8a0","build_date":"2017-06-16.14:59","state":1,"start_time":1497617981,"last_change":1497617981,"last_update":1497617991}

To verify the agent status via HTTP (e.g. for Kubernetes liveness and readiness probes, use the /liveness and /readiness URLs:

$ curl -X GET http://localhost:9191/liveness
{"build_version":"e059fdfcd96565eb976a947b59ce56cfb7b1e8a0","build_date":"2017-06-16.14:59","state":1,"start_time":1497617981,"last_change":1497617981,"last_update":1497617991}
$ curl -X GET http://localhost:9191/readiness
{"build_version":"e059fdfcd96565eb976a947b59ce56cfb7b1e8a0","build_date":"2017-06-16.14:59","state":1,"start_time":1497617981,"last_change":1497617981,"last_update":1497617991}

To change the HTTP server port (default 9191), use the http-port option of the agent, e.g.:

$ vpp-agent -http-port 9090

Plugin Status

Plugin may use PluginStatusWriter.ReportStateChange API to PUSH the status information at any time. For optimum performance, 'statuscheck' will then propagate the status report further to external clients only if it has changed since the last update.

Alternatively, plugin may chose to use the PULL based approach and define the probe function passed to PluginStatusWriter.Register API. statuscheck will then periodically probe the plugin for the current status. Once again, the status is propagated further only if it has changed since the last enquiry.

It is recommended not to mix the PULL and the PUSH based approach within the same plugin.

To retrieve the current status of a plugin from ETCD, use the following key template: /vnf-agent/<agent-label>/check/status/v1/plugin/<PLUGIN_NAME>

For example, to retrieve the status of the GoVPP plugin, use:

$ etcdctl get /vnf-agent/<agent-label>/check/status/v1/plugin/GOVPP
/vnf-agent/<agent-label>/check/status/v1/plugin/GOVPP
{"state":2,"last_change":1496322205,"last_update":1496322361,"error":"VPP disconnected"}

PUSH Plugin Status:

PULL Plugins Status - PROBING:

Documentation

Overview

Package statuscheck defines the status report API for other CN-Infra plugins and implements the health status aggregator/exporter. Health status is collected from other plugins through the plugin status report API and aggregated and exported/exposed via ETCD or a REST API.

The API provides only two functions, one for registering the plugin for status change reporting and one for reporting status changes.

To register a plugin for providing status reports, use Register() function:

statuscheck.Register(PluginID, probe)

If probe is not nil, statuscheck will periodically probe the plugin state through the provided function. Otherwise, it is expected that the plugin itself will report state updates through ReportStateChange(), e.g.:

statuscheck.ReportStateChange(PluginID, statuscheck.OK, nil)

The default status of a plugin after registering is Init.

Index

• Variables
• type AgentStatusReader
• type Deps
• type InterfaceStatusReader
• type Option
  • func UseDeps(cb func(*Deps)) Option
• type Plugin
  • func NewPlugin(opts ...Option) *Plugin
  • func (p *Plugin) AfterInit() error
  • func (p *Plugin) Close() error
  • func (p *Plugin) GetAgentStatus() status.AgentStatus
  • func (p *Plugin) GetAllPluginStatus() map[string]*status.PluginStatus
  • func (p *Plugin) GetInterfaceStats() status.InterfaceStats
  • func (p *Plugin) Init() error
  • func (p *Plugin) Register(pluginName infra.PluginName, probe PluginStateProbe)
  • func (p *Plugin) ReportStateChange(pluginName infra.PluginName, state PluginState, lastError error)
  • func (p *Plugin) ReportStateChangeWithMeta(pluginName infra.PluginName, state PluginState, lastError error, ...)
• type PluginState
• type PluginStateProbe
• type PluginStatusWriter
• type StatusReader

Variables

var (
	// PeriodicWriteTimeout is frequency of periodic writes of state data into ETCD.
	PeriodicWriteTimeout = time.Second * 10
	// PeriodicProbingTimeout is frequency of periodic plugin state probing.
	PeriodicProbingTimeout = time.Second * 5
)

var DefaultPlugin = *NewPlugin()

DefaultPlugin is a default instance of Plugin.

Types

type AgentStatusReader

type AgentStatusReader interface {
	// GetAgentStatus returns the current global operational state of the agent.
	GetAgentStatus() status.AgentStatus
}

AgentStatusReader allows to lookup agent status by other plugins.

type Deps

type Deps struct {
	infra.PluginName                            // inject
	Log              logging.PluginLogger       // inject
	Transport        datasync.KeyProtoValWriter // inject (optional)
}

Deps lists the dependencies of statuscheck plugin.

type InterfaceStatusReader

type InterfaceStatusReader interface {
	GetInterfaceStats() status.InterfaceStats
}

InterfaceStatusReader looks up the interface state and returns updated state data

type Option

type Option func(*Plugin)

Option is a function that can be used in NewPlugin to customize Plugin.

func UseDeps

func UseDeps(cb func(*Deps)) Option

UseDeps returns Option that can inject custom dependencies.

type Plugin

type Plugin struct {
	Deps
	// contains filtered or unexported fields
}

Plugin struct holds all plugin-related data.

func NewPlugin

func NewPlugin(opts ...Option) *Plugin

NewPlugin creates a new Plugin with the provided Options.

func (*Plugin) AfterInit

func (p *Plugin) AfterInit() error

AfterInit starts go routines for periodic probing and periodic updates. Initial state data are published via the injected transport.

func (*Plugin) Close

func (p *Plugin) Close() error

Close stops go routines for periodic probing and periodic updates.

func (*Plugin) GetAgentStatus

func (p *Plugin) GetAgentStatus() status.AgentStatus

GetAgentStatus return current global operational state of the agent.

func (*Plugin) GetAllPluginStatus

func (p *Plugin) GetAllPluginStatus() map[string]*status.PluginStatus

GetAllPluginStatus returns a map containing pluginname and its status, for all plugins

func (*Plugin) GetInterfaceStats

func (p *Plugin) GetInterfaceStats() status.InterfaceStats

GetInterfaceStats returns current global operational status of interfaces

func (*Plugin) Init

func (p *Plugin) Init() error

Init prepares the initial status data.

func (*Plugin) Register

func (p *Plugin) Register(pluginName infra.PluginName, probe PluginStateProbe)

Register a plugin for status change reporting.

func (*Plugin) ReportStateChange

func (p *Plugin) ReportStateChange(pluginName infra.PluginName, state PluginState, lastError error)

ReportStateChange can be used to report a change in the status of a previously registered plugin.

func (*Plugin) ReportStateChangeWithMeta

func (p *Plugin) ReportStateChangeWithMeta(pluginName infra.PluginName, state PluginState, lastError error, meta proto.Message)

ReportStateChangeWithMeta can be used to report a change in the status of a previously registered plugin and report the specific metadata state

type PluginState

type PluginState string

PluginState is a data type used to describe the current operational state of a plugin.

const (
	// Init state means that the initialization of the plugin is in progress.
	Init PluginState = "init"
	// OK state means that the plugin is healthy.
	OK PluginState = "ok"
	// Error state means that some error has occurred in the plugin.
	Error PluginState = "error"
)

type PluginStateProbe

type PluginStateProbe func() (PluginState, error)

PluginStateProbe defines parameters of a function used for plugin state probing, referred to as "probe".

type PluginStatusWriter

type PluginStatusWriter interface {
	// Register registers a plugin for status change reporting.
	// If <probe> is not nil, Statuscheck will periodically probe the plugin
	// state through the provided function. Otherwise, it is expected that the
	// plugin itself will report state updates through ReportStateChange().
	Register(pluginName infra.PluginName, probe PluginStateProbe)

	// ReportStateChange can be used to report a change in the status
	// of a previously registered plugin. It is not a bug, however, to report
	// the same status in consecutive calls. Statuscheck is smart enough
	// to detect an actual status change and propagate only updates to remote
	// clients.
	ReportStateChange(pluginName infra.PluginName, state PluginState, lastError error)

	// ReportStateChangeWithMeta can be used to report a change in the status
	// of a previously registered plugin with added metadata value stored in
	// global agent status. Metadata type is specified in statuscheck model.
	ReportStateChangeWithMeta(pluginName infra.PluginName, state PluginState, lastError error, meta proto.Message)
}

PluginStatusWriter allows to register & write plugin status by other plugins

type StatusReader

type StatusReader interface {
	AgentStatusReader
	InterfaceStatusReader
	GetAllPluginStatus() map[string]*status.PluginStatus
}

StatusReader allows to lookup agent status and retrieve a map containing status of all plugins.