metrics

Documentation

Index

• Constants
• func Init()
• func MustInit(cfg *config.Config, registry ...*prometheus.Registry)
• func PublishMetrics()
• type APIType
• type GenericMetrics
  • func NewGenericMetricsClient(subsystem string, registry *prometheus.Registry) *GenericMetrics
  • func (gm *GenericMetrics) FireCallEnd(callHash, callName string, timestamp time.Time, callStarted chan bool)
  • func (gm *GenericMetrics) FireCallStart(callHash, callName string, timestamp time.Time, callStarted chan bool)
  • func (gm *GenericMetrics) IncrementCallCount(callName string)
  • func (gm *GenericMetrics) RecordCall(callID, callName string, callTime time.Time, callStarted chan bool) string
• type MetricsClient
  • func NewMetricsClient(api APIType, registry *prometheus.Registry) MetricsClient
• type MetricsEngine
  • func NewMetricsEngine(cfg *config.Config, registry *prometheus.Registry) *MetricsEngine
  • func (engine *MetricsEngine) RecordDockerMetric(callName string) func()
  • func (engine *MetricsEngine) RecordECSClientMetric(callName string) func()
  • func (engine *MetricsEngine) RecordStateManagerMetric(callName string) func()
  • func (engine *MetricsEngine) RecordTaskEngineMetric(callName string) func()
• type MetricsEngine_

Constants

const (
	AgentNamespace        = "AgentMetrics"
	DockerSubsystem       = "DockerAPI"
	TaskEngineSubsystem   = "TaskEngine"
	StateManagerSubsystem = "StateManager"
	ECSClientSubsystem    = "ECSClient"
)

Functions

func Init

func Init()

func MustInit

func MustInit(cfg *config.Config, registry ...*prometheus.Registry)

Initializes the Global MetricsEngine used throughout Agent Currently, we use the Prometheus Global Default Registerer, which also collecs basic Go application metrics that we use (like memory usage). In future cases, we can use a custom Prometheus Registry to group metrics. For unit testing purposes, we only focus on API calls and use our own Registry

func PublishMetrics

func PublishMetrics()

Function called during Agent start up to expose metrics on a local endpoint

Types

type APIType

type APIType int32

const (
	DockerAPI APIType = iota
	TaskEngine
	StateManager
	ECSClient
)

type GenericMetrics

type GenericMetrics struct {
	// contains filtered or unexported fields
}

A GenericMetricsClient records 3 metrics:

1. A Prometheus summary vector representing call durations for different API calls
2. A durations guage vector that updates the last recorded duration for the API call allowing for a time series view in the Prometheus browser
3. A counter vector that increments call counts for each API call

The outstandingCalls map allows Fired CallStarts to be matched with Fired CallEnds

func NewGenericMetricsClient

func NewGenericMetricsClient(subsystem string, registry *prometheus.Registry) *GenericMetrics

func (*GenericMetrics) FireCallEnd

func (gm *GenericMetrics) FireCallEnd(callHash, callName string, timestamp time.Time, callStarted chan bool)

func (*GenericMetrics) FireCallStart

func (gm *GenericMetrics) FireCallStart(callHash, callName string, timestamp time.Time, callStarted chan bool)

func (*GenericMetrics) IncrementCallCount

func (gm *GenericMetrics) IncrementCallCount(callName string)

This function increments the call count for a specific API call This is invoked at the API call's start, whereas the duration metrics are updated at the API call's end.

func (*GenericMetrics) RecordCall

func (gm *GenericMetrics) RecordCall(callID, callName string, callTime time.Time, callStarted chan bool) string

This function creates a hash of the callName and a randomly generated number. The hash serves to match the call invokation with the call's end, which is required to compute the call duration. The final metric is observed when the FireCallEnd function is called. If callID is empty, then we initiate the call with FireCallStart We use a channel holding 1 bool to ensure that the FireCallEnd is called AFTER the FireCallStart (because these are done in separate go routines)

type MetricsClient

type MetricsClient interface {
	// RecordCall is responsible for accepting a call ID, call Name,
	// and timestamp for the call.
	// The specific behavior of RecordCall is dependent on the type of client
	// that is used. It is the responsibility of the MetricsEngine to call the
	// appropriate RecordCall(...) method.
	// This method is the defining function for this interface.
	// We use a channel holding 1 bool to ensure that the FireCallEnd is called AFTER
	// the FireCallStart (because these are done in separate go routines)
	RecordCall(string, string, time.Time, chan bool) string

	// In order to record call duration, we must fire a method's start and end
	// at different times (once at the beginning and once at the end). Ideally,
	// RecordCall should handle this through an execution and returned function
	// to be deferred (see docker_client.go for usage examples)
	FireCallStart(string, string, time.Time, chan bool)
	FireCallEnd(string, string, time.Time, chan bool)

	// This function will increment the call count. The metric for call duration
	// should ideally have the accurate call count as well, but the Prometheus
	// summary metric is updated for count when the FireCallEnd(...) function is
	// called. When a FireCallStart(...) is called but its FireCallEnd(...) is not,
	// we will see a discrepancy between the Summary's count and this Gauge count
	IncrementCallCount(string)
}

MetricsClient defines the behavior for any Client that uses the MetricsEngine_ interface to collect metrics for Prometheus. Clients should all have the same type of metrics collected (Durations, counts, etc.) In particular, the API calls we monitor in the Agent code (TaskEngine, Docker, ECSClient, etc.) utilize call durations and counts which are encapsulated in the GenericMetricsClient. This interface is extensible for future clients that require different collection behaviors.

func NewMetricsClient

func NewMetricsClient(api APIType, registry *prometheus.Registry) MetricsClient

A factory method that enables various MetricsClients to be created.

type MetricsEngine

type MetricsEngine struct {
	Registry *prometheus.Registry
	// contains filtered or unexported fields
}

var (
	MetricsEngineGlobal *MetricsEngine = &MetricsEngine{
		collection: false,
	}
)

Maintained list of APIs for which we collect metrics. MetricsClients will be initialized using Factory method when a MetricsEngine is created.

func NewMetricsEngine

func NewMetricsEngine(cfg *config.Config, registry *prometheus.Registry) *MetricsEngine

We create a MetricsClient for all managed APIs (APIs for which we will collect metrics)

func (*MetricsEngine) RecordDockerMetric

func (engine *MetricsEngine) RecordDockerMetric(callName string) func()

Wrapper function that allows APIs to call a single function

func (*MetricsEngine) RecordECSClientMetric

func (engine *MetricsEngine) RecordECSClientMetric(callName string) func()

Wrapper function that allows APIs to call a single function

func (*MetricsEngine) RecordStateManagerMetric

func (engine *MetricsEngine) RecordStateManagerMetric(callName string) func()

Wrapper function that allows APIs to call a single function

func (*MetricsEngine) RecordTaskEngineMetric

func (engine *MetricsEngine) RecordTaskEngineMetric(callName string) func()

Wrapper function that allows APIs to call a single function

type MetricsEngine_

type MetricsEngine_ interface {
	// This init function initializes the Global MetricsEngine variable that
	// can be accessed throughout the Agent codebase on packages that the
	// metrics package does not depend on (cyclic dependencies not allowed).
	MustInit(*config.Config)
	// contains filtered or unexported methods
}

MetricsEngine_ is an interface that drives metric collection over all existing MetricsClients. The APIType parameter corresponds to the type of MetricsClient RecordCall(...) function that will be used. The MetricsEngine_ should be instantiated at Agent startup time with Agent configurations and context passed as parameters