metrics

Documentation

Index

• Constants
• Variables
• func GetNamespace(ctx context.Context) string
• func RegisterMetrics() error
• func StreamMonitoring() grpc.StreamServerInterceptor
• func UnaryMonitoring() grpc.UnaryServerInterceptor
• func UpdateNamespace(ctx context.Context, namespace string)
• func WithSharedNamespace(ctx context.Context) context.Context
• type MetricsKey
• type MetricsService
  • func New(conf config.MetricsConfig) (*MetricsService, error)
  • func (m *MetricsService) Serve() error
  • func (m *MetricsService) Shutdown(ctx context.Context) error
• type MonitoredStream
  • func (s *MonitoredStream) Context() context.Context
  • func (s *MonitoredStream) Msgs() float64
  • func (s *MonitoredStream) RecvMsg(m interface{}) (err error)
  • func (s *MonitoredStream) SendMsg(m interface{}) (err error)
• type SharedNamespace

Constants

const (
	PmNamespaceTrtl = "trtl"
	PmNamespaceGRPC = "grpc"
)

Prometheus namespaces for the collectors defined in this package.

Variables

var (
	// Basic RPC Metrics
	PmRPCStarted        *prometheus.CounterVec   // RPCs started by method, namespace
	PmRPCHandled        *prometheus.CounterVec   // RPCs completed by method, namespace, and code
	PmRPCUnaryLatency   *prometheus.HistogramVec // the time it is taking for successful unary RPC calls to complete, labeled by RPC type, namespace, and code
	PmRPCStreamDuration *prometheus.HistogramVec // the time it is taking for successful streaming RPC calls to complete, labeled by type, namespace, and code
	PmMsgsPerStream     *prometheus.HistogramVec // the number of messages sent and recv per streaming RPC, labeled by type, namespace, and code

	// Storage Metrics
	// TODO: add version metrics
	PmTrtlReads         *prometheus.CounterVec   // number of reads, e.g. Get and Iter to the embedded database, by namespace
	PmTrtlBytesRead     *prometheus.CounterVec   // number of bytes read by trtl operations by namespace
	PmTrtlWrites        *prometheus.CounterVec   // number of writes, e.g. Puts and Deletes to the embedded database, by namespace
	PmTrtlBytesWritten  *prometheus.CounterVec   // number of bytes written by trtl operations by namespace
	PmObjectSize        *prometheus.HistogramVec // average size in bytes of objects stored in trtl, by namespace
	PmDatabaseSize      *prometheus.GaugeVec     // current size in bytes of all objects in the database, by namespace
	PmCurrentObjects    *prometheus.GaugeVec     // current number of objects in the database, by namespace
	PmCurrentTombstones *prometheus.GaugeVec     // current number of tombstones int he database, by namespace

	// Anti-Entropy Metrics
	PmAESyncs         *prometheus.CounterVec   // count of anti entropy sessions per peer, per region, and by perspective (initiator/remote)
	PmAESyncLatency   *prometheus.HistogramVec // duration of anti entropy sessions (initiator perspective), by peer and region
	PmAEPhase1Latency *prometheus.HistogramVec // time phase 1 of anti-entropy is taking from the perspective of the initiator, by peer
	PmAEPhase2Latency *prometheus.HistogramVec // time phase 2 of anti-entropy is taking from the perspective of the remote, by peer
	PmAEVersions      *prometheus.HistogramVec // count of all observed versions, per peer and region
	PmAEUpdates       *prometheus.HistogramVec // pushed objects during anti entropy, by peer and region
	PmAERepairs       *prometheus.HistogramVec // pulled objects during anti entropy, by peer and region
	PmAEStomps        *prometheus.CounterVec   // count of stomped versions, per peer and region
	PmAESkips         *prometheus.CounterVec   // count of skipped versions, per peer and region
)

All trtl specific collectors for observability are defined here.

Functions

func GetNamespace

func GetNamespace(ctx context.Context) string

GetNamespace returns the namespace from the shared namespace context value.

func RegisterMetrics

func RegisterMetrics() error

Initializes and registers the metrics collectors in Prometheus. This function can safely be called multiple times and the collectors will only be registered once. This method can be used prior to tests to ensure that there are no nil panics for handlers that make use of the collectors; otherwise it will be called when creating a new metrics server in preparation for exposing application metrics.

func StreamMonitoring

func StreamMonitoring() grpc.StreamServerInterceptor

StreamMonitoring is an interceptor that handels generic gRPC streaming monitoring, updating prometheus metrics for trtl and tracking each RPC by service and method. Streaming RPC handlers should update the namespace on the context for this interceptor to correctly track what is happening on a per-namespace basis.

func UnaryMonitoring

func UnaryMonitoring() grpc.UnaryServerInterceptor

UnaryMonitoring is an interceptor that handles the generic gRPC monitoring prometheus metrics for trtl, tracking each RPC by service and method name. Unary RPC handlers should update the namespace on the context for this interceptor to correctly track what is happening on a per-namespace basis.

func UpdateNamespace

func UpdateNamespace(ctx context.Context, namespace string)

UpdateNamespace updates the context with the shared namespace value if available.

func WithSharedNamespace

func WithSharedNamespace(ctx context.Context) context.Context

WithSharedNamespace updates the context with a shared namespace value.

Types

type MetricsKey

type MetricsKey uint8

var NamespaceKey MetricsKey = 1

type MetricsService

type MetricsService struct {
	// contains filtered or unexported fields
}

A MetricsService manages Prometheus metrics

func New

func New(conf config.MetricsConfig) (*MetricsService, error)

New creates a metrics service and also initializes all of the prometheus metrics. The trtl server *must* create the metrics service by calling New before any metrics are logged to Prometheus.

func (*MetricsService) Serve

func (m *MetricsService) Serve() error

Serve serves the Prometheus metrics

func (*MetricsService) Shutdown

func (m *MetricsService) Shutdown(ctx context.Context) error

Gracefully shutdown the Prometheus metrics service

type MonitoredStream

type MonitoredStream struct {
	grpc.ServerStream
	// contains filtered or unexported fields
}

MonitoredStream wraps a grpc.ServerStream allowing it to increment Sent and Recv message counters when they are called by the application.

func (*MonitoredStream) Context

func (s *MonitoredStream) Context() context.Context

func (*MonitoredStream) Msgs

func (s *MonitoredStream) Msgs() float64

func (*MonitoredStream) RecvMsg

func (s *MonitoredStream) RecvMsg(m interface{}) (err error)

Increment the number of received messages if there is no error on Recv.

func (*MonitoredStream) SendMsg

func (s *MonitoredStream) SendMsg(m interface{}) (err error)

Increment the number of sent messages if there is no error on Send.

type SharedNamespace

type SharedNamespace struct {
	Namespace string
}

SharedNamespace is a bit of a hack, using a pointer we allow the child handler to update the namespace so that the parent interceptor can use the namespace as a label for monitoring. This is not a correct or standard way to use contexts and it is not thread-safe even though contexts should be.