orca

Documentation

Overview

Package orca implements Open Request Cost Aggregation, which is an open standard for request cost aggregation and reporting by backends and the corresponding aggregation of such reports by L7 load balancers (such as Envoy) on the data plane. In a proxyless world with gRPC enabled applications, aggregation of such reports will be done by the gRPC client.

Experimental

Notice: All APIs is this package are EXPERIMENTAL and may be changed or removed in a later release.

Index

• func CallMetricsServerOption(smp ServerMetricsProvider) grpc.ServerOption
• func Register(s *grpc.Server, opts ServiceOptions) error
• func RegisterOOBListener(sc balancer.SubConn, l OOBListener, opts OOBListenerOptions) (stop func())
• type CallMetricsRecorder
  • func CallMetricsRecorderFromContext(ctx context.Context) CallMetricsRecorder
• type OOBListener
• type OOBListenerOptions
• type ServerMetrics
• type ServerMetricsProvider
• type ServerMetricsRecorder
  • func NewServerMetricsRecorder() ServerMetricsRecorder
• type Service
  • func NewService(opts ServiceOptions) (*Service, error)
  • func (s *Service) StreamCoreMetrics(req *v3orcaservicepb.OrcaLoadReportRequest, ...) error
• type ServiceOptions

Functions

func CallMetricsServerOption

func CallMetricsServerOption(smp ServerMetricsProvider) grpc.ServerOption

CallMetricsServerOption returns a server option which enables the reporting of per-RPC custom backend metrics for unary and streaming RPCs.

Server applications interested in injecting custom backend metrics should pass the server option returned from this function as the first argument to grpc.NewServer().

Subsequently, server RPC handlers can retrieve a reference to the RPC specific custom metrics recorder CallMetricsRecorder to be used, via a call to CallMetricsRecorderFromContext(), and inject custom metrics at any time during the RPC lifecycle.

The injected custom metrics will be sent as part of trailer metadata, as a binary-encoded ORCA LoadReport protobuf message, with the metadata key being set be "endpoint-load-metrics-bin".

If a non-nil ServerMetricsProvider is provided, the gRPC server will transmit the metrics it provides, overwritten by any per-RPC metrics given to the CallMetricsRecorder. A ServerMetricsProvider is typically obtained by calling NewServerMetricsRecorder.

func Register

func Register(s *grpc.Server, opts ServiceOptions) error

Register creates a new ORCA service implementation configured using the provided options and registers the same on the provided grpc Server.

func RegisterOOBListener

func RegisterOOBListener(sc balancer.SubConn, l OOBListener, opts OOBListenerOptions) (stop func())

RegisterOOBListener registers an out-of-band load report listener on sc. Any OOBListener may only be registered once per subchannel at a time. The returned stop function must be called when no longer needed. Do not register a single OOBListener more than once per SubConn.

Types

type CallMetricsRecorder

type CallMetricsRecorder interface {
	ServerMetricsRecorder

	// SetRequestCost sets the relevant server metric.
	SetRequestCost(name string, val float64)
	// DeleteRequestCost deletes the relevant server metric to prevent it
	// from being sent.
	DeleteRequestCost(name string)

	// SetNamedMetric sets the relevant server metric.
	SetNamedMetric(name string, val float64)
	// DeleteNamedMetric deletes the relevant server metric to prevent it
	// from being sent.
	DeleteNamedMetric(name string)
}

CallMetricsRecorder allows a service method handler to record per-RPC metrics. It contains all utilization-based metrics from ServerMetricsRecorder as well as additional request cost metrics.

func CallMetricsRecorderFromContext

func CallMetricsRecorderFromContext(ctx context.Context) CallMetricsRecorder

CallMetricsRecorderFromContext returns the RPC-specific custom metrics recorder embedded in the provided RPC context.

Returns nil if no custom metrics recorder is found in the provided context, which will be the case when custom metrics reporting is not enabled.

type OOBListener

type OOBListener interface {
	// OnLoadReport is called when a load report is received.
	OnLoadReport(*v3orcapb.OrcaLoadReport)
}

OOBListener is used to receive out-of-band load reports as they arrive.

type OOBListenerOptions

type OOBListenerOptions struct {
	// ReportInterval specifies how often to request the server to provide a
	// load report.  May be provided less frequently if the server requires a
	// longer interval, or may be provided more frequently if another
	// subscriber requests a shorter interval.
	ReportInterval time.Duration
}

OOBListenerOptions contains options to control how an OOBListener is called.

type ServerMetrics

type ServerMetrics struct {
	CPUUtilization float64 // CPU utilization: [0, inf); unset=-1
	MemUtilization float64 // Memory utilization: [0, 1.0]; unset=-1
	AppUtilization float64 // Application utilization: [0, inf); unset=-1
	QPS            float64 // queries per second: [0, inf); unset=-1
	EPS            float64 // errors per second: [0, inf); unset=-1

	Utilization  map[string]float64 // Custom fields: [0, 1.0]
	RequestCost  map[string]float64 // Custom fields: [0, inf); not sent OOB
	NamedMetrics map[string]float64 // Custom fields: [0, inf); not sent OOB
}

ServerMetrics is the data returned from a server to a client to describe the current state of the server and/or the cost of a request when used per-call.

type ServerMetricsProvider

type ServerMetricsProvider interface {
	// ServerMetrics returns the current set of server metrics.  It should
	// return a read-only, immutable copy of the data that is active at the
	// time of the call.
	ServerMetrics() *ServerMetrics
}

A ServerMetricsProvider provides ServerMetrics upon request.

type ServerMetricsRecorder

type ServerMetricsRecorder interface {
	ServerMetricsProvider

	// SetCPUUtilization sets the CPU utilization server metric.  Must be
	// greater than zero.
	SetCPUUtilization(float64)
	// DeleteCPUUtilization deletes the CPU utilization server metric to
	// prevent it from being sent.
	DeleteCPUUtilization()

	// SetMemoryUtilization sets the memory utilization server metric.  Must be
	// in the range [0, 1].
	SetMemoryUtilization(float64)
	// DeleteMemoryUtilization deletes the memory utiliztion server metric to
	// prevent it from being sent.
	DeleteMemoryUtilization()

	// SetApplicationUtilization sets the application utilization server
	// metric.  Must be greater than zero.
	SetApplicationUtilization(float64)
	// DeleteApplicationUtilization deletes the application utilization server
	// metric to prevent it from being sent.
	DeleteApplicationUtilization()

	// SetQPS sets the Queries Per Second server metric.  Must be greater than
	// zero.
	SetQPS(float64)
	// DeleteQPS deletes the Queries Per Second server metric to prevent it
	// from being sent.
	DeleteQPS()

	// SetEPS sets the Errors Per Second server metric.  Must be greater than
	// zero.
	SetEPS(float64)
	// DeleteEPS deletes the Errors Per Second server metric to prevent it from
	// being sent.
	DeleteEPS()

	// SetNamedUtilization sets the named utilization server metric for the
	// name provided.  val must be in the range [0, 1].
	SetNamedUtilization(name string, val float64)
	// DeleteNamedUtilization deletes the named utilization server metric for
	// the name provided to prevent it from being sent.
	DeleteNamedUtilization(name string)
}

ServerMetricsRecorder allows for recording and providing out of band server metrics.

func NewServerMetricsRecorder

func NewServerMetricsRecorder() ServerMetricsRecorder

NewServerMetricsRecorder returns an in-memory store for ServerMetrics and allows for safe setting and retrieving of ServerMetrics. Also implements ServerMetricsProvider for use with NewService.

type Service

type Service struct {
	v3orcaservicegrpc.UnimplementedOpenRcaServiceServer
	// contains filtered or unexported fields
}

Service provides an implementation of the OpenRcaService as defined in the ORCA service protos. Instances of this type must be created via calls to Register() or NewService().

Server applications can use the SetXxx() and DeleteXxx() methods to record measurements corresponding to backend metrics, which eventually get pushed to clients who have initiated the SteamCoreMetrics streaming RPC.

func NewService

func NewService(opts ServiceOptions) (*Service, error)

NewService creates a new ORCA service implementation configured using the provided options.

func (*Service) StreamCoreMetrics

func (s *Service) StreamCoreMetrics(req *v3orcaservicepb.OrcaLoadReportRequest, stream v3orcaservicegrpc.OpenRcaService_StreamCoreMetricsServer) error

StreamCoreMetrics streams custom backend metrics injected by the server application.

type ServiceOptions

type ServiceOptions struct {
	// ServerMetricsProvider is the provider to be used by the service for
	// reporting OOB server metrics to clients.  Typically obtained via
	// NewServerMetricsRecorder.  This field is required.
	ServerMetricsProvider ServerMetricsProvider

	// MinReportingInterval sets the lower bound for how often out-of-band
	// metrics are reported on the streaming RPC initiated by the client. If
	// unspecified, negative or less than the default value of 30s, the default
	// is used. Clients may request a higher value as part of the
	// StreamCoreMetrics streaming RPC.
	MinReportingInterval time.Duration
	// contains filtered or unexported fields
}

ServiceOptions contains options to configure the ORCA service implementation.