loadshedding

Documentation

Index

• Constants
• func ThresholdExceeded(eval LoadEvaluation) bool
• type GRPCLatencyEvaluator
  • func NewGRPCLatencyEvaluator(sampleFrequency rate.Limit, averageHalfLife time.Duration) *GRPCLatencyEvaluator
  • func (g *GRPCLatencyEvaluator) EvaluateAgainst(ri RequestInfo, threshold float64) LoadEvaluation
  • func (g *GRPCLatencyEvaluator) HandleConn(context.Context, stats.ConnStats)
  • func (g *GRPCLatencyEvaluator) HandleRPC(ctx context.Context, rs stats.RPCStats)
  • func (g GRPCLatencyEvaluator) Name() string
  • func (g *GRPCLatencyEvaluator) TagConn(ctx context.Context, cti *stats.ConnTagInfo) context.Context
  • func (g *GRPCLatencyEvaluator) TagRPC(ctx context.Context, rti *stats.RPCTagInfo) context.Context
• type LoadEvaluation
• type LoadEvaluator
• type LoadStatus
• type Options
  • func DefaultOptions() Options
  • func (o *Options) AttachCobraFlags(cmd *cobra.Command)
• type RateLimitEvaluator
  • func NewRateLimitEvaluator(limit rate.Limit, burstSize int) *RateLimitEvaluator
  • func (r *RateLimitEvaluator) EvaluateAgainst(ri RequestInfo, threshold float64) LoadEvaluation
  • func (r RateLimitEvaluator) Name() string
• type RequestInfo
• type Throttler
  • func NewThrottler(opts Options) *Throttler
  • func (t *Throttler) Evaluator(name string) LoadEvaluator
  • func (t *Throttler) Throttle(ri RequestInfo) bool
• type ThrottlerMode

Constants

const (
	// DefaultSampleFrequency controls the base sampling rate of latency averaging calculation.
	DefaultSampleFrequency = rate.Inf
	// DefaultHalfLife controls the decay rate of an individual sample.
	DefaultHalfLife = 1 * time.Second // Impact of each sample is expected to last ~2s.

	// GRPCLatencyEvaluatorName is the name of the gRPC Response Latency LoadEvaluator.
	GRPCLatencyEvaluatorName = "grpcResponseLatency"
)

const (
	// RateLimitEvaluatorName is the canonical name of the RateLimitEvaluator.
	RateLimitEvaluatorName = "RateLimit"
)

Functions

func ThresholdExceeded

func ThresholdExceeded(eval LoadEvaluation) bool

ThresholdExceeded determines if a load evaluation status is `ExceedsThreshold`.

Types

type GRPCLatencyEvaluator

type GRPCLatencyEvaluator struct {
	// contains filtered or unexported fields
}

GRPCLatencyEvaluator calculates the moving average of response latency (as reported via the gRPC stats.Handler interface). It then evaluates incoming requests by comparing the average response latency against a threshold.

func NewGRPCLatencyEvaluator

func NewGRPCLatencyEvaluator(sampleFrequency rate.Limit, averageHalfLife time.Duration) *GRPCLatencyEvaluator

NewGRPCLatencyEvaluator creates a new LoadEvaluator that uses an average of gRPC Response Latency.

func (*GRPCLatencyEvaluator) EvaluateAgainst

func (g *GRPCLatencyEvaluator) EvaluateAgainst(ri RequestInfo, threshold float64) LoadEvaluation

EvaluateAgainst implements the LoadEvaluator interface.

func (*GRPCLatencyEvaluator) HandleConn

func (g *GRPCLatencyEvaluator) HandleConn(context.Context, stats.ConnStats)

HandleConn processes the Conn stats.

func (*GRPCLatencyEvaluator) HandleRPC

func (g *GRPCLatencyEvaluator) HandleRPC(ctx context.Context, rs stats.RPCStats)

HandleRPC processes the RPC stats.

func (GRPCLatencyEvaluator) Name

func (g GRPCLatencyEvaluator) Name() string

Name implements the LoadEvaluator interface.

func (*GRPCLatencyEvaluator) TagConn

func (g *GRPCLatencyEvaluator) TagConn(ctx context.Context, cti *stats.ConnTagInfo) context.Context

TagConn can attach some information to the given context.

func (*GRPCLatencyEvaluator) TagRPC

func (g *GRPCLatencyEvaluator) TagRPC(ctx context.Context, rti *stats.RPCTagInfo) context.Context

TagRPC can attach some information to the given context.

type LoadEvaluation

type LoadEvaluation struct {
	// Status indicates whether or not the threshold was exceeded.
	Status LoadStatus

	// Message enables LoadEvaluators to provide custom error messages when the threshold is
	// exceeded.
	Message string
}

LoadEvaluation holds the result of the evaluation of a current request against a threshold.

type LoadEvaluator

type LoadEvaluator interface {
	// Name returns a canonical name for the LoadEvaluator.
	Name() string

	// EvaluateAgainst compares the current request and known load against the supplied
	// threshold to determine if the threshold is/will be exceeded.
	EvaluateAgainst(ri RequestInfo, threshold float64) LoadEvaluation
}

LoadEvaluator evaluates the current request against a threshold. LoadEvaluators may calculate load on an on-going basis or through some external mechanism.

type LoadStatus

type LoadStatus int

LoadStatus records the determination of a LoadEvaluation.

const (
	// BelowThreshold indicates that a given request will not exceed a threshold.
	BelowThreshold LoadStatus = iota
	// ExceedsThreshold indicates that a given request will exceed a threshold.
	ExceedsThreshold
)

type Options

type Options struct {
	// Mode controls the server loadshedding behavior.
	Mode ThrottlerMode

	// AverageLatencyThreshold is the threshold for response times
	// over which the server will start rejecting requests (Unavailable).
	// Providing a value for AverageLatencyThreshold will enable the gRPC
	// Latency evaluator.
	AverageLatencyThreshold time.Duration

	// SamplesPerSecond controls how often gRPC response latencies are
	// recorded for calculating the average response latency.
	SamplesPerSecond rate.Limit

	// SampleHalfLife controls the decay rate of observations of response latencies.
	SampleHalfLife time.Duration

	// MaxRequestsPerSecond controls the rate of requests over which the
	// server will start rejecting requests (Unavailable). Providing a value
	// for MaxRequestsPerSecond will enable the rate limit evaluator.
	//
	// In Mixer, a single Report() request may translate to multiple requests
	// counted against this limit, depending on batch size of the Report.
	MaxRequestsPerSecond rate.Limit

	// BurstSize controls the number of requests that are permitted beyond the
	// configured maximum for a period of time. This allows for handling bursty
	// traffic patterns. If this is set to 0, no traffic will be allowed.
	BurstSize int
}

Options define the set of configuration parameters for controlling loadshedding behavior.

func DefaultOptions

func DefaultOptions() Options

DefaultOptions returns a new set of options, initialized to the defaults

func (*Options) AttachCobraFlags

func (o *Options) AttachCobraFlags(cmd *cobra.Command)

AttachCobraFlags attaches a set of Cobra flags to the given Cobra command.

Cobra is the command-line processor that Istio uses. This command attaches the necessary set of flags to expose a CLI to let the user control all tracing options.

type RateLimitEvaluator

type RateLimitEvaluator struct {
	// contains filtered or unexported fields
}

RateLimitEvaluator enforces a configured rate limit, using a rate.Limiter.

func NewRateLimitEvaluator

func NewRateLimitEvaluator(limit rate.Limit, burstSize int) *RateLimitEvaluator

NewRateLimitEvaluator builds a new RateLimitEvaluator.

func (*RateLimitEvaluator) EvaluateAgainst

func (r *RateLimitEvaluator) EvaluateAgainst(ri RequestInfo, threshold float64) LoadEvaluation

EvaluateAgainst implements the LoadEvaluator interface.

func (RateLimitEvaluator) Name

func (r RateLimitEvaluator) Name() string

Name implements the LoadEvaluator interface.

type RequestInfo

type RequestInfo struct {
	// PredictedCost enables the server to pass information about the relative
	// size (or impact) of the request into the throttler. For instance, it can
	// be used to distinguish between Check() and Report() calls by setting the
	// value to the size of the batch.
	PredictedCost float64
}

RequestInfo is used to hold information related to a request that could be relevant to a LoadEvaluator.

type Throttler

type Throttler struct {
	// contains filtered or unexported fields
}

Throttler provides the loadshedding behavior by evaluating current request information against a set of configured LoadEvaluators.

func NewThrottler

func NewThrottler(opts Options) *Throttler

NewThrottler builds a Throttler based on the configured options.

func (*Throttler) Evaluator

func (t *Throttler) Evaluator(name string) LoadEvaluator

Evaluator returns a configured LoadEvaluator based on the supplied name. If no LoadEvaluator with the given name is known to the Throttler, a nil value will be returned.

func (*Throttler) Throttle

func (t *Throttler) Throttle(ri RequestInfo) bool

Throttle returns a verdict on whether or not the server should drop the request, based on the current set of configured LoadEvaluators.

type ThrottlerMode

type ThrottlerMode int

ThrottlerMode controls the behavior a throttler.

const (
	// Disabled removes all throttling behavior for the server.
	Disabled ThrottlerMode = iota
	// LogOnly enables an advisory mode for throttling behavior on the server.
	LogOnly
	// Enforce turns the throttling behavior on for the server.
	Enforce
)