adaptive

Documentation

Overview

Package adaptive provides functionality for adaptive client-side throttling.

Index

• type Throttler
  • func New() *Throttler
  • func (t *Throttler) RegisterBackendResponse(throttled bool)
  • func (t *Throttler) ShouldThrottle() bool

Types

type Throttler

type Throttler struct {
	// contains filtered or unexported fields
}

Throttler implements a client-side throttling recommendation system. All methods are safe for concurrent use by multiple goroutines.

The throttler has the following knobs for which we will use defaults for now. If there is a need to make them configurable at a later point in time, support for the same will be added.

• Duration: amount of recent history that will be taken into account for making client-side throttling decisions. A default of 30 seconds is used.
• Bins: number of bins to be used for bucketing historical data. A default of 100 is used.
• RatioForAccepts: ratio by which accepts are multiplied, typically a value slightly larger than 1.0. This is used to make the throttler behave as if the backend had accepted more requests than it actually has, which lets us err on the side of sending to the backend more requests than we think it will accept for the sake of speeding up the propagation of state. A default of 2.0 is used.
• RequestsPadding: is used to decrease the (client-side) throttling probability in the low QPS regime (to speed up propagation of state), as well as to safeguard against hitting a client-side throttling probability of 100%. The weight of this value decreases as the number of requests in recent history grows. A default of 8 is used.

The adaptive throttler attempts to estimate the probability that a request will be throttled using recent history. Server requests (both throttled and accepted) are registered with the throttler (via the RegisterBackendResponse method), which then recommends client-side throttling (via the ShouldThrottle method) with probability given by: (requests - RatioForAccepts * accepts) / (requests + RequestsPadding)

func New

func New() *Throttler

New initializes a new adaptive throttler with the default values.

func (*Throttler) RegisterBackendResponse

func (t *Throttler) RegisterBackendResponse(throttled bool)

RegisterBackendResponse registers a response received from the backend for a request allowed by ShouldThrottle. This should be called for every response received from the backend (i.e., once for each request for which ShouldThrottle returned false).

func (*Throttler) ShouldThrottle

func (t *Throttler) ShouldThrottle() bool

ShouldThrottle returns a probabilistic estimate of whether the server would throttle the next request. This should be called for every request before allowing it to hit the network. If the returned value is true, the request should be aborted immediately (as if it had been throttled by the server).