lbv2

Documentation

Overview

Modular building blocks for a traffic control engine.

Index

• Variables
• type LoadBalancer
  • func New(name string) *LoadBalancer
  • func (l *LoadBalancer) AddFilter(f NodeFilter)
  • func (l *LoadBalancer) Choose(ctx RequestContext) (Node, error)
  • func (l *LoadBalancer) DisableUtilizationPredictors()
  • func (l *LoadBalancer) GetPredictor(dimension int) Predictor
  • func (l *LoadBalancer) ServeHTTP(w http.ResponseWriter, r *http.Request)
  • func (l *LoadBalancer) SetPolicy(p Policy)
  • func (l *LoadBalancer) Update(nodes NodeList)
• type Node
• type NodeFilter
  • func NewActiveNodesFilter() NodeFilter
  • func NewCapacityAvailableFilter(pred Predictor) NodeFilter
  • func NewCapacityAvailableScorer(pred Predictor) NodeFilter
  • func NodeScorerFilter(s NodeScorer) NodeFilter
• type NodeList
• type NodeScore
  • func (c NodeScore) SetScore(w float64) NodeScore
• type NodeScorer
• type NodeScorerFunc
  • func (f NodeScorerFunc) Score(ctx RequestContext, n Node) float64
• type NodeUtilization
• type Policy
• type PolicyFunc
  • func (f PolicyFunc) GetNode(wnodes []NodeScore) Node
• type Predictor
• type RequestContext

Variables

var (
	ErrNoNodes          = errors.New("no nodes are known")
	ErrAllNodesFiltered = errors.New("all nodes were filtered")
	ErrPolicy           = errors.New("nodes rejected by policy")
)

Possible error values returned by Choose.

var HighestScorePolicy = PolicyFunc(highestScorePolicyFunc)

var RandomPolicy = PolicyFunc(randomPolicyFunc)

var WeightedPolicy = PolicyFunc(weightedPolicyFunc)

Types

type LoadBalancer

type LoadBalancer struct {
	// contains filtered or unexported fields
}

The LoadBalancer object makes decisions about where traffic should go. It contains the list of active backend nodes (updated asynchronously by calling Update), and a list of filters and policies used to select a backend for every incoming request.

Filters look at each available backend node and assign them a score, possibly depending on the incoming request. The policy then picks a backend among the available nodes based on their score. By convention, it is possible to remove a node from the list of candidates by setting its score to zero.

Combinations of simple filters can implement relatively complex traffic control behaviors: from straightforward round-robin to geoip latency minimization to capacity-aware, utilization-based load balancing.

The LoadBalancer keeps track of how many requests were sent to each backend node, to compute an up-to-date estimation of its current utilization. This should help mitigate "thundering herd" and "laser death ray" scenarios caused by delays in the utilization feedback loop.

The computational model makes some generic assumptions about incoming traffic, the results will be better the more real traffic actually matches these assumptions. First, all requests are assumed to be identical, utilization-wise: there is a single query cost metric. While this implies that the accuracy of the cost estimation for each specific query may be low, the effect evens out with large numbers as long as the statistical distribution of request types varies little over time. Secondly, since the estimation logic is local to each frontend, the model assumes that each frontend receives an equal share of incoming traffic.

func New

func New(name string) *LoadBalancer

New returns a new LoadBalancer with no filters or policy set. The node name is only used to set a label on the associated Prometheus collector (in case multiple LoadBalancers are created within the same process).

func (*LoadBalancer) AddFilter

func (l *LoadBalancer) AddFilter(f NodeFilter)

AddFilters appends a filter to the filter list.

func (*LoadBalancer) Choose

func (l *LoadBalancer) Choose(ctx RequestContext) (Node, error)

Choose a node according to the specified policies.

func (*LoadBalancer) DisableUtilizationPredictors

func (l *LoadBalancer) DisableUtilizationPredictors()

Only use for testing purposes.

func (*LoadBalancer) GetPredictor

func (l *LoadBalancer) GetPredictor(dimension int) Predictor

GetPredictor returns an utilization predictor for the specified dimension.

func (*LoadBalancer) ServeHTTP

func (l *LoadBalancer) ServeHTTP(w http.ResponseWriter, r *http.Request)

func (*LoadBalancer) SetPolicy

func (l *LoadBalancer) SetPolicy(p Policy)

SetPolicy sets the node selection policy.

func (*LoadBalancer) Update

func (l *LoadBalancer) Update(nodes NodeList)

Update the set of known nodes. The new utilization numbers will be used to calibrate the utilization predictors.

type Node

type Node interface {
	// Name of the node (unique identifier).
	Name() string

	// Current utilization for the specified dimension.
	Utilization(int) NodeUtilization
}

type NodeFilter

type NodeFilter interface {
	Filter(RequestContext, []NodeScore) []NodeScore
}

Node filters operate on the list of available nodes (and associated weights), possibly modifying it in-place. They can be composed out of simpler parts, like individual nodeScorers wrapped by an nodeScorerFilter (when no request-specific initialization step is required).

func NewActiveNodesFilter

func NewActiveNodesFilter() NodeFilter

func NewCapacityAvailableFilter

func NewCapacityAvailableFilter(pred Predictor) NodeFilter

func NewCapacityAvailableScorer

func NewCapacityAvailableScorer(pred Predictor) NodeFilter

func NodeScorerFilter

func NodeScorerFilter(s NodeScorer) NodeFilter

type NodeList

type NodeList interface {
	Len() int
	Get(int) Node
}

NodeList is a wrapper for a container of Nodes.

type NodeScore

type NodeScore struct {
	Score float64
	Node  Node
}

func (NodeScore) SetScore

func (c NodeScore) SetScore(w float64) NodeScore

SetScore applies a multiplier to the current score and returns the modified object.

type NodeScorer

type NodeScorer interface {
	Score(RequestContext, Node) float64
}

Score a single node.

type NodeScorerFunc

type NodeScorerFunc func(RequestContext, Node) float64

func (NodeScorerFunc) Score

func (f NodeScorerFunc) Score(ctx RequestContext, n Node) float64

type NodeUtilization

type NodeUtilization struct {
	// Utilization is a number between 0 and 1.
	Utilization float64

	// Some request-related metric. It doesn't really matter
	// whether it is a counter or a gauge, as long as it is
	// roughly proportional to the utilization.
	Requests int
}

Node utilization along a specific dimension. Utilization is treated as a vector, the LoadBalancer is opaque to the actual meaning of the dimensions used (though it makes sense for Requests to be the same across dimensions).

type Policy

type Policy interface {
	GetNode([]NodeScore) Node
}

type PolicyFunc

type PolicyFunc func([]NodeScore) Node

func (PolicyFunc) GetNode

func (f PolicyFunc) GetNode(wnodes []NodeScore) Node

type Predictor

type Predictor interface {
	Utilization(Node) float64
}

A Predictor estimates utilization along a specific dimension using a (relatively simple) cost query model.

type RequestContext

type RequestContext interface {
	RemoteAddr() net.IP
}

Provides contextual information on the incoming request.