balancer

Documentation

Overview

package balancer implements a custom gRPC load balancer.

Similarly to gRPC's built-in "pick_first" balancer, our balancer will pin the client to a single connection/server. However, it will switch servers as soon as an RPC error occurs (e.g. if the client has exhausted its rate limit on that server). It also provides a method that will be called periodically by the Consul router to randomize the connection priorities to rebalance load.

Our balancer aims to keep exactly one TCP connection (to the current server) open at a time. This is different to gRPC's "round_robin" and "base" balancers which connect to *all* resolved addresses up-front so that you can quickly cycle between them - which we want to avoid because of the overhead on the servers. It's also slightly different to gRPC's "pick_first" balancer which will attempt to remain connected to the same server as long its address is returned by the resolver - we previously had to work around this behavior in order to shuffle the servers, which had some unfortunate side effects as documented in this issue: https://github.com/hashicorp/consul/issues/10603.

If a server is in a perpetually bad state, the balancer's standard error handling will steer away from it but it will *not* be removed from the set and will remain in a TRANSIENT_FAILURE state to possibly be retried in the future. It is expected that Consul's router will remove servers from the resolver which have been network partitioned etc.

Quick primer on how gRPC's different components work together:

• Targets (e.g. consul://.../server.dc1) represent endpoints/collections of hosts. They're what you pass as the first argument to grpc.Dial.

• ClientConns represent logical connections to targets. Each ClientConn may have many SubConns (and therefore TCP connections to different hosts).

• SubConns represent connections to a single host. They map 1:1 with TCP connections (that's actually a bit of a lie, but true for our purposes).

• Resolvers are responsible for turning Targets into sets of addresses (e.g. via DNS resolution) and updating the ClientConn when they change. They map 1:1 with ClientConns. gRPC creates them for a ClientConn using the builder registered for the Target's scheme (i.e. the protocol part of the URL).

• Balancers are responsible for turning resolved addresses into SubConns and a Picker. They're called whenever the Resolver updates the ClientConn's state (e.g. with new addresses) or when the SubConns change state.

  Like Resolvers, they also map 1:1 with ClientConns and are created using a builder registered with a name that is specified in the "service config".

• Pickers are responsible for deciding which SubConn will be used for an RPC.

Index

• type Builder
  • func NewBuilder(name string, logger hclog.Logger) *Builder
  • func (b *Builder) Build(cc gbalancer.ClientConn, opts gbalancer.BuildOptions) gbalancer.Balancer
  • func (b *Builder) Name() string
  • func (b *Builder) Rebalance(target resolver.Target)
  • func (b *Builder) Register()

Types

type Builder

type Builder struct {
	// contains filtered or unexported fields
}

Builder implements gRPC's balancer.Builder interface to construct balancers.

func NewBuilder

func NewBuilder(name string, logger hclog.Logger) *Builder

NewBuilder constructs a new Builder with the given name.

func (*Builder) Build

func (b *Builder) Build(cc gbalancer.ClientConn, opts gbalancer.BuildOptions) gbalancer.Balancer

Build is called by gRPC (e.g. on grpc.Dial) to construct a balancer for the given ClientConn.

func (*Builder) Name

func (b *Builder) Name() string

Name implements the gRPC Balancer interface by returning its given name.

func (*Builder) Rebalance

func (b *Builder) Rebalance(target resolver.Target)

Rebalance randomizes the priority order of servers for the given target to rebalance load.

func (*Builder) Register

func (b *Builder) Register()

Register the Builder in gRPC's global registry using its given name.