throughputlb

README

gRPC Throughput Load Balancer

The gRPC throughput load balancer is a load balancer that implements the grpc.Balancer interface. It will open a configured number of connections to a single address and not allow more than a given number of concurrent requests per address.

In your code when you make a gRPC request (stream or RPC), the gRPC throughput load balancer will return the connection with the least number of active requests.

Load Balancer Lifecycle

The order gRPC calls methods on the gRPC throughput load balancer are:

1. Start
2. Notify
3. Up
4. Get

Example

lb := throughputlb.NewThroughputLoadBalancer(100, 20)

conn, err := grpc.Dial(os.Getenv("GRPC_ADDR"),
    grpc.WithBalancer(lb))
if err != nil {
    panic(err)
}

Documentation

Index

• type ThroughputLoadBalancer
  • func NewThroughputLoadBalancer(maxRequests int, numAddrs int) *ThroughputLoadBalancer
  • func (*ThroughputLoadBalancer) Close() error
  • func (lb *ThroughputLoadBalancer) Get(ctx context.Context, opts grpc.BalancerGetOptions) (grpc.Address, func(), error)
  • func (lb *ThroughputLoadBalancer) Notify() <-chan []grpc.Address
  • func (lb *ThroughputLoadBalancer) Start(target string, cfg grpc.BalancerConfig) error
  • func (lb *ThroughputLoadBalancer) Up(addr grpc.Address) func(error)

Types

type ThroughputLoadBalancer

type ThroughputLoadBalancer struct {
	// contains filtered or unexported fields
}

ThroughputLoadBalancer is a gRPC load balancer that satisfies the grpc.Balancer interface. This load balancer will open multiple connections to a single address and enfoce a maximum number of concurrent requests per connection.

func NewThroughputLoadBalancer

func NewThroughputLoadBalancer(
	maxRequests int,
	numAddrs int,
) *ThroughputLoadBalancer

NewThroughputLoadBalancer returns an initialized throughput load balancer with the given number of maxRequests per address and max number of addresses. This throughput load balancer can be passed to gRPC as a dial option.

lb := throughputlb.NewThroughputLoadBalancer(100, 20)
conn, err := grpc.Dial("localhost:8080", grpc.WithBalancer(lb))
...

func (*ThroughputLoadBalancer) Close

func (*ThroughputLoadBalancer) Close() error

Close is called by gRPC when a connection is being closed. For this load balancer it is a no-op.

func (*ThroughputLoadBalancer) Get

func (lb *ThroughputLoadBalancer) Get(ctx context.Context, opts grpc.BalancerGetOptions) (grpc.Address, func(), error)

Get is called by gRPC when your client initiates a request. Up will return find the least used address and return that for a client to use. If BlockingWait is true on the BalancerGetOptions this method will block until an address is available, otherwise an error will be returned if no address have any available capacity.

func (*ThroughputLoadBalancer) Notify

func (lb *ThroughputLoadBalancer) Notify() <-chan []grpc.Address

Notify returns a channel that is used to notify gRPC of changes to the address set.

func (*ThroughputLoadBalancer) Start

func (lb *ThroughputLoadBalancer) Start(target string, cfg grpc.BalancerConfig) error

Start is called by gRPC when dial is called to configure the load balancer with a target and a BalancerConfig. The BalancerConfig is not used by this load balancer. When start is called by gRPC the load balancer will be configured with a configured number of addresses and notify gRPC of those addresses. All of the address targets are the same with an integer as metidata to make them unique.

func (*ThroughputLoadBalancer) Up

func (lb *ThroughputLoadBalancer) Up(addr grpc.Address) func(error)

Up is called by gRPC when a connection has been established for the given address. Internally in the load balancer we will mark the address as up and it can now be used by calling Get. The function returned by this method will be called by gRPC when the address goes down and the address is no longer usable.