balance

README

Client-side L7 load balancing for Kubernetes Services

This repository implements in-process L7 load balancing algorithms for Kubernetes processes.

• Go library for your Go microservices.
• Uses the Kubernetes built-in Service discovery mechanism to find out which endpoints requests can be routed to.
• Offers an abstraction to implement a variety of L7 load balancing mechanisms.
• Implements affinity and non-affinity algorithms.
• List of supported algorithms:
  • Consistent hashing
  • Consistent hashing with bounded Loads

Using the library

XXX

Sure but, why?

The Kubernetes Service abstraction is implemented as a L4 load balancer using iptables DNAT on each node to route the Service Virtual IPs to pod IPs.

Unfortunately, this leaves user without much control about where the requests are going. Worse there are cases where L4 fails entirely to load balance anything: think about HTTP/2 multiplexing all requests onto the same connection!

My Kubecon talk (Copenhagen, May 2018), The Untapped Power of Services - L7 Load Balancing Without a Service Mesh gives a lot more details about the reasons I've started this work.

The talk is available in video format:

Documentation

Index

• type Algorithm
  • func WithServiceFallback(next Algorithm, service *Service) Algorithm
• type Consistent
  • func NewConsistent(config ConsistentConfig) *Consistent
  • func (c *Consistent) AddEndpoints(endpoints ...Endpoint)
  • func (c *Consistent) Get(keys ...string) Endpoint
  • func (c *Consistent) Put(endpoint Endpoint)
  • func (c *Consistent) RemoveEndpoints(endpoints ...Endpoint)
• type ConsistentConfig
• type Endpoint
• type EndpointSet
• type EndpointWatcher
  • func (w *EndpointWatcher) Start(close <-chan interface{})
• type Fallback
• type Hash
• type LoadBalancer
  • func NewLoadBalancer(service string, algo Algorithm, options LoadBalancerOptions) *LoadBalancer
  • func (lb *LoadBalancer) Get(key ...string) Endpoint
  • func (lb *LoadBalancer) Put(endpoint Endpoint)
  • func (lb *LoadBalancer) Start(stop <-chan interface{}) error
• type LoadBalancerOptions
• type Service
  • func NewServiceFromString(s string) (*Service, error)
  • func (s *Service) String() string

Types

type Algorithm

type Algorithm interface {
	EndpointSet
	// Get returns the Service Endpoint to use for the next request.
	//
	// Get is called when wanting to send a request to a Service. It returns the
	// Endpoint that request should be directed to. When used with an affinity load
	// balancing scheme, the affinity key needs to be given as argument to this
	// function.
	//
	// When used with an affinity load balancing scheme, Get will panic if no key
	// or more than one key is given. The variadic form is only used for
	// aesthetics, ie. being able to use Get() which non-affinity load balancers.
	//
	// Get may return nil when the load balancer is not aware of any
	// Endpoint.
	Get(key ...string) Endpoint
	// Put releases the Endpoint when it has finished processing the request.
	Put(endpoint Endpoint)
}

Algorithm is an interface abstracting load balancing algorithms.

func WithServiceFallback

func WithServiceFallback(next Algorithm, service *Service) Algorithm

WithServiceFallback wraps a load balancer, falling back to the service DNS name when there's no available endpoint to serve the request.

type Consistent

type Consistent struct {
	sync.Mutex
	// contains filtered or unexported fields
}

Consistent implements a consistent hashing algorithm.

func NewConsistent

func NewConsistent(config ConsistentConfig) *Consistent

NewConsistent creates a new Consistent object.

func (*Consistent) AddEndpoints

func (c *Consistent) AddEndpoints(endpoints ...Endpoint)

AddEndpoints implements EndpointSet

func (*Consistent) Get

func (c *Consistent) Get(keys ...string) Endpoint

Get implements Algorithm.

func (*Consistent) Put

func (c *Consistent) Put(endpoint Endpoint)

Put implements Algorithm.

func (*Consistent) RemoveEndpoints

func (c *Consistent) RemoveEndpoints(endpoints ...Endpoint)

RemoveEndpoints implements EndpointSet

type ConsistentConfig

type ConsistentConfig struct {
	// Name used for metrics and reporting
	Name string

	// Hash is the hashing function used for hash Endpoints and keys onto the hash
	// ring. You may want to use an interesting hash function like xxHash.
	// Defaults to CRC32.
	Hash Hash

	// ReplicationCount controls the number of virtual nodes to add to the hash
	// ring for each Endpoint.
	// Defaults to 128.
	ReplicationCount int

	// LoadFactor controls the maximum load of any endpoint. The load is defined as
	// the number of requests currently being handled by an endpoint. When set to a
	// value > 1.0, Consistent implements the bounded loads variant of consistent
	// hashing and ensures no endpoint has a load > LoadFactor * averageLoad.
	//
	// See https://arxiv.org/abs/1608.01350 for details about consistent hashing
	// with bounded loads.
	LoadFactor float64
}

ConsistentConfig holds the configuration for the Consistent hash algorithm.

type Endpoint

type Endpoint interface {
	Key() string
}

Endpoint is service endpoint.

type EndpointSet

type EndpointSet interface {
	AddEndpoints(...Endpoint)
	RemoveEndpoints(...Endpoint)
}

EndpointSet holds a set of Endpoints.

type EndpointWatcher

type EndpointWatcher struct {
	Client   client.Interface
	Service  Service
	Receiver EndpointSet
	// contains filtered or unexported fields
}

EndpointWatcher load balances requests.

func (*EndpointWatcher) Start

func (w *EndpointWatcher) Start(close <-chan interface{})

Start will start an internal goroutine that watches the kubernetes service and notify the Receiver of Endpoints changes. close is a channel the caller can close to terminate this goroutine.

type Fallback

type Fallback string

Fallback is a fallback strategy.

const (
	// FallbackNone disables fallback strategies. Get will return nil when the
	// Service has no endpoint.
	FallbackNone Fallback = "none"
	// FallbackService is a fallback strategy that will make the load balancer use
	// the Service DNS name.
	FallbackService Fallback = "service"
)

type Hash

type Hash func(data []byte) uint32

Hash is a 32-bit hash function.

type LoadBalancer

type LoadBalancer struct {
	// contains filtered or unexported fields
}

LoadBalancer is a Kubernetes Service load balancer.

func NewLoadBalancer

func NewLoadBalancer(service string, algo Algorithm, options LoadBalancerOptions) *LoadBalancer

NewLoadBalancer creates a new load balancer for service.

func (*LoadBalancer) Get

func (lb *LoadBalancer) Get(key ...string) Endpoint

Get returns the Service Endpoint to use for the next request.

Get is called when wanting to send a request to a Service. It returns the Endpoint that request should be directed to. When used with an affinity load balancing scheme, the affinity key needs to be given as argument to this function.

When used with an affinity load balancing scheme, Get will panic if no key or more than one key is given. The variadic form is only used for aesthetics, ie. being able to use Get() which non-affinity load balancers.

Get may return nil when the load balancer is not aware of any Endpoint.

func (*LoadBalancer) Put

func (lb *LoadBalancer) Put(endpoint Endpoint)

Put releases the Endpoint when it has finished processing the request.

func (*LoadBalancer) Start

func (lb *LoadBalancer) Start(stop <-chan interface{}) error

Start initializes the load balancer. Start must be called before any other function.

The load balancer object may create goroutines or other precious resources. Stopping a load balancer is done by closing the stop channel.

type LoadBalancerOptions

type LoadBalancerOptions struct {
	// Fallback defines the strategy to adopt when the load balancer had no
	// endpoint. If not specified, defaults to FallbackService.
	Fallback Fallback
}

LoadBalancerOptions are configuration parameters.

type Service

type Service struct {
	Namespace, Name, Port string
}

Service describes a Kubernetes service. Port can be a named port or the port number.

func NewServiceFromString

func NewServiceFromString(s string) (*Service, error)

NewServiceFromString parses the kubernetes Service hostname and port and returns a Service object.

func (*Service) String

func (s *Service) String() string