rpcproxy

Documentation

Overview

Package rpcproxy provides a proxy interface to Nomad Servers. The RPCProxy periodically shuffles which server a Nomad Client communicates with in order to redistribute load across Nomad Servers. Nomad Servers that fail an RPC request are automatically cycled to the end of the list until the server list is reshuffled.

The rpcproxy package does not provide any external API guarantees and should be called only by `hashicorp/nomad`.

Index

• type EndpointKey
  • func (k *EndpointKey) Equal(x *EndpointKey) bool
• type NomadConfigInfo
• type Pinger
• type RPCProxy
  • func NewRPCProxy(logger *log.Logger, shutdownCh chan struct{}, configInfo NomadConfigInfo, ...) *RPCProxy
  • func (p *RPCProxy) AddPrimaryServer(rpcAddr string) *ServerEndpoint
  • func (p *RPCProxy) FindServer() *ServerEndpoint
  • func (p *RPCProxy) LeaderAddr() string
  • func (p *RPCProxy) NotifyFailedServer(s *ServerEndpoint)
  • func (p *RPCProxy) NumNodes() int
  • func (p *RPCProxy) NumServers() int
  • func (p *RPCProxy) RebalanceServers()
  • func (p *RPCProxy) RefreshServerLists(servers []*structs.NodeServerInfo, numNodes int32, leaderRPCAddr string) error
  • func (p *RPCProxy) RemoveServer(s *ServerEndpoint)
  • func (p *RPCProxy) ResetRebalanceTimer()
  • func (p *RPCProxy) Run()
  • func (p *RPCProxy) ServerRPCAddrs() []string
  • func (p *RPCProxy) SetBackupServers(addrs []string) error
• type ServerEndpoint
  • func NewServerEndpoint(name string) (*ServerEndpoint, error)
  • func (s *ServerEndpoint) Key() *EndpointKey
  • func (s *ServerEndpoint) String() string

Types

type EndpointKey

type EndpointKey struct {
	// contains filtered or unexported fields
}

EndpointKey is used in maps and for equality tests. A key is based on endpoints.

func (*EndpointKey) Equal

func (k *EndpointKey) Equal(x *EndpointKey) bool

Equal compares two EndpointKey objects

type NomadConfigInfo

type NomadConfigInfo interface {
	Datacenter() string
	RPCMajorVersion() int
	RPCMinorVersion() int
	Region() string
}

NomadConfigInfo is an interface wrapper around this Nomad Agent's configuration to prevents a cyclic import dependency.

type Pinger

type Pinger interface {
	PingNomadServer(region string, apiMajorVersion int, s *ServerEndpoint) (bool, error)
}

Pinger is an interface wrapping client.ConnPool to prevent a cyclic import dependency

type RPCProxy

type RPCProxy struct {
	// contains filtered or unexported fields
}

RPCProxy is the manager type responsible for returning and managing Nomad addresses.

func NewRPCProxy

func NewRPCProxy(logger *log.Logger, shutdownCh chan struct{}, configInfo NomadConfigInfo, connPoolPinger Pinger) *RPCProxy

NewRPCProxy is the only way to safely create a new RPCProxy.

func (*RPCProxy) AddPrimaryServer

func (p *RPCProxy) AddPrimaryServer(rpcAddr string) *ServerEndpoint

AddPrimaryServer takes the RPC address of a Nomad server, creates a new endpoint, and adds it to both the primaryServers list and the active serverList used in the RPC Proxy. If the endpoint is not known by the RPCProxy, appends the endpoint to the list. The new endpoint will begin seeing use after the rebalance timer fires (or enough servers fail organically). Any values in the primary server list are overridden by the next successful heartbeat.

func (*RPCProxy) FindServer

func (p *RPCProxy) FindServer() *ServerEndpoint

FindServer takes out an internal "read lock" and searches through the list of servers to find a "healthy" server. If the server is actually unhealthy, we rely on heartbeats to detect this and remove the node from the server list. If the server at the front of the list has failed or fails during an RPC call, it is rotated to the end of the list. If there are no servers available, return nil.

func (*RPCProxy) LeaderAddr

func (p *RPCProxy) LeaderAddr() string

LeaderAddr returns the current leader address. If an empty string, then the Nomad Server for this Nomad Agent is in the minority or the Nomad Servers are in the middle of an election.

func (*RPCProxy) NotifyFailedServer

func (p *RPCProxy) NotifyFailedServer(s *ServerEndpoint)

NotifyFailedServer marks the passed in server as "failed" by rotating it to the end of the server list.

func (*RPCProxy) NumNodes

func (p *RPCProxy) NumNodes() int

NumNodes returns the estimated number of nodes according to the last Nomad Heartbeat.

func (*RPCProxy) NumServers

func (p *RPCProxy) NumServers() int

NumServers takes out an internal "read lock" and returns the number of servers. numServers includes both healthy and unhealthy servers.

func (*RPCProxy) RebalanceServers

func (p *RPCProxy) RebalanceServers()

RebalanceServers shuffles the list of servers on this agent. The server at the front of the list is selected for the next RPC. RPC calls that fail for a particular server are rotated to the end of the list. This method reshuffles the list periodically in order to redistribute work across all known Nomad servers (i.e. guarantee that the order of servers in the server list is not positively correlated with the age of a server in the Nomad cluster). Periodically shuffling the server list prevents long-lived clients from fixating on long-lived servers.

Unhealthy servers are removed from the server list during the next client heartbeat. Before the newly shuffled server list is saved, the new remote endpoint is tested to ensure its responsive.

func (*RPCProxy) RefreshServerLists

func (p *RPCProxy) RefreshServerLists(servers []*structs.NodeServerInfo, numNodes int32, leaderRPCAddr string) error

RefreshServerLists is called when the Client receives an update from a Nomad Server. The response from Nomad Client Heartbeats contain a list of Nomad Servers that the Nomad Client should use for RPC requests. RefreshServerLists does not rebalance its serverLists (that is handled elsewhere via a periodic timer). New Nomad Servers learned via the heartbeat are appended to the RPCProxy's activated serverList. Servers that are no longer present in the Heartbeat are removed immediately from all server lists. Nomad Servers speaking a newer major or minor API version are filtered from the serverList.

func (*RPCProxy) RemoveServer

func (p *RPCProxy) RemoveServer(s *ServerEndpoint)

RemoveServer takes out an internal write lock and removes a server from the activated server list.

func (*RPCProxy) ResetRebalanceTimer

func (p *RPCProxy) ResetRebalanceTimer()

ResetRebalanceTimer resets the rebalance timer. This method exists for testing and should not be used directly.

func (*RPCProxy) Run

func (p *RPCProxy) Run()

Run is used to start and manage the task of automatically shuffling and rebalancing the list of Nomad servers. This maintenance only happens periodically based on the expiration of the timer. Failed servers are automatically cycled to the end of the list. New servers are appended to the list. The order of the server list must be shuffled periodically to distribute load across all known and available Nomad servers.

func (*RPCProxy) ServerRPCAddrs

func (p *RPCProxy) ServerRPCAddrs() []string

ServerRPCAddrs returns one RPC Address per server

func (*RPCProxy) SetBackupServers

func (p *RPCProxy) SetBackupServers(addrs []string) error

SetBackupServers sets a list of Nomad Servers to be used in the event that the Nomad Agent lost contact with the list of Nomad Servers provided via the Nomad Agent's heartbeat. If available, the backup servers are populated via Consul.

type ServerEndpoint

type ServerEndpoint struct {
	// Name is the unique lookup key for a Server instance
	Name string
	Host string
	Port string
	Addr net.Addr
}

ServerEndpoint contains the address information for to connect to a Nomad server.

TODO(sean@): Server is stubbed out so that in the future it can hold a reference to Node (and ultimately Node.ID).

func NewServerEndpoint

func NewServerEndpoint(name string) (*ServerEndpoint, error)

NewServerEndpoint creates a new Server instance with a resolvable endpoint. `name` can be either an IP address or a DNS name. If `name` is a DNS name, it must be resolvable to an IP address (most inputs are IP addresses, not DNS names, but both work equally well when the name is resolvable).

func (*ServerEndpoint) Key

func (s *ServerEndpoint) Key() *EndpointKey

Key returns the corresponding Key

func (*ServerEndpoint) String

func (s *ServerEndpoint) String() string

String returns a string representation of Server