consistent

Documentation

Overview

Package consistent provides a consistent hashing function.

Consistent hashing is often used to distribute requests to a changing set of servers. For example, say you have some cache servers cacheA, cacheB, and cacheC. You want to decide which cache server to use to look up information on a user.

You could use a typical hash table and hash the user id to one of cacheA, cacheB, or cacheC. But with a typical hash table, if you add or remove a server, almost all keys will get remapped to different results, which basically could bring your service to a grinding halt while the caches get rebuilt.

With a consistent hash, adding or removing a server drastically reduces the number of keys that get remapped.

Read more about consistent hashing on wikipedia: http://en.wikipedia.org/wiki/Consistent_hashing

Index

• Variables
• type Consistent
  • func InitConsistent(storePath string, persistImpl Persistence, initBP bool) (c *Consistent, err error)
  • func (c *Consistent) Add(node proto.Node) (err error)
  • func (c *Consistent) AddCache(node proto.Node)
  • func (c *Consistent) GetNeighbor(name string) (proto.Node, error)
  • func (c *Consistent) GetNeighbors(name string, n int) ([]proto.Node, error)
  • func (c *Consistent) GetNeighborsEx(name string, n int, roles proto.ServerRoles) ([]proto.Node, error)
  • func (c *Consistent) GetNode(name string) (*proto.Node, error)
  • func (c *Consistent) GetTwoNeighbors(name string) (proto.Node, proto.Node, error)
  • func (c *Consistent) Remove(nodeID proto.NodeID) (err error)
  • func (c *Consistent) RemoveCache(nodeID proto.NodeID)
  • func (c *Consistent) ResetCache()
  • func (c *Consistent) Set(nodes []proto.Node) (err error)
• type KMSStorage
  • func (s *KMSStorage) DelNode(nodeID proto.NodeID) (err error)
  • func (s *KMSStorage) GetAllNodeInfo() (nodes []proto.Node, err error)
  • func (s *KMSStorage) Init(storePath string, initNodes []proto.Node) (err error)
  • func (s *KMSStorage) Reset() (err error)
  • func (s *KMSStorage) SetNode(node *proto.Node) (err error)
• type NodeKeys
  • func (x NodeKeys) Len() int
  • func (x NodeKeys) Less(i, j int) bool
  • func (x NodeKeys) Swap(i, j int)
• type Persistence

Variables

var (
	// ErrEmptyCircle is the error returned when trying to get an node when nothing has been added to hash.
	ErrEmptyCircle = errors.New("empty circle")
	// ErrKeyNotFound is the error returned when no key in circle
	ErrKeyNotFound = errors.New("node key not found")
)

Types

type Consistent

type Consistent struct {
	NumberOfReplicas int

	sync.RWMutex
	// contains filtered or unexported fields
}

Consistent holds the information about the members of the consistent hash circle.

func InitConsistent

func InitConsistent(storePath string, persistImpl Persistence, initBP bool) (c *Consistent, err error)

InitConsistent creates a new Consistent object with a default setting of 20 replicas for each entry.

func (*Consistent) Add

func (c *Consistent) Add(node proto.Node) (err error)

Add inserts a string node in the consistent hash.

func (*Consistent) AddCache

func (c *Consistent) AddCache(node proto.Node)

AddCache only adds c.circle skips persist.

func (*Consistent) GetNeighbor

func (c *Consistent) GetNeighbor(name string) (proto.Node, error)

GetNeighbor returns an node close to where name hashes to in the circle.

func (*Consistent) GetNeighbors

func (c *Consistent) GetNeighbors(name string, n int) ([]proto.Node, error)

GetNeighbors returns the N closest distinct nodes to the name input in the circle.

func (*Consistent) GetNeighborsEx

func (c *Consistent) GetNeighborsEx(name string, n int, roles proto.ServerRoles) ([]proto.Node, error)

GetNeighborsEx returns the N closest distinct nodes to the name input in the circle.

func (*Consistent) GetNode

func (c *Consistent) GetNode(name string) (*proto.Node, error)

GetNode returns an node by its node id.

func (*Consistent) GetTwoNeighbors

func (c *Consistent) GetTwoNeighbors(name string) (proto.Node, proto.Node, error)

GetTwoNeighbors returns the two closest distinct nodes to the name input in the circle.

func (*Consistent) Remove

func (c *Consistent) Remove(nodeID proto.NodeID) (err error)

Remove removes an node from the hash.

func (*Consistent) RemoveCache

func (c *Consistent) RemoveCache(nodeID proto.NodeID)

RemoveCache removes an node from the hash cache.

func (*Consistent) ResetCache

func (c *Consistent) ResetCache()

ResetCache removes all node from the hash cache.

func (*Consistent) Set

func (c *Consistent) Set(nodes []proto.Node) (err error)

Set sets all the nodes in the hash. If there are existing nodes not present in nodes, they will be removed.

type KMSStorage

type KMSStorage struct{}

KMSStorage implements Persistence.

func (*KMSStorage) DelNode

func (s *KMSStorage) DelNode(nodeID proto.NodeID) (err error)

DelNode implements Persistence interface.

func (*KMSStorage) GetAllNodeInfo

func (s *KMSStorage) GetAllNodeInfo() (nodes []proto.Node, err error)

GetAllNodeInfo implements Persistence interface.

func (*KMSStorage) Init

func (s *KMSStorage) Init(storePath string, initNodes []proto.Node) (err error)

Init implements Persistence interface.

func (*KMSStorage) Reset

func (s *KMSStorage) Reset() (err error)

Reset implements Persistence interface.

func (*KMSStorage) SetNode

func (s *KMSStorage) SetNode(node *proto.Node) (err error)

SetNode implements Persistence interface.

type NodeKeys

type NodeKeys []proto.NodeKey

NodeKeys is NodeKey array.

func (NodeKeys) Len

func (x NodeKeys) Len() int

Len returns the length of the uints array.

func (NodeKeys) Less

func (x NodeKeys) Less(i, j int) bool

Less returns true if node i is less than node j.

func (NodeKeys) Swap

func (x NodeKeys) Swap(i, j int)

Swap exchanges nodes i and j.

type Persistence

type Persistence interface {
	Init(storePath string, initNode []proto.Node) (err error)
	SetNode(node *proto.Node) (err error)
	DelNode(nodeID proto.NodeID) (err error)
	Reset() error
	GetAllNodeInfo() (nodes []proto.Node, err error)
}

Persistence is the interface for consistent persistence.