swim

package
v0.0.0-...-6475220 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 25, 2018 License: MIT Imports: 25 Imported by: 0

Documentation

Index

Examples

Constants

View Source
const (
	// Alive is the member "alive" state
	Alive = "alive"

	// Suspect is the member "suspect" state
	Suspect = "suspect"

	// Faulty is the member "faulty" state
	Faulty = "faulty"

	// Leave is the member "leave" state
	Leave = "leave"

	// Tombstone is the member "tombstone" state
	Tombstone = "tombstone"
)

Variables

View Source
var (
	// DefaultLabelOptions contain the default values to be used to limit the
	// amount of data being gossipped over the network. The defaults have been
	// chosen with the following assumptions.
	// 1. 1000 node cluster
	// 2. Every byte available is used (yes, characters are not bytes but these
	//    are ballpark figures to protect developers)
	// 3. Worst case would have continues fullsync's, meaning the complete
	//    memberlist being sent over the wire 5 times a second.
	// When all contiditions are met the Labels would add the following load
	// (non-compressed) to the network:
	//    (32+128)*5*1000*5*8 = ~32mbit/s
	DefaultLabelOptions = LabelOptions{
		KeySize:   32,
		ValueSize: 128,
		Count:     5,
	}

	// ErrLabelSizeExceeded indicates that an operation on labels would exceed
	// the configured size limits on labels. This is to prevent applications
	// bloating the gossip protocol with too much data. Ringpop can be
	// configured with the settings to control the amount of data that can be
	// put in a nodes labels.
	ErrLabelSizeExceeded = errors.New("label operation exceeds configured label limits")

	// ErrLabelInternalKey is an error that is returned when an application
	// tries to set a label in the internal namespace that is used by ringpop
	ErrLabelInternalKey = errors.New("label can't be altered by application because it is in the internal ringpop namespace")
)
View Source
var (
	// ErrDuplicateHook is returned when a hook that has already been registered
	// is registered again
	ErrDuplicateHook = errors.New("hook already registered")

	// ErrSelfEvictionInProgress is returned when ringpop is already in the process
	// of evicting itself from the network.
	ErrSelfEvictionInProgress = errors.New("ringpop is already executing a self-eviction")
)
View Source
var (
	// ErrNodeNotReady is returned when a remote request is being handled while the node is not yet ready
	ErrNodeNotReady = errors.New("node is not ready to handle requests")
)

Functions

func AttemptHeal

func AttemptHeal(node *Node, target string) ([]string, error)

AttemptHeal attempts to heal a partition between the node and the target.

Be mindfull that calling this function will not result in a heal when there are nodes that need to be reincarated to take precedence over the faulty declarations that occur during a network partition. A cluster may therefore need multiple calls to this function with some time in between to heal.

Check out ringpop-common/docs for a full description of the algorithm.

func MemberMatchesPredicates

func MemberMatchesPredicates(member Member, predicates ...MemberPredicate) bool

MemberMatchesPredicates can take multiple predicates and test them against a member returning if the member satisfies all the predicates. This means that if one test fails it will stop executing and return with false.

Types

type AddJoinListEvent

type AddJoinListEvent struct {
	Duration time.Duration `json:"duration"`
}

AddJoinListEvent is sent when a join list is added to the membership

type AttemptHealEvent

type AttemptHealEvent struct{}

AttemptHealEvent is sent when the healer is triggered

type BootstrapOptions

type BootstrapOptions struct {
	// The DiscoverProvider resolves a list of bootstrap hosts.
	DiscoverProvider discovery.DiscoverProvider

	// Whether or not gossip should be started immediately after a successful
	// bootstrap.
	Stopped bool

	// Amount of time before individual join requests time out.
	JoinTimeout time.Duration

	// Minimum number of nodes to join to satisfy a bootstrap.
	JoinSize int

	// Maximum time to attempt joins before the entire bootstrap process times
	// out.
	MaxJoinDuration time.Duration

	// A higher ParallelismFactor increases the number of nodes that a
	// bootstrapping node will attempt to reach out to in order to satisfy
	// `JoinSize` (the number of nodes that will be contacted at a time is
	// `ParallelismFactor * JoinSize`).
	ParallelismFactor int
}

BootstrapOptions is a configuration struct passed to Node.Bootstrap.

type Change

type Change struct {
	Source            string            `json:"source"`
	SourceIncarnation int64             `json:"sourceIncarnationNumber"`
	Address           string            `json:"address"`
	Incarnation       int64             `json:"incarnationNumber"`
	Status            string            `json:"status"`
	Tombstone         bool              `json:"tombstone,omitempty"`
	Labels            map[string]string `json:"labels,omitempty"`
	// Use util.Timestamp for bi-direction binding to time encoded as
	// integer Unix timestamp in JSON
	Timestamp util.Timestamp `json:"timestamp"`
}

A Change is a change a member to be applied

type ChangeFilteredEvent

type ChangeFilteredEvent struct {
	Change Change
}

A ChangeFilteredEvent is sent when a change has been filtered from the list to be disseminated

type ChangesCalculatedEvent

type ChangesCalculatedEvent struct {
	Changes []Change
}

A ChangesCalculatedEvent is sent when the disseminator generated the list of changes to send in a ping or its response

type ChecksumComputeEvent

type ChecksumComputeEvent struct {
	Duration    time.Duration `json:"duration"`
	Checksum    uint32        `json:"checksum"`
	OldChecksum uint32        `json:"oldchecksum"`
}

A ChecksumComputeEvent is sent when a the rings checksum is computed

type DiscoHealEvent

type DiscoHealEvent struct{}

DiscoHealEvent is sent when the discover provider healer attempts to heal a partition

type Endpoint

type Endpoint string

Endpoint is an identifier for an internal swim endpoint

const (
	// PingEndpoint is the identifier for /protocol/ping
	PingEndpoint Endpoint = "ping"

	// PingReqEndpoint is the identifier for /protocol/ping-req
	PingReqEndpoint Endpoint = "ping-req"
)

type FullSyncEvent

type FullSyncEvent struct {
	Remote         string `json:"remote"`
	RemoteChecksum uint32 `json:"remoteChecksum"`
}

A FullSyncEvent is sent when the disseminator's node issues changes a full sync of the memberlist

type HealResponse

type HealResponse struct {
	Targets []string `json:"targets"`
	Error   string   `json:"error"`
}

HealResponse contains a list of nodes where healing was attempted

type JoinCompleteEvent

type JoinCompleteEvent struct {
	Duration  time.Duration `json:"duration"`
	NumJoined int           `json:"numJoined"`
	Joined    []string      `json:"joined"`
}

A JoinCompleteEvent is sent when a join request to remote node successfully completes

type JoinFailedEvent

type JoinFailedEvent struct {
	Reason JoinFailedReason
	Error  error
}

A JoinFailedEvent is sent when a join request to remote node did not successfully

type JoinFailedReason

type JoinFailedReason string

JoinFailedReason indicates the reason a join failed

const (
	// Error as a JoinFailedReason indicates that the join failed because of an error
	Error JoinFailedReason = "err"

	// Destroyed as a JoinFailedReason indicates that the join failed because ringpop was destroyed during the join
	Destroyed = "destroyed"
)

type JoinReceiveEvent

type JoinReceiveEvent struct {
	Local  string `json:"local"`
	Source string `json:"source"`
}

A JoinReceiveEvent is sent when a join request is received by a node

type JoinTriesUpdateEvent

type JoinTriesUpdateEvent struct {
	Retries int
}

A JoinTriesUpdateEvent is sent when the joiner tries to join a group

type LabelMap

type LabelMap map[string]string

LabelMap is a type Used by Member to store the labels of a member. It stores string to string mappings containing user data that is gossiped around in SWIM.

type LabelOptions

type LabelOptions struct {
	// KeySize is the length a key may use at most
	KeySize int

	// ValueSize is the length a value may use at most
	ValueSize int

	// Count is the number of maximum allowed (public) labels on a node
	Count int
}

LabelOptions controlls the limits on labels. Since labels are gossiped on every ping/ping-req/fullsync we need to limit the amount of data an application stores in their labels. When needed the defaults can be overwritten during the construction of ringpop. This should be done with care to not overwhelm the network with data.

type MakeNodeStatusEvent

type MakeNodeStatusEvent struct {
	Status string
}

A MakeNodeStatusEvent is sent when Make[Status] is called on member list

type MaxPAdjustedEvent

type MaxPAdjustedEvent struct {
	OldPCount int `json:"oldPCount"`
	NewPCount int `json:"newPCount"`
}

A MaxPAdjustedEvent occurs when the disseminator adjusts the max propagation count for changes

type Member

type Member struct {
	Address     string   `json:"address"`
	Status      string   `json:"status"`
	Incarnation int64    `json:"incarnationNumber"`
	Labels      LabelMap `json:"labels,omitempty"`
}

A Member is a member in the member list

Example (ChecksumString)
var b bytes.Buffer
m := Member{
	Address:     "192.0.2.1:1234",
	Status:      Alive,
	Incarnation: 42,
}
m.checksumString(&b)
fmt.Println(b.String())
Output:

192.0.2.1:1234alive42
Example (ChecksumString_labels)
var b bytes.Buffer
m := Member{
	Address:     "192.0.2.1:1234",
	Status:      Alive,
	Incarnation: 42,
	Labels: LabelMap{
		"hello": "world",
	},
}
m.checksumString(&b)
fmt.Println(b.String())
Output:

192.0.2.1:1234alive42#labels975109414
Example (ChecksumString_multilabels)
var b bytes.Buffer
m := Member{
	Address:     "192.0.2.1:1234",
	Status:      Alive,
	Incarnation: 42,
	Labels: LabelMap{
		"hello": "world",
		"foo":   "baz",
	},
}
m.checksumString(&b)
fmt.Println(b.String())
Output:

192.0.2.1:1234alive42#labels-1625122257

func (Member) GetAddress

func (m Member) GetAddress() string

GetAddress returns the Address of a member.

func (Member) Identity

func (m Member) Identity() string

Identity returns the identity of a member. If a specific identity is not set for the member the address will be used as the identity

func (Member) Label

func (m Member) Label(key string) (value string, has bool)

Label returns the value of a label named by key. The `has` boolean indicates if the label was set on the member or not

type MemberPredicate

type MemberPredicate func(member Member) bool

MemberPredicate is a function that tests if a Member satisfies a condition. It is advised to use exported functions on Member instead of its exported fields in case we want to extract the functionality of Member to an Interface in the future. This is likely to happen if we pursue plugable membership.

func MemberWithLabelAndValue

func MemberWithLabelAndValue(key, value string) MemberPredicate

MemberWithLabelAndValue returns a predicate able to test if the value of a label on a member is equal to the provided value.

type MemberStats

type MemberStats struct {
	Checksum uint32   `json:"checksum"`
	Members  []Member `json:"members"`
}

MemberStats contains members in a memberlist and the checksum of those members

type MemberlistChangesAppliedEvent

type MemberlistChangesAppliedEvent struct {
	Changes     []Change `json:"changes"`
	OldChecksum uint32   `json:"oldChecksum"`
	NewChecksum uint32   `json:"newChecksum"`
	NumMembers  int      `json:"numMembers"`
}

A MemberlistChangesAppliedEvent contains changes that were applied to the node's memberlist as well as the previous and new checksums and the number of members in the memberlist

type MemberlistChangesReceivedEvent

type MemberlistChangesReceivedEvent struct {
	Changes []Change `json:"changes"`
}

A MemberlistChangesReceivedEvent contains changes received by the node's memberlist, pending application

type Node

type Node struct {
	events.SyncEventEmitter
	// contains filtered or unexported fields
}

A Node is a SWIM member

func NewNode

func NewNode(app, address string, channel shared.SubChannel, opts *Options) *Node

NewNode returns a new SWIM Node.

func (*Node) Address

func (n *Node) Address() string

Address returns the address of the SWIM node.

func (*Node) App

func (n *Node) App() string

App returns the Node's application name.

func (*Node) Bootstrap

func (n *Node) Bootstrap(opts *BootstrapOptions) ([]string, error)

Bootstrap joins a node to a cluster. The channel provided to the node must be listening for the bootstrap to complete.

func (*Node) CountReachableMembers

func (n *Node) CountReachableMembers(predicates ...MemberPredicate) int

CountReachableMembers returns the number of reachable members currently in this node's membership list that satisfies all predicates passed in.

func (*Node) Destroy

func (n *Node) Destroy()

Destroy stops the SWIM protocol and all sub-protocols.

func (*Node) Destroyed

func (n *Node) Destroyed() bool

Destroyed returns whether or not the node has been destroyed.

func (*Node) GetChecksum

func (n *Node) GetChecksum() uint32

GetChecksum returns the current checksum of the node's memberlist.

func (*Node) GetReachableMembers

func (n *Node) GetReachableMembers(predicates ...MemberPredicate) []Member

GetReachableMembers returns a slice of members containing only the reachable members that satisfies the predicates passed in.

func (*Node) HasChanges

func (n *Node) HasChanges() bool

HasChanges reports whether Node has changes to disseminate.

func (*Node) Incarnation

func (n *Node) Incarnation() int64

Incarnation returns the incarnation number of the Node.

func (*Node) Labels

func (n *Node) Labels() *NodeLabels

Labels returns a mutator for the labels kept on this local node. This mutator interacts with the local node and memberlist to change labels on this node and gossip those changes around.

func (*Node) MemberStats

func (n *Node) MemberStats() MemberStats

MemberStats returns the current checksum of the node's memberlist and a slice of the members in the memberlist in lexographically sorted order by address

func (*Node) ProtocolStats

func (n *Node) ProtocolStats() ProtocolStats

ProtocolStats returns stats about the node's SWIM protocol.

func (*Node) Ready

func (n *Node) Ready() bool

Ready returns whether or not the node has bootstrapped and is ready for use.

func (*Node) RegisterSelfEvictHook

func (n *Node) RegisterSelfEvictHook(hooks SelfEvictHook) error

RegisterSelfEvictHook registers systems that want to hook into the eviction sequence of the swim protocol.

func (*Node) SelfEvict

func (n *Node) SelfEvict() error

SelfEvict initiates the self eviction sequence of ringpop, it will mark the node as faulty and calls systems that want to hook in to the sequence at the corresponding times.

func (*Node) SetIdentity

func (n *Node) SetIdentity(identity string) error

SetIdentity changes the identity of the local node. This will change the state of the local node and will be gossiped around in the network.

func (*Node) Start

func (n *Node) Start()

Start starts the SWIM protocol and all sub-protocols.

func (*Node) Stop

func (n *Node) Stop()

Stop stops the SWIM protocol and all sub-protocols.

func (*Node) Stopped

func (n *Node) Stopped() bool

Stopped returns whether or not the SWIM protocol is currently running.

func (*Node) Uptime

func (n *Node) Uptime() time.Duration

Uptime returns the amount of time the node has been running for

type NodeInterface

type NodeInterface interface {
	Bootstrap(opts *BootstrapOptions) ([]string, error)
	CountReachableMembers(predicates ...MemberPredicate) int
	Destroy()
	GetChecksum() uint32
	GetReachableMembers(predicates ...MemberPredicate) []Member
	Labels() *NodeLabels
	MemberStats() MemberStats
	ProtocolStats() ProtocolStats
	Ready() bool

	AddListener(events.EventListener) bool
	RemoveListener(events.EventListener) bool

	// swim.SelfEvict
	// mockery has troubles generating a working mock when the interface is
	// embedded therefore the definitions are copied here.
	RegisterSelfEvictHook(hooks SelfEvictHook) error
	SelfEvict() error

	// SetIdentity changes the identity of the local node to a different
	// identity
	SetIdentity(identity string) error
}

NodeInterface specifies the public-facing methods that a SWIM Node implements.

type NodeLabels

type NodeLabels struct {
	// contains filtered or unexported fields
}

NodeLabels implements the ringpop.Labels interface and proxies the calls to the swim.Node backing the membership protocol.

func (*NodeLabels) AsMap

func (n *NodeLabels) AsMap() map[string]string

AsMap gets a readonly copy of all the labels assigned to Node. Changes to the map will not be refelected in the node.

func (*NodeLabels) Get

func (n *NodeLabels) Get(key string) (value string, has bool)

Get the value of a label for this node

func (*NodeLabels) Remove

func (n *NodeLabels) Remove(key string) (removed bool, err error)

Remove a key from the labels

func (*NodeLabels) Set

func (n *NodeLabels) Set(key, value string) error

Set the key to a specific value. Returning an error when it failed eg. when the storage capacity for labels has exceed the maximum ammount. (Currently the storage limit is not implemented)

type OmitReverseFullSyncEvent

type OmitReverseFullSyncEvent struct {
	Target string `json:"target"`
}

OmitReverseFullSyncEvent is sent when a node omits the reverse full sync prodedure because there are already the max number of reverse full sync processes running.

type Options

type Options struct {
	StateTimeouts     StateTimeouts
	MinProtocolPeriod time.Duration

	JoinTimeout, PingTimeout, PingRequestTimeout time.Duration

	PingRequestSize int

	RollupFlushInterval time.Duration
	RollupMaxUpdates    int

	MaxReverseFullSyncJobs int

	// When started, the partition healing algorithm attempts a partition heal
	// every PartitionHealPeriod with a probability of:
	// PartitionHealBaseProbabillity / # Nodes in discoverProvider.
	//
	// When in a 100 node cluster BaseProbabillity = 3 and Period = 30s,
	// every 30 seconds a node will have a probability of 3/100 to start the
	// partition healing procedure. This means that for the entire cluster
	// the discover provider receives 6 calls per minute on average.
	PartitionHealPeriod           time.Duration
	PartitionHealBaseProbabillity float64

	LabelLimits   LabelOptions
	InitialLabels LabelMap

	Clock clock.Clock

	SelfEvict SelfEvictOptions

	// If set to true, ping requests without app name return error
	RequiresAppInPing bool
}

Options is a configuration struct passed the NewNode constructor.

type PingReceiveEvent

type PingReceiveEvent struct {
	Local   string   `json:"local"`
	Source  string   `json:"source"`
	Changes []Change `json:"changes"`
}

A PingReceiveEvent is sent when the node receives a ping from a remote node

type PingRequestPingEvent

type PingRequestPingEvent struct {
	Local    string        `json:"local"`
	Source   string        `json:"source"`
	Target   string        `json:"target"`
	Duration time.Duration `json:"duration"`
}

A PingRequestPingEvent is sent when the node sends a ping to the target node at the behest of the source node and receives a response

type PingRequestReceiveEvent

type PingRequestReceiveEvent struct {
	Local   string   `json:"local"`
	Source  string   `json:"source"`
	Target  string   `json:"target"`
	Changes []Change `json:"changes"`
}

A PingRequestReceiveEvent is sent when the node receives a pign request from a remote node

type PingRequestSendErrorEvent

type PingRequestSendErrorEvent struct {
	Local  string   `json:"local"`
	Target string   `json:"target"`
	Peers  []string `json:"peers"`
	Peer   string   `json:"peer"`
}

A PingRequestSendErrorEvent is sent when the node can't get a response sending ping requests to remote nodes

type PingRequestsSendCompleteEvent

type PingRequestsSendCompleteEvent struct {
	Local    string        `json:"local"`
	Target   string        `json:"target"`
	Peers    []string      `json:"peers"`
	Peer     string        `json:"peer"`
	Duration time.Duration `json:"duration"`
}

A PingRequestsSendCompleteEvent is sent when the node finished sending ping requests to remote nodes

type PingRequestsSendEvent

type PingRequestsSendEvent struct {
	Local  string   `json:"local"`
	Target string   `json:"target"`
	Peers  []string `json:"peers"`
}

A PingRequestsSendEvent is sent when the node sends ping requests to remote nodes

type PingSendCompleteEvent

type PingSendCompleteEvent struct {
	Local    string        `json:"local"`
	Remote   string        `json:"remote"`
	Changes  []Change      `json:"changes"`
	Duration time.Duration `json:"duration"`
}

A PingSendCompleteEvent is sent when the node finished sending a ping to a remote node

type PingSendEvent

type PingSendEvent struct {
	Local   string   `json:"local"`
	Remote  string   `json:"remote"`
	Changes []Change `json:"changes"`
}

A PingSendEvent is sent when the node sends a ping to a remote node

type ProtocolDelayComputeEvent

type ProtocolDelayComputeEvent struct {
	Duration time.Duration `json:"duration"`
}

A ProtocolDelayComputeEvent is sent when protocol delay is computed during a gossip run

type ProtocolFrequencyEvent

type ProtocolFrequencyEvent struct {
	Duration time.Duration `json:"duration"`
}

A ProtocolFrequencyEvent is sent when a gossip run is finished

type ProtocolStats

type ProtocolStats struct {
	Timing     Timing        `json:"timing"`
	Rate       time.Duration `json:"protocolRate"`
	ClientRate float64       `json:"clientRate"`
	ServerRate float64       `json:"serverRate"`
	TotalRate  float64       `json:"totalRate"`
}

ProtocolStats contains stats about the SWIM Protocol for the node

type RedundantReverseFullSyncEvent

type RedundantReverseFullSyncEvent struct {
	Target string `json:"target"`
}

RedundantReverseFullSyncEvent is sent when no new changes were added due to the reverse full sync.

type RefuteUpdateEvent

type RefuteUpdateEvent struct{}

A RefuteUpdateEvent is sent when a node detects gossip about its own state that needs to be corrected

type RequestBeforeReadyEvent

type RequestBeforeReadyEvent struct {
	Endpoint Endpoint
}

A RequestBeforeReadyEvent is sent if a remote request came in for a ringpop endpoint while ringpop was not ready to process requests

type SelfEvict

type SelfEvict interface {
	// RegisterSelfEvictHook is used to register a SelfEvictHook interface to be
	// called during the shutting down of ringpop. Hooks can't be registered
	// after the self eviction has started.
	RegisterSelfEvictHook(hooks SelfEvictHook) error

	// SelfEvict should be called before shutting down the application to notify
	// the members of the membership that this node is going down and should not
	// receive reqeusts anymore.
	SelfEvict() error
}

SelfEvict defines the functions that interact with the self eviction of nodes from the membership prior to shutting down

type SelfEvictHook

type SelfEvictHook interface {
	// PreEvict is the hook that will be called before ringpop evicts itself
	// from the membership
	PreEvict()

	// PostEvict is the hook that will be called after ringpop has evicted
	// itself from them memership
	PostEvict()
}

SelfEvictHook is an interface describing a module that can be registered to the self eviction hooks

type SelfEvictOptions

type SelfEvictOptions struct {
	PingRatio float64
}

SelfEvictOptions configures how self eviction should behave. Applications can configure if ringpop should proactively ping members of the network on self eviction and what percentage/ratio of the memberlist should be pinged at most

type SelfEvictedEvent

type SelfEvictedEvent struct {
	// PhasesCount the total number of phases executed
	PhasesCount int

	// Duration in milliseconds the self-eviction took
	Duration time.Duration
}

SelfEvictedEvent is emitted when self eviction has been completed

type StartReverseFullSyncEvent

type StartReverseFullSyncEvent struct {
	Target string `json:"target"`
}

A StartReverseFullSyncEvent is sent when a node starts the reverse full sync procedure

type StateTimeouts

type StateTimeouts struct {
	// Suspect is the timeout it takes a node in suspect mode to transition to faulty
	Suspect time.Duration

	// Faulty is the timeout it takes a node in faulty mode to transition to tombstone
	Faulty time.Duration

	// Tombstone is the timeout it takes a node in tombstone mode to be evicted
	Tombstone time.Duration
}

StateTimeouts contains the configured timeouts for every state before transitioning to the new state

type Status

type Status struct {
	Status string `json:"status"`
}

Status contains a status string of the response from a handler.

type Timing

type Timing struct {
	Type     string  `json:"type"`
	Min      int64   `json:"min"`
	Max      int64   `json:"max"`
	Sum      int64   `json:"sum"`
	Variance float64 `json:"variance"`
	Mean     float64 `json:"mean"`
	StdDev   float64 `json:"std_dev"`
	Count    int64   `json:"count"`
	Median   float64 `json:"median"`
	P75      float64 `json:"p75"`
	P95      float64 `json:"p95"`
	P99      float64 `json:"p99"`
	P999     float64 `json:"p999"`
}

Timing contains timing information for the SWIM protocol for the node

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL