gossip

Documentation

Index

• type Config
• type GossipEventReceiver
  • func NewGossipEventReceiver(logger pilosa.Logger) *GossipEventReceiver
  • func (g *GossipEventReceiver) NotifyJoin(n *memberlist.Node)
  • func (g *GossipEventReceiver) NotifyLeave(n *memberlist.Node)
  • func (g *GossipEventReceiver) NotifyUpdate(n *memberlist.Node)
  • func (g *GossipEventReceiver) Start(h pilosa.EventHandler) error
• type GossipMemberSet
  • func NewGossipMemberSet(name string, host string, cfg Config, ger *GossipEventReceiver, ...) (*GossipMemberSet, error)
  • func (g *GossipMemberSet) GetBindAddr() string
  • func (g *GossipMemberSet) GetBroadcasts(overhead, limit int) [][]byte
  • func (g *GossipMemberSet) LocalState(join bool) []byte
  • func (g *GossipMemberSet) MergeRemoteState(buf []byte, join bool)
  • func (g *GossipMemberSet) NodeMeta(limit int) []byte
  • func (g *GossipMemberSet) NotifyMsg(b []byte)
  • func (g *GossipMemberSet) Open(n *pilosa.Node) error
  • func (g *GossipMemberSet) SendAsync(pb proto.Message) error
  • func (g *GossipMemberSet) SendSync(pb proto.Message) error
  • func (g *GossipMemberSet) Start(h pilosa.BroadcastHandler) error
• type GossipMemberSetOption
  • func WithLogger(logger *log.Logger) GossipMemberSetOption
  • func WithTransport(transport *Transport) GossipMemberSetOption
• type Transport
  • func NewTransport(host string, port int, logger *log.Logger) (*Transport, error)

Types

type Config

type Config struct {
	// Port indicates the port to which pilosa should bind for internal state sharing.
	Port  string   `toml:"port"`
	Seeds []string `toml:"seeds"`
	Key   string   `toml:"key"`
	// StreamTimeout is the timeout for establishing a stream connection with
	// a remote node for a full state sync, and for stream read and write
	// operations. Maps to memberlist TCPTimeout.
	StreamTimeout toml.Duration `toml:"stream-timeout"`
	// SuspicionMult is the multiplier for determining the time an
	// inaccessible node is considered suspect before declaring it dead.
	// The actual timeout is calculated using the formula:
	//
	//   SuspicionTimeout = SuspicionMult * log(N+1) * ProbeInterval
	//
	// This allows the timeout to scale properly with expected propagation
	// delay with a larger cluster size. The higher the multiplier, the longer
	// an inaccessible node is considered part of the cluster before declaring
	// it dead, giving that suspect node more time to refute if it is indeed
	// still alive.
	SuspicionMult int `toml:"suspicion-mult"`
	// PushPullInterval is the interval between complete state syncs.
	// Complete state syncs are done with a single node over TCP and are
	// quite expensive relative to standard gossiped messages. Setting this
	// to zero will disable state push/pull syncs completely.
	//
	// Setting this interval lower (more frequent) will increase convergence
	// speeds across larger clusters at the expense of increased bandwidth
	// usage.
	PushPullInterval toml.Duration `toml:"push-pull-interval"`
	// ProbeInterval and ProbeTimeout are used to configure probing behavior
	// for memberlist.
	//
	// ProbeInterval is the interval between random node probes. Setting
	// this lower (more frequent) will cause the memberlist cluster to detect
	// failed nodes more quickly at the expense of increased bandwidth usage.
	//
	// ProbeTimeout is the timeout to wait for an ack from a probed node
	// before assuming it is unhealthy. This should be set to 99-percentile
	// of RTT (round-trip time) on your network.
	ProbeInterval toml.Duration `toml:"probe-interval"`
	ProbeTimeout  toml.Duration `toml:"probe-timeout"`

	// Interval and Nodes are used to configure the gossip
	// behavior of memberlist.
	//
	// Interval is the interval between sending messages that need
	// to be gossiped that haven't been able to piggyback on probing messages.
	// If this is set to zero, non-piggyback gossip is disabled. By lowering
	// this value (more frequent) gossip messages are propagated across
	// the cluster more quickly at the expense of increased bandwidth.
	//
	// Nodes is the number of random nodes to send gossip messages to
	// per Interval. Increasing this number causes the gossip messages
	// to propagate across the cluster more quickly at the expense of
	// increased bandwidth.
	//
	// ToTheDeadTime is the interval after which a node has died that
	// we will still try to gossip to it. This gives it a chance to refute.
	Interval      toml.Duration `toml:"interval"`
	Nodes         int           `toml:"nodes"`
	ToTheDeadTime toml.Duration `toml:"to-the-dead-time"`
}

Config holds toml-friendly memberlist configuration.

type GossipEventReceiver

type GossipEventReceiver struct {
	Logger pilosa.Logger
	// contains filtered or unexported fields
}

GossipEventReceiver is used to enable an application to receive events about joins and leaves over a channel.

Care must be taken that events are processed in a timely manner from the channel, since this delegate will block until an event can be sent.

func NewGossipEventReceiver

func NewGossipEventReceiver(logger pilosa.Logger) *GossipEventReceiver

NewGossipEventReceiver returns a new instance of GossipEventReceiver.

func (*GossipEventReceiver) NotifyJoin

func (g *GossipEventReceiver) NotifyJoin(n *memberlist.Node)

func (*GossipEventReceiver) NotifyLeave

func (g *GossipEventReceiver) NotifyLeave(n *memberlist.Node)

func (*GossipEventReceiver) NotifyUpdate

func (g *GossipEventReceiver) NotifyUpdate(n *memberlist.Node)

func (*GossipEventReceiver) Start

func (g *GossipEventReceiver) Start(h pilosa.EventHandler) error

Start implements the pilosa.EventReceiver interface and sets the EventHandler.

type GossipMemberSet

type GossipMemberSet struct {
	Logger pilosa.Logger
	// contains filtered or unexported fields
}

GossipMemberSet represents a gossip implementation of MemberSet using memberlist.

func NewGossipMemberSet

func NewGossipMemberSet(name string, host string, cfg Config, ger *GossipEventReceiver, sh pilosa.StatusHandler, options ...GossipMemberSetOption) (*GossipMemberSet, error)

NewGossipMemberSet returns a new instance of GossipMemberSet based on options.

func (*GossipMemberSet) GetBindAddr

func (g *GossipMemberSet) GetBindAddr() string

GetBindAddr returns the gossip bind address based on config and auto bind port. This method is currently only used in a test scenario where a second node needs the auto-bind address of the first node to use as its gossip seed.

func (*GossipMemberSet) GetBroadcasts

func (g *GossipMemberSet) GetBroadcasts(overhead, limit int) [][]byte

GetBroadcasts implementation of the memberlist.Delegate interface called when user data messages can be broadcast.

func (*GossipMemberSet) LocalState

func (g *GossipMemberSet) LocalState(join bool) []byte

LocalState implementation of the memberlist.Delegate interface sends this Node's state data.

func (*GossipMemberSet) MergeRemoteState

func (g *GossipMemberSet) MergeRemoteState(buf []byte, join bool)

MergeRemoteState implementation of the memberlist.Delegate interface receive and process the remote side's LocalState.

func (*GossipMemberSet) NodeMeta

func (g *GossipMemberSet) NodeMeta(limit int) []byte

NodeMeta implementation of the memberlist.Delegate interface.

func (*GossipMemberSet) NotifyMsg

func (g *GossipMemberSet) NotifyMsg(b []byte)

NotifyMsg implementation of the memberlist.Delegate interface called when a user-data message is received.

func (*GossipMemberSet) Open

func (g *GossipMemberSet) Open(n *pilosa.Node) error

Open implements the MemberSet interface to start network activity.

func (*GossipMemberSet) SendAsync

func (g *GossipMemberSet) SendAsync(pb proto.Message) error

SendAsync implementation of the Gossiper interface.

func (*GossipMemberSet) SendSync

func (g *GossipMemberSet) SendSync(pb proto.Message) error

SendSync implementation of the Broadcaster interface.

func (*GossipMemberSet) Start

func (g *GossipMemberSet) Start(h pilosa.BroadcastHandler) error

Start implements the BroadcastReceiver interface and sets the BroadcastHandler.

type GossipMemberSetOption

type GossipMemberSetOption func(*GossipMemberSet) error

GossipMemberSetOption describes a functional option for GossipMemberSet.

func WithLogger

func WithLogger(logger *log.Logger) GossipMemberSetOption

WithLogger is a functional option for providing a logger to NewGossipMemberSet.

func WithTransport

func WithTransport(transport *Transport) GossipMemberSetOption

WithTransport is a functional option for providing a transport to NewGossipMemberSet.

type Transport

type Transport struct {
	//memberlist.Transport
	Net *memberlist.NetTransport
	URI *pilosa.URI
}

Transport is a gossip transport for binding to a port.

func NewTransport

func NewTransport(host string, port int, logger *log.Logger) (*Transport, error)

NewTransport returns a NetTransport based on the given host and port. It will dynamically bind to a port if port is 0. This is useful for test cases where specifying a port is not reasonable. func NewTransport(host string, port int) (*memberlist.NetTransport, error) {