gossip

package
v1.1.5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 29, 2018 License: Apache-2.0 Imports: 34 Imported by: 0

Documentation

Overview

Package gossip implements a protocol for sharing information between Cockroach nodes using an ad-hoc, peer-to-peer network. The self-assembled network aims to minimize time for new information to reach each node, and minimize network traffic required.

Gossiped information is identified by key. Gossip information is captured by info objects.

Single-valued info values can have any type.

A map of info objects is kept by a Gossip instance. Single-valued info objects can be added via Gossip.AddInfo(). Info can be queried for single-valued keys via Gossip.GetInfo. 

Package gossip is a generated protocol buffer package.

It is generated from these files:
	cockroach/pkg/gossip/gossip.proto

It has these top-level messages:
	BootstrapInfo
	Request
	Response
	InfoStatus
	Info

Index

Constants

View Source 
const (
	// KeyClusterID is the unique UUID for this Cockroach cluster.
	// The value is a string UUID for the cluster.  The cluster ID is
	// gossiped by all nodes that contain a replica of the first range,
	// and it serves as a check for basic gossip connectivity. The
	// Gossip.Connected channel is closed when we see this key.
	KeyClusterID = "cluster-id"

	// KeyStorePrefix is the key prefix for gossiping stores in the network.
	// The suffix is a store ID and the value is roachpb.StoreDescriptor.
	KeyStorePrefix = "store"

	// KeyDeadReplicasPrefix is the key prefix for gossiping dead replicas in the
	// network. The suffix is a store ID and the value is
	// roachpb.StoreDeadReplicas.
	KeyDeadReplicasPrefix = "replica-dead"

	// KeyNodeIDPrefix is the key prefix for gossiping node id
	// addresses. The actual key is suffixed with the decimal
	// representation of the node id and the value is the host:port
	// string address of the node. E.g. node:1 => 127.0.0.1:24001
	KeyNodeIDPrefix = "node"

	// KeyNodeLivenessPrefix is the key prefix for gossiping node liveness info.
	KeyNodeLivenessPrefix = "liveness"

	// KeySentinel is a key for gossip which must not expire or
	// else the node considers itself partitioned and will retry with
	// bootstrap hosts.  The sentinel is gossiped by the node that holds
	// the range lease for the first range.
	KeySentinel = "sentinel"

	// KeyFirstRangeDescriptor is the descriptor for the "first"
	// range. The "first" range contains the meta1 key range, the first
	// level of the bi-level key addressing scheme. The value is a slice
	// of storage.Replica structs.
	KeyFirstRangeDescriptor = "first-range"

	// KeySystemConfig is the gossip key for the system DB span.
	// The value if a config.SystemConfig which holds all key/value
	// pairs in the system DB span.
	KeySystemConfig = "system-db"

	// KeyDistSQLNodeVersionKeyPrefix is key prefix for each node's DistSQL
	// version.
	KeyDistSQLNodeVersionKeyPrefix = "distsql-version"
)

Constants for gossip keys.

Variables

View Source 
var (
	MetaConnectionsIncomingGauge = metric.Metadata{
		Name: "gossip.connections.incoming",
		Help: "Number of active incoming gossip connections"}
	MetaConnectionsOutgoingGauge = metric.Metadata{
		Name: "gossip.connections.outgoing",
		Help: "Number of active outgoing gossip connections"}
	MetaConnectionsRefused = metric.Metadata{
		Name: "gossip.connections.refused",
		Help: "Number of refused incoming gossip connections"}
	MetaInfosSent = metric.Metadata{
		Name: "gossip.infos.sent",
		Help: "Number of sent gossip Info objects"}
	MetaInfosReceived = metric.Metadata{
		Name: "gossip.infos.received",
		Help: "Number of received gossip Info objects"}
	MetaBytesSent = metric.Metadata{
		Name: "gossip.bytes.sent",
		Help: "Number of sent gossip bytes"}
	MetaBytesReceived = metric.Metadata{
		Name: "gossip.bytes.received",
		Help: "Number of received gossip bytes"}
)

Gossip metrics counter names.

View Source 
var (
	ErrInvalidLengthGossip = fmt.Errorf("proto: negative length found during unmarshaling")
	ErrIntOverflowGossip   = fmt.Errorf("proto: integer overflow")
)
View Source 
var (
	// GossipStoresInterval is the interval for gossipping storage-related info.
	GossipStoresInterval = envutil.EnvOrDefaultDuration("COCKROACH_GOSSIP_STORES_INTERVAL",
		defaultGossipStoresInterval)
)

Functions

func IsNodeIDKey 

func IsNodeIDKey(key string) bool

IsNodeIDKey returns true iff the provided key is a valid node ID key.

func MakeDeadReplicasKey 

func MakeDeadReplicasKey(storeID roachpb.StoreID) string

MakeDeadReplicasKey returns the dead replicas gossip key for the given store.

func MakeDistSQLNodeVersionKey added in v1.1.0 

func MakeDistSQLNodeVersionKey(nodeID roachpb.NodeID) string

MakeDistSQLNodeVersionKey returns the gossip key for the given store.

func MakeKey 

func MakeKey(components ...string) string

MakeKey creates a canonical key under which to gossip a piece of information. The first argument will typically be one of the key constants defined in this package.

func MakeNodeIDKey 

func MakeNodeIDKey(nodeID roachpb.NodeID) string

MakeNodeIDKey returns the gossip key for node ID info.

func MakeNodeLivenessKey 

func MakeNodeLivenessKey(nodeID roachpb.NodeID) string

MakeNodeLivenessKey returns the gossip key for node liveness info.

func MakePrefixPattern 

func MakePrefixPattern(prefix string) string

MakePrefixPattern returns a regular expression pattern that matches precisely the Gossip keys created by invocations of MakeKey with multiple arguments for which the first argument is equal to the given prefix.

func MakeStoreKey 

func MakeStoreKey(storeID roachpb.StoreID) string

MakeStoreKey returns the gossip key for the given store.

func NewKeyNotPresentError added in v1.1.0 

func NewKeyNotPresentError(key string) error

NewKeyNotPresentError creates a new KeyNotPresentError.

func NodeIDFromKey 

func NodeIDFromKey(key string) (roachpb.NodeID, error)

NodeIDFromKey attempts to extract a NodeID from the provided key. The key should have been constructed by MakeNodeIDKey. Returns an error if the key is not of the correct type or is not parsable.

func RegisterGossipServer 

func RegisterGossipServer(s *grpc.Server, srv GossipServer)

Types

type BootstrapInfo 

type BootstrapInfo struct {
	// Addresses of other nodes in the cluster.
	Addresses []cockroach_util.UnresolvedAddr `protobuf:"bytes,1,rep,name=addresses" json:"addresses"`
	// Timestamp at which the bootstrap info was written.
	Timestamp cockroach_util_hlc.Timestamp `protobuf:"bytes,2,opt,name=timestamp" json:"timestamp"`
}

BootstrapInfo contains information necessary to bootstrap the gossip network from a cold start.

func (*BootstrapInfo) Descriptor 

func (*BootstrapInfo) Descriptor() ([]byte, []int)

func (*BootstrapInfo) Marshal 

func (m *BootstrapInfo) Marshal() (dAtA []byte, err error)

func (*BootstrapInfo) MarshalTo 

func (m *BootstrapInfo) MarshalTo(dAtA []byte) (int, error)

func (*BootstrapInfo) ProtoMessage 

func (*BootstrapInfo) ProtoMessage()

func (*BootstrapInfo) Reset 

func (m *BootstrapInfo) Reset()

func (*BootstrapInfo) Size 

func (m *BootstrapInfo) Size() (n int)

func (*BootstrapInfo) String 

func (m *BootstrapInfo) String() string

func (*BootstrapInfo) Unmarshal 

func (m *BootstrapInfo) Unmarshal(dAtA []byte) error

type Callback 

type Callback func(string, roachpb.Value)

Callback is a callback method to be invoked on gossip update of info denoted by key.

type Gossip 

type Gossip struct {
	Connected chan struct{} // Closed upon initial connection
	// contains filtered or unexported fields
}

Gossip is an instance of a gossip node. It embeds a gossip server. During bootstrapping, the bootstrap list contains candidates for entry to the gossip network.

func New 

func New(
	ambient log.AmbientContext,
	nodeID *base.NodeIDContainer,
	rpcContext *rpc.Context,
	grpcServer *grpc.Server,
	stopper *stop.Stopper,
	registry *metric.Registry,
) *Gossip

New creates an instance of a gossip node. The higher level manages the NodeIDContainer instance (which can be shared by various server components). The ambient context is expected to already contain the node ID.

grpcServer: The server on which the new Gossip instance will register its RPC 

service. Can be nil, in which case the Gossip will not register the
service.

rpcContext: The context used to connect to other nodes. Can be nil for tests 

that also specify a nil grpcServer and that plan on using the Gossip in a
restricted way by populating it with data manually.

func NewTest 

func NewTest(
	nodeID roachpb.NodeID,
	rpcContext *rpc.Context,
	grpcServer *grpc.Server,
	stopper *stop.Stopper,
	registry *metric.Registry,
) *Gossip

NewTest is a simplified wrapper around New that creates the NodeIDContainer internally. Used for testing.

grpcServer: The server on which the new Gossip instance will register its RPC 

service. Can be nil, in which case the Gossip will not register the
service.

rpcContext: The context used to connect to other nodes. Can be nil for tests 

that also specify a nil grpcServer and that plan on using the Gossip in a
restricted way by populating it with data manually.

func (*Gossip) AddInfo 

func (g *Gossip) AddInfo(key string, val []byte, ttl time.Duration) error

AddInfo adds or updates an info object. Returns an error if info couldn't be added.

func (*Gossip) AddInfoProto 

func (g *Gossip) AddInfoProto(key string, msg proto.Message, ttl time.Duration) error

AddInfoProto adds or updates an info object. Returns an error if info couldn't be added.

func (*Gossip) EnableSimulationCycler 

func (g *Gossip) EnableSimulationCycler(enable bool)

EnableSimulationCycler is for TESTING PURPOSES ONLY. It sets a condition variable which is signaled at each cycle of the simulation via SimulationCycle(). The gossip server makes each connecting client wait for the cycler to signal before responding.

func (Gossip) GetClusterID 

func (s Gossip) GetClusterID() uuid.UUID

func (*Gossip) GetInfo 

func (g *Gossip) GetInfo(key string) ([]byte, error)

GetInfo returns an info value by key or an KeyNotPresentError if specified key does not exist or has expired.

func (*Gossip) GetInfoProto 

func (g *Gossip) GetInfoProto(key string, msg proto.Message) error

GetInfoProto returns an info value by key or KeyNotPresentError if specified key does not exist or has expired.

func (*Gossip) GetInfoStatus 

func (g *Gossip) GetInfoStatus() InfoStatus

GetInfoStatus returns the a copy of the contents of the infostore.

func (Gossip) GetNodeAddr 

func (s Gossip) GetNodeAddr() *util.UnresolvedAddr

GetNodeAddr returns the node's address stored in the Infostore.

func (*Gossip) GetNodeDescriptor 

func (g *Gossip) GetNodeDescriptor(nodeID roachpb.NodeID) (*roachpb.NodeDescriptor, error)

GetNodeDescriptor looks up the descriptor of the node by ID.

func (*Gossip) GetNodeIDAddress 

func (g *Gossip) GetNodeIDAddress(nodeID roachpb.NodeID) (*util.UnresolvedAddr, error)

GetNodeIDAddress looks up the address of the node by ID.

func (*Gossip) GetNodeMetrics 

func (g *Gossip) GetNodeMetrics() *Metrics

GetNodeMetrics returns the gossip node metrics.

func (*Gossip) GetResolvers 

func (g *Gossip) GetResolvers() []resolver.Resolver

GetResolvers returns a copy of the resolvers slice.

func (*Gossip) GetSystemConfig 

func (g *Gossip) GetSystemConfig() (config.SystemConfig, bool)

GetSystemConfig returns the local unmarshalled version of the system config. The second return value indicates whether the system config has been set yet.

func (Gossip) Gossip 

func (s Gossip) Gossip(stream Gossip_GossipServer) error

Gossip receives gossiped information from a peer node. The received delta is combined with the infostore, and this node's own gossip is returned to requesting client.

func (*Gossip) Incoming 

func (g *Gossip) Incoming() []roachpb.NodeID

Incoming returns a slice of incoming gossip client connection node IDs.

func (*Gossip) InfoOriginatedHere added in v1.1.0 

func (g *Gossip) InfoOriginatedHere(key string) bool

InfoOriginatedHere returns true iff the latest info for the provided key originated on this node. This is useful for ensuring that the system config is regossiped as soon as possible when its lease changes hands.

func (*Gossip) LogStatus 

func (g *Gossip) LogStatus()

LogStatus logs the current status of gossip such as the incoming and outgoing connections.

func (*Gossip) MaxHops 

func (g *Gossip) MaxHops() uint32

MaxHops returns the maximum number of hops to reach any other node in the system, according to the infos which have reached this node via gossip network.

func (*Gossip) Outgoing 

func (g *Gossip) Outgoing() []roachpb.NodeID

Outgoing returns a slice of outgoing gossip client connection node IDs. Note that these outgoing client connections may not actually be legitimately connected. They may be in the process of trying, or may already have failed, but haven't yet been processed by the gossip instance.

func (*Gossip) RegisterCallback 

func (g *Gossip) RegisterCallback(pattern string, method Callback) func()

RegisterCallback registers a callback for a key pattern to be invoked whenever new info for a gossip key matching pattern is received. The callback method is invoked with the info key which matched pattern. Returns a function to unregister the callback.

func (*Gossip) RegisterSystemConfigChannel 

func (g *Gossip) RegisterSystemConfigChannel() <-chan struct{}

RegisterSystemConfigChannel registers a channel to signify updates for the system config. It is notified after registration, and whenever a new system config is successfully unmarshalled.

func (*Gossip) SetBootstrapInterval 

func (g *Gossip) SetBootstrapInterval(interval time.Duration)

SetBootstrapInterval sets a minimum interval between successive attempts to connect to new hosts in order to join the gossip network.

func (Gossip) SetClusterID 

func (s Gossip) SetClusterID(clusterID uuid.UUID)

SetClusterID sets the cluster ID to prevent nodes from illegally connecting to incorrect clusters, or from allowing nodes from other clusters to incorrectly connect to this one.

func (*Gossip) SetCullInterval 

func (g *Gossip) SetCullInterval(interval time.Duration)

SetCullInterval sets the interval between periodic shutdown of outgoing gossip client connections in an effort to improve the fitness of the network.

func (*Gossip) SetNodeDescriptor 

func (g *Gossip) SetNodeDescriptor(desc *roachpb.NodeDescriptor) error

SetNodeDescriptor adds the node descriptor to the gossip network.

func (*Gossip) SetStallInterval 

func (g *Gossip) SetStallInterval(interval time.Duration)

SetStallInterval sets the interval between successive checks to determine whether this host is not connected to the gossip network, or else is connected to a partition which doesn't include the host which gossips the sentinel info.

func (*Gossip) SetStorage 

func (g *Gossip) SetStorage(storage Storage) error

SetStorage provides an instance of the Storage interface for reading and writing gossip bootstrap data from persistent storage. This should be invoked as early in the lifecycle of a gossip instance as possible, but can be called at any time.

func (*Gossip) SimulationCycle 

func (g *Gossip) SimulationCycle()

SimulationCycle cycles this gossip node's server by allowing all connected clients to proceed one step.

func (*Gossip) Start 

func (g *Gossip) Start(advertAddr net.Addr, resolvers []resolver.Resolver)

Start launches the gossip instance, which commences joining the gossip network using the supplied rpc server and previously known peer addresses in addition to any bootstrap addresses specified via --join and passed to this method via the resolvers parameter.

The supplied advertised address is used to identify the gossip instance in the gossip network; it will be used by other instances to connect to this instance.

This method starts bootstrap loop, gossip server, and client management in separate goroutines and returns.

type GossipClient 

type GossipClient interface {
	Gossip(ctx context.Context, opts ...grpc.CallOption) (Gossip_GossipClient, error)
}

func NewGossipClient 

func NewGossipClient(cc *grpc.ClientConn) GossipClient

type GossipServer 

type GossipServer interface {
	Gossip(Gossip_GossipServer) error
}

type Gossip_GossipClient 

type Gossip_GossipClient interface {
	Send(*Request) error
	Recv() (*Response, error)
	grpc.ClientStream
}

type Gossip_GossipServer 

type Gossip_GossipServer interface {
	Send(*Response) error
	Recv() (*Request, error)
	grpc.ServerStream
}

type Info 

type Info struct {
	Value cockroach_roachpb1.Value `protobuf:"bytes,1,opt,name=value" json:"value"`
	// Wall time of info when generated by originating node (Unix-nanos).
	OrigStamp int64 `protobuf:"varint,2,opt,name=orig_stamp,json=origStamp,proto3" json:"orig_stamp,omitempty"`
	// Wall time when info is to be discarded (Unix-nanos).
	TTLStamp int64 `protobuf:"varint,3,opt,name=ttl_stamp,json=ttlStamp,proto3" json:"ttl_stamp,omitempty"`
	// Number of hops from originator.
	Hops uint32 `protobuf:"varint,4,opt,name=hops,proto3" json:"hops,omitempty"`
	// Originating node's ID.
	NodeID github_com_cockroachdb_cockroach_pkg_roachpb.NodeID `` /* 141-byte string literal not displayed */
	// Peer node ID which passed this info.
	PeerID github_com_cockroachdb_cockroach_pkg_roachpb.NodeID `` /* 141-byte string literal not displayed */
}

Info is the basic unit of information traded over the gossip network.

func (*Info) Descriptor 

func (*Info) Descriptor() ([]byte, []int)

func (*Info) Marshal 

func (m *Info) Marshal() (dAtA []byte, err error)

func (*Info) MarshalTo 

func (m *Info) MarshalTo(dAtA []byte) (int, error)

func (*Info) ProtoMessage 

func (*Info) ProtoMessage()

func (*Info) Reset 

func (m *Info) Reset()

func (*Info) Size 

func (m *Info) Size() (n int)

func (*Info) String 

func (m *Info) String() string

func (*Info) Unmarshal 

func (m *Info) Unmarshal(dAtA []byte) error

type InfoStatus 

type InfoStatus struct {
	Infos map[string]Info `` /* 127-byte string literal not displayed */
}

InfoStatus contains information about the current status of the infoStore.

func (*InfoStatus) Descriptor 

func (*InfoStatus) Descriptor() ([]byte, []int)

func (*InfoStatus) Marshal 

func (m *InfoStatus) Marshal() (dAtA []byte, err error)

func (*InfoStatus) MarshalTo 

func (m *InfoStatus) MarshalTo(dAtA []byte) (int, error)

func (*InfoStatus) ProtoMessage 

func (*InfoStatus) ProtoMessage()

func (*InfoStatus) Reset 

func (m *InfoStatus) Reset()

func (*InfoStatus) Size 

func (m *InfoStatus) Size() (n int)

func (*InfoStatus) String 

func (m *InfoStatus) String() string

func (*InfoStatus) Unmarshal 

func (m *InfoStatus) Unmarshal(dAtA []byte) error

type KeyNotPresentError added in v1.1.0 

type KeyNotPresentError struct {
	// contains filtered or unexported fields
}

KeyNotPresentError is returned by gossip when queried for a key that doesn't exist of has expired.

func (KeyNotPresentError) Error added in v1.1.0 

func (err KeyNotPresentError) Error() string

Error implements the error interface.

type Metrics 

type Metrics struct {
	ConnectionsRefused *metric.Counter
	BytesReceived      *metric.Counter
	BytesSent          *metric.Counter
	InfosReceived      *metric.Counter
	InfosSent          *metric.Counter
}

Metrics contains gossip metrics used per node and server.

func (Metrics) String 

func (m Metrics) String() string

type Request 

type Request struct {
	// Requesting node's ID.
	NodeID github_com_cockroachdb_cockroach_pkg_roachpb.NodeID `` /* 141-byte string literal not displayed */
	// Address of the requesting client.
	Addr cockroach_util.UnresolvedAddr `protobuf:"bytes,2,opt,name=addr" json:"addr"`
	// Map of high water timestamps from infos originating at other
	// nodes, as seen by the requester.
	HighWaterStamps map[github_com_cockroachdb_cockroach_pkg_roachpb.NodeID]int64 `` /* 241-byte string literal not displayed */
	// Delta of Infos originating at sender.
	Delta map[string]*Info `` /* 137-byte string literal not displayed */
	// Cluster ID to prevent illegal connections.
	ClusterID github_com_cockroachdb_cockroach_pkg_util_uuid.UUID `` /* 141-byte string literal not displayed */
}

Request is the request struct passed with the Gossip RPC.

func (*Request) Descriptor 

func (*Request) Descriptor() ([]byte, []int)

func (*Request) GetUser 

func (*Request) GetUser() string

GetUser implements security.RequestWithUser. Gossip messages are always sent by the node user.

func (*Request) Marshal 

func (m *Request) Marshal() (dAtA []byte, err error)

func (*Request) MarshalTo 

func (m *Request) MarshalTo(dAtA []byte) (int, error)

func (*Request) ProtoMessage 

func (*Request) ProtoMessage()

func (*Request) Reset 

func (m *Request) Reset()

func (*Request) Size 

func (m *Request) Size() (n int)

func (*Request) String 

func (m *Request) String() string

func (*Request) Unmarshal 

func (m *Request) Unmarshal(dAtA []byte) error

type Response 

type Response struct {
	// Responding Node's ID.
	NodeID github_com_cockroachdb_cockroach_pkg_roachpb.NodeID `` /* 141-byte string literal not displayed */
	// Address of the responding client.
	Addr cockroach_util.UnresolvedAddr `protobuf:"bytes,2,opt,name=addr" json:"addr"`
	// Non-nil means client should retry with this address.
	AlternateAddr *cockroach_util.UnresolvedAddr `protobuf:"bytes,3,opt,name=alternate_addr,json=alternateAddr" json:"alternate_addr,omitempty"`
	// Node ID of the alternate address, if alternate_addr is not nil.
	AlternateNodeID github_com_cockroachdb_cockroach_pkg_roachpb.NodeID `` /* 170-byte string literal not displayed */
	// Delta of Infos which are fresh according to the map of Node info messages
	// passed with the request.
	Delta map[string]*Info `` /* 137-byte string literal not displayed */
	// Map of high water timestamps from infos originating at other
	// nodes, as seen by the responder.
	HighWaterStamps map[github_com_cockroachdb_cockroach_pkg_roachpb.NodeID]int64 `` /* 241-byte string literal not displayed */
}

Response is returned from the Gossip.Gossip RPC. Delta will be nil in the event that Alternate is set.

func (*Response) Descriptor 

func (*Response) Descriptor() ([]byte, []int)

func (*Response) Marshal 

func (m *Response) Marshal() (dAtA []byte, err error)

func (*Response) MarshalTo 

func (m *Response) MarshalTo(dAtA []byte) (int, error)

func (*Response) ProtoMessage 

func (*Response) ProtoMessage()

func (*Response) Reset 

func (m *Response) Reset()

func (*Response) Size 

func (m *Response) Size() (n int)

func (*Response) String 

func (m *Response) String() string

func (*Response) Unmarshal 

func (m *Response) Unmarshal(dAtA []byte) error

type Storage 

type Storage interface {
	// ReadBootstrapInfo fetches the bootstrap data from the persistent
	// store into the provided bootstrap protobuf. Returns nil or an
	// error on failure.
	ReadBootstrapInfo(*BootstrapInfo) error
	// WriteBootstrapInfo stores the provided bootstrap data to the
	// persistent store. Returns nil or an error on failure.
	WriteBootstrapInfo(*BootstrapInfo) error
}

Storage is an interface which allows the gossip instance to read and write bootstrapping data to persistent storage between instantiations.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.