Documentation ¶
Overview ¶
Package reign implements the Erlang-like clustering support.
If you are coming in directly via the godoc, please see the README.md for this package for more information about what this is. The godoc is being used here only for API-type documentation.
Index ¶
- Variables
- func CreateFromReader(r io.Reader, thisNode NodeID, log ClusterLogger) (ConnectionService, Names, error)
- func CreateFromSpec(spec *ClusterSpec, thisNode NodeID, log ClusterLogger) (ConnectionService, Names, error)
- func CreateFromSpecFile(clusterSpecLocation string, thisNode NodeID, log ClusterLogger) (ConnectionService, Names, error)
- func NoClustering(log ClusterLogger) (ConnectionService, Names)
- func RegisterType(value interface{})
- type Address
- func (a *Address) GetID() MailboxID
- func (a *Address) MarshalBinary() ([]byte, error)
- func (a *Address) MarshalJSON() ([]byte, error)
- func (a *Address) MarshalText() ([]byte, error)
- func (a *Address) OnCloseNotify(addr *Address)
- func (a *Address) RemoveNotify(addr *Address)
- func (a *Address) Send(m interface{}) error
- func (a *Address) String() string
- func (a *Address) UnmarshalBinary(b []byte) error
- func (a *Address) UnmarshalFromID(mID MailboxID)
- func (a *Address) UnmarshalJSON(b []byte) error
- func (a *Address) UnmarshalText(b []byte) error
- type Cluster
- type ClusterLogger
- type ClusterSpec
- type ConnectionService
- type Mailbox
- func (m *Mailbox) Close()
- func (m *Mailbox) MessageCount() int
- func (m *Mailbox) Receive() interface{}
- func (m *Mailbox) ReceiveAsync() (interface{}, bool)
- func (m *Mailbox) ReceiveMatch(matcher func(interface{}) bool) interface{}
- func (m *Mailbox) ReceiveTimeout(timeout time.Duration) (interface{}, bool)
- type MailboxClosed
- type MailboxID
- type MultipleClaim
- type Names
- type NamesDebugger
- type NodeDefinition
- type NodeID
Constants ¶
This section is empty.
Variables ¶
var ( // DefaultPingInterval determines the minimum interval between PING messages. DefaultPingInterval = time.Second * 30 // MaxSequentialPingFailures is the maximum number of sequential ping failures // tolerable before the pinger panics, triggering a service restart if running // under suture. MaxSequentialPingFailures uint8 = 5 )
var DeadlineInterval = time.Minute * 5
DeadlineInterval determine how long to keep the network connection open after a successful read. The net.Conn deadline value will be reset to time.Now().Add(DeadlineInterval) upon each successful message read over the network. Defaults to 5 minutes.
var ErrCantGloballyRegister = errors.New("can't globally register this address")
ErrCantGloballyRegister is returned when you are trying to register an address with the registry that can not be so registered. Only local mailboxes created with New() can be registered with the registry.
var ErrFailOnClusterHandshake = errors.New("Failing on ssl handshake, as instructed")
ErrFailOnClusterHandshake is the error returned when a node connector's failOnClusterHandshake property is true.
var ErrIllegalAddressFormat = errors.New("illegally-formatted address")
ErrIllegalAddressFormat is returned when something attempts to unmarshal an illegal text or binary string into an Address.
var ErrMailboxClosed = errors.New("mailbox has been closed")
ErrMailboxClosed is returned when the target mailbox has (already) been closed.
var ErrNoAddressRegistered = errors.New("no address is registered with that name")
ErrNoAddressRegistered is returned when there are no addresses at the given name.
var ErrNotLocalMailbox = errors.New("function required a local mailbox ID but this is a remote MailboxID")
ErrNotLocalMailbox is returned when a remote mailbox's MailboxID is passed into a function that only works on local mailboxes.
var NullLogger = nullLogger{}
NullLogger implements ClusterLogger, and throws all logging messages away.
var StdLogger = stdLogger{}
StdLogger is a ClusterLogger that will use the log.Output function from the standard logging package.
Functions ¶
func CreateFromReader ¶
func CreateFromReader(r io.Reader, thisNode NodeID, log ClusterLogger) (ConnectionService, Names, error)
CreateFromReader creates a cluster based on the io.Reader of your choice.
func CreateFromSpec ¶
func CreateFromSpec(spec *ClusterSpec, thisNode NodeID, log ClusterLogger) (ConnectionService, Names, error)
CreateFromSpec creates a cluster directly from a *ClusterSpec, the ultimate in control.
func CreateFromSpecFile ¶
func CreateFromSpecFile(clusterSpecLocation string, thisNode NodeID, log ClusterLogger) (ConnectionService, Names, error)
CreateFromSpecFile is the most automated way of creating a cluster, using the command-line parameter "clusterspec" to specify the location of the cluster specification .json file, and creating a cluster from there.
Once created, you still need to call .Serve()
Note *Cluster conforms to the suture.Service interface.
nil may be passed as the ClusterLogger, in which case the standard log.Printf will be used.
func NoClustering ¶
func NoClustering(log ClusterLogger) (ConnectionService, Names)
NoClustering is called to say you have no interest in clustering.
This configures reign to work in a no-clustering state. You can use all mailbox functionality, and there will be no network activity or configuration required.
func RegisterType ¶
func RegisterType(value interface{})
RegisterType registers a type to be sent across the cluster.
This wraps gob.Register, in case we ever change the encoding method.
Types ¶
type Address ¶
type Address struct {
// contains filtered or unexported fields
}
An Address is the public face of the Mailbox. It is fine to pass this by value.
WARNING: It is not safe to use either Address or *Address for equality testing or as a key in maps! Use .GetID() to obtain a MailboxID, which is. (Both Address and *Address are fine to store as values.)
func (*Address) MarshalBinary ¶
MarshalBinary implements binary marshalling for Addresses.
A marshalled Address only carries its identifier. When unmarshalled on the same node, the unmarshalled address will be reconnected to the original Mailbox. If unmarshalled on a different node, a reference to the remote mailbox will be unmarshaled.
func (*Address) MarshalJSON ¶ added in v0.9.1
MarshalJSON implements JSON marshalling for Addresses.
func (*Address) MarshalText ¶
MarshalText implements text marshalling for Addresses.
See MarshalBinary.
func (*Address) OnCloseNotify ¶ added in v0.9.1
OnCloseNotify requests that the target address receive a close notice when the target address is closed.
This is like linking in Erlang, and is intended to provide the same guarantees.
While addresses and goroutines are not technically bound together, it is convenient to think of an address "belonging" to a goroutine. From that point of view, note the caller is the *argument*, not the object. Calls look like:
otherAddress.OnCloseNotify(myAddress)
Read that as something like, "You over there, upon your closing notify (me)."
Calling this more than once with the same address may or may not cause multiple notifications to occur.
func (*Address) RemoveNotify ¶ added in v0.9.1
RemoveNotify will remove the notification request from the Address you call this on.
This does not guarantee that you will not receive a closed notification from the Address, due to (inherent) race conditions.
func (*Address) Send ¶
Send something to the mailbox corresponding to this address.
All concrete types that you wish to send across the cluster must have .Register called on them. See the documentation on gob.Register for the reason why. (The local .RegisterType abstracts our dependency on gob. If you don't register through reign's .RegisterType, future versions of this package may require you to fix that.)
The error is primarily for internal purposes. If the mailbox is local, and has been closed, ErrMailboxClosed will be returned.
An error guarantees failure, but lack of error does not guarantee success! Arguably, "ErrMailboxClosed" should be seen as a purely internal detail, and just like in Erlang, if you want a guarantee you must implement an acknowledgement. However, just like in Erlang, we leak this internal detail a bit. I don't know if that's a good idea; use with caution. (See: erlang:is_process_alive, which similarly leaks out whether the process is local or not.)
func (*Address) UnmarshalBinary ¶
UnmarshalBinary implements binary unmarshalling for Addresses.
func (*Address) UnmarshalFromID ¶
UnmarshalFromID allows you to obtain a legal address from an AddressID. Use as:
var addr reign.Address err := addr.UnmarshalFromID(addressID) if err == nil { addr.Send(...) }
func (*Address) UnmarshalJSON ¶ added in v0.9.1
UnmarshalJSON implements JSON unmarshalling for Addresses.
func (*Address) UnmarshalText ¶
UnmarshalText implements text unmarshalling for Addresses.
type Cluster ¶
type Cluster struct { Nodes map[NodeID]*NodeDefinition ThisNode *NodeDefinition // Populate this with the desired protocols you may want from // the crypto/tls constants list. By default, this library uses // TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 if you don't specify. PermittedProtocols []uint16 // The root signing certificate used by the entire cluster. ClusterCertificate *x509.Certificate // The CertPool containing that certificate RootCAs *x509.CertPool // This node's certificate Certificate tls.Certificate ClusterLogger // contains filtered or unexported fields }
A Cluster describes a cluster.
func (*Cluster) AddConnectionStatusCallback ¶
AddConnectionStatusCallback allows you to register a callback to be called when a node becomes connected or disconnected from this node. Note that while the connection callback is reliable, the "disconnected" callback is not, simply due to the nature of networks. The boolean parameter is true if this is a connection event, and false if this is a disconnection event.
This function should not block, or should be guaranteed to run very quickly, as you'll be blocking the reconnection attempt itself if the function is slow. This function is also guaranteed to be called within the same process for a given node, thus, it is safe on a per-node basis to send a message via a mailbox; they're guaranteed to be in order on a per-node basis.
While there is certain advanced functionality that can only be implemented via this functionality, bear in mind that the clustering system is already able to tell you when you become disconnected from a remote Address, via the general NotifyAddressOnTerminate. In general, if you can get away with using NotifyAddressOnTerminate on an address, rather than this, you'll probably be happier.
For example, it is not worthwhile to try to maintain a local dictionary of "connectedness" to Nodes with this callback, then try to check if a Node is connected before doing something. This is a classic error. Better to just do it and try to pick up the pieces.
(This is, internally, the mechanism used to implement that functionality, and it seems reasonable to expose this.)
There is no way to remove a callback once added.
Future note: The true semantic meaning of this callback is "we have given up on this node's connection for now and are dropping messages on the floor". At the moment, this fires the instant the TCP connection drops. Should reign ever change that behavior, this callback will only be fired when we finally give up. "Giving up" will be defined as ceasing to buffer messages to be sent, and dropping all future messages on the floor.
type ClusterLogger ¶
type ClusterLogger interface { Error(...interface{}) Errorf(format string, args ...interface{}) Warn(...interface{}) Warnf(format string, args ...interface{}) Info(...interface{}) Infof(format string, args ...interface{}) Trace(...interface{}) Tracef(format string, args ...interface{}) }
A ClusterLogger is the logging interface used by the Cluster system.
The clustering system uses Info for situations that are not problems. This includes:
- Address resolution progress of remote cluster nodes. (Common DNS problems or misconfigurations can cause excessive times for resolution. This should give enough visibility into the resolution process to rapidly identify the problem.)
The clustering system uses Warn for situations that are problematic and you need to know about them, but are generally "expected" and may resolve themselves without any direction action. (That is, in general, losing network connections is "bad", but also perfectly normal and expected.) The clustering system uses Warn for:
- Connections established and lost to the other nodes
- Attempts to update the cluster configuration that fail due to invalid configuration
The clustering system uses Error for situations that prevent connection to some target node, and will most likely not resolve themselves without active human intervention. The clustering system will user Error for:
- Handshake with foreign node failed due to:
- Remote said they had a different NodeID than I expected.
- Incompatible clustering version.
- Failed SSL handshake.
The goal is that all Errors are things that should fire alarming systems, and all things that should fire alarming systems are Errors.
You can wrap a standard *log.Logger with the provided WrapLogger.
func WrapLogger ¶
func WrapLogger(l *log.Logger) ClusterLogger
WrapLogger takes as standard *log.Logger and returns a ClusterLogger that uses that logger.
type ClusterSpec ¶
type ClusterSpec struct { Nodes []*NodeDefinition `json:"nodes"` PermittedProtocols []string `json:"permitted_protocols,omit_empty"` // To specify the path for the node's cert, set either both of // NodeKeyPath and NodeCertPath to load from disk, or // NodeKeyPEM and NodeCertPEM to load the certs from some other source. // // The paths may use %d as a placeholder, to fill in the node ID. NodeKeyPath string `json:"node_key_path,omitempty"` NodeCertPath string `json:"node_cert_path,omitempty"` NodeKeyPEM string `json:"node_key_pem,omitempty"` NodeCertPEM string `json:"node_cert_pem,omitempty"` // And to specify the path for the cluster's cert, set either // ClusterCertPath to load it from disk, or ClusterKeyPEM to load // it from source. // // Note you SHOULD NOT distribute the cluster's private keys to all // the nodes. ClusterCertPath string `json:"cluster_cert_path,omitempty"` ClusterCertPEM string `json:"cluster_cert_pem,omitempty"` }
ClusterSpec defines how to create a cluster. The primary purpose of this data type is to define the JSON serialization via the standard Go encoding/json serialization.
Note that Nodes should use string representations of the numbers 0-255 to specify the NodeID as the key to "nodes". (encoding/json does not permit anything except strings as keys for the map.)
type ConnectionService ¶
type ConnectionService interface { NewMailbox() (*Address, *Mailbox) Terminate() // Inherited from suture.Service Serve() Stop() // Inherited from reign.Cluster AddConnectionStatusCallback(f func(NodeID, bool)) }
ConnectionService provides an interface to the reign connectionServer and registry objects. It inherits from suture.Service and reign.Cluster.
type Mailbox ¶
type Mailbox struct {
// contains filtered or unexported fields
}
A Mailbox is what you receive messages from via Receive or ReceiveNext.
func (*Mailbox) Close ¶ added in v0.9.1
func (m *Mailbox) Close()
Close shuts down a given mailbox. Once closed, a mailbox will reject messages without even looking at them, and can no longer have any Receive used on them.
Further, it will notify any registered Addresses that it has been closed.
This facility is used analogously to Erlang's "link" functionality. Of course in Go you can't be notified when a goroutine terminates, but if you defer mailbox.Close() in the proper place for your mailbox user, you can get most of the way there.
It is not an error to Close an already-Closed mailbox.
func (*Mailbox) MessageCount ¶ added in v0.9.1
MessageCount returns the number of messages in the mailbox.
0 is always returned if the mailbox is closed.
func (*Mailbox) Receive ¶
func (m *Mailbox) Receive() interface{}
Receive will receive the next message sent to this mailbox. It blocks until the next message comes in, which may be forever. If the mailbox is closed, it will receive a MailboxClosed reply.
If you've got multiple receivers on a single mailbox, be sure to check for MailboxClosed.
func (*Mailbox) ReceiveAsync ¶ added in v0.9.1
ReceiveAsync will return immediately with (obj, true) if, and only if, there was a message in the inbox, or else (nil, false). Works the same way as ReceiveNext, otherwise.
func (*Mailbox) ReceiveMatch ¶ added in v0.9.1
ReceiveMatch will receive the next message sent to this mailbox that matches according to the passed-in function.
ReceiveMatch assumes that it is the only function running against the Mailbox. If you ReceiveMatch from multiple goroutines, or ReceiveMatch in one and ReceiveNext in another, you *will* miss messages in the routine calling ReceiveMatch.
I recommend that your matcher function be:
func (i) bool { _, ok = i.(SomeType) return ok }
If the mailbox gets closed, this will return a MailboxClosed, regardless of the behavior of the matcher.
type MailboxClosed ¶ added in v0.9.1
type MailboxClosed MailboxID
MailboxClosed is sent to Addresses that request notification of when a Mailbox is being closed, with OnCloseNotify. If you request close notification of multiple mailboxes, this can be converted to an MailboxID which can be used to distinguish them.
type MultipleClaim ¶
type MultipleClaim struct {
Name string
}
MultipleClaim messages are send to name claimers in the event that there are multiple name claims to a given name. Note that the claimants array is a static snapshot of the claimants at the time of conflict, and that the "current" situation (to the extent that is definable) may change at any time.
type Names ¶
type Names interface { GetDebugger() NamesDebugger Lookup(string) *Address LookupAll(string) []*Address MessageCount() int MultipleClaimCount() int32 Register(string, *Address) error SeenNames(...string) []bool Sync() Unregister(string, *Address) }
Names exposes some functionality of registry
Lookup looks up a given name and returns a mailbox that can be used to send messages and request termination notifications.
Be sure to consult the documentation about the Registry in the documentation section above; use of this address could in some circumstances result in the message being delivered to more than one Mailbox.
In particular, this function does no checking as to whether the address exists, as that information is intrinsically racy anyhow. If you want to take extra care about it, use NotifyAddressOnTerminate, just as with local addresses.
Register claims the given global name in the registry. It can then be accessed and manipulated via Lookup.
This does not happen synchronously, as there seems to be no reason for the caller to synchronously wait for this.
A registered mailbox should stand ready to receive MultipleClaim messages from the cluster.
The passed-in Address must be something that directly came from a New() call. Addressed obtained from the network or from Lookup itself will return an error instead of registering.
Unregister removes the given claim from a given global name. Unregistration will only occur if the current registrant matches the address passed in. It is not an error for it not to match; the call will simply be ignored.
On a given node, only one Address can have a claim on a name. If you wish to supercede a claim with a new address, you can simply register the new claim, and it will overwrite the previous one.
If the address passed in is not the current registrant, the call is ignored, thus it is safe to call this.
type NamesDebugger ¶
type NamesDebugger interface { AddressCount() uint AllNames() []string DumpClaims() map[string][]MailboxID DumpJSON() string MessageCount() int MultipleClaimCount() int32 SeenNames(...string) []bool }
NamesDebugger is an interface over the registry struct. These functions acquire locks and are not supposed to be called in a production setting.
type NodeDefinition ¶
type NodeDefinition struct { ID NodeID `json:"id"` Address string `json:"address"` ListenAddress string `json:"listen_address,omit_empty"` LocalAddress string `json:"local_address,omit_empty"` // contains filtered or unexported fields }
A NodeDefinition gives information about the node in question. This is primarily used to create a static JSON file that represents the node, using the standard encoding/json to produce this structure.
The Address is the IP address and port (separated by colon) that the other nodes will use to talk to this cluster node. This is the only required field.
The ListenAddress is what the cluster will actually bind to. If this is the same as the address, you may leave it unspecified. This is for cases where due to network routing, load balancers, proxies, etc. the address the rest of the cluster uses to connect is not the same as the internal bind address. In simple cases, leave this blank.
The LocalAddress is the address to use for the outgoing connections to the cluster. If blank, net.DialTCP will be passed nil for the laddr. In simple cases, leave this blank.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
cmd
|
|
reign_init
Executable reign_init can be used to quickly set up a certificate authority and usable certificates for reign.
|
Executable reign_init can be used to quickly set up a certificate authority and usable certificates for reign. |
reign_sample
Executable reign_sample contains a simple demonstration of using the reign library to run a cluster of nodes.
|
Executable reign_sample contains a simple demonstration of using the reign library to run a cluster of nodes. |
Package internal segments off things that must be public for serialization but have no place in the main documentation.
|
Package internal segments off things that must be public for serialization but have no place in the main documentation. |