Documentation
¶
Index ¶
- Constants
- Variables
- func GenAddress() string
- func GetExternalIP() string
- func GetExternalIPAsInt() int
- func IsRoutableIPv4(ip string) bool
- func StripNanomsgAddressPrefix(nanomsgAddr string) (suffix string, err error)
- func UtilCleanupTestUserDatabases()
- func UtilTestSetupThree(callmePreStart func(peer *Peer)) (p0, p1, p2 *Peer, peers *LeadAndFollowList)
- type BoltSaver
- type Inventory
- type LeadAndFollowList
- type Peer
- func (peer *Peer) AmFollow() bool
- func (peer *Peer) BackgroundReceiveBcastSetAndWriteToBolt()
- func (peer *Peer) BcastSet(ki *api.KeyInv) error
- func (peer *Peer) ClientInitiateBcastGet(key []byte, includeValue bool, timeout time.Duration, who string) (kis []*api.KeyInv, err error)
- func (peer *Peer) GetFollowSess() (sessF *swp.Session)
- func (peer *Peer) GetGrpcAddr() string
- func (peer *Peer) GetGrpcPorts() (xport, iport int)
- func (peer *Peer) GetLatest(key []byte, includeValue bool) (ki *api.KeyInv, err error)
- func (peer *Peer) GetPeerList(timeout time.Duration) (*LeadAndFollowList, error)
- func (peer *Peer) LeadTransferCheckpoint(key, chkptData []byte, when time.Time) error
- func (peer *Peer) LocalGet(key []byte, includeValue bool) (ki *api.KeyInv, err error)
- func (peer *Peer) LocalSet(ki *api.KeyInv) error
- func (peer *Peer) ServerHandleBcastGet(msg *nats.Msg) error
- func (peer *Peer) SetFollowSess(sessF *swp.Session)
- func (peer *Peer) SetGrpcPorts(xport, iport int, xhost string)
- func (peer *Peer) Start() error
- func (peer *Peer) StartBackgroundCheckpointRecv(myID, myFollowSubj string)
- func (peer *Peer) StartPeriodicClusterAgentLocQueries()
- func (peer *Peer) Stop()
- func (peer *Peer) WaitForPeerCount(n int, timeout time.Duration) (*LeadAndFollowList, error)
- type Saver
Constants ¶
const RequestChanLen = 8
Variables ¶
var BoltDataBucketName = []byte("data") // bucket
var BoltMetaBucketName = []byte("meta") // bucket
var BoltSizeBucketName = []byte("size") // bucket
var ErrAmFollower = fmt.Errorf("LeadTransferCheckpoint error: I am follower, not transmitting checkpoint")
var ErrAmLead = fmt.Errorf("error: I am lead")
var ErrNoFollowers = fmt.Errorf("error: no followers")
var ErrShutdown = fmt.Errorf("shutting down")
var ErrTimedOut = fmt.Errorf("timed out")
Functions ¶
func GenAddress ¶
func GenAddress() string
GenAddress generates a local address by calling GetAvailPort() and GetExternalIP(), then prefixing them with 'tcp://'.
func GetExternalIP ¶
func GetExternalIP() string
GetExternalIP tries to determine the external IP address used on this host.
func GetExternalIPAsInt ¶
func GetExternalIPAsInt() int
GetExternalIPAsInt calls GetExternalIP() and then converts the resulting IPv4 string into an integer.
func IsRoutableIPv4 ¶
IsRoutableIPv4 returns true if the string in ip represents an IPv4 address that is not private. See http://en.wikipedia.org/wiki/Private_network#Private_IPv4_address_spaces for the numeric ranges that are private. 127.0.0.1, 192.168.0.1, and 172.16.0.1 are examples of non-routables IP addresses.
func StripNanomsgAddressPrefix ¶
StripNanomsgAddressPrefix removes the 'tcp://' prefix from nanomsgAddr.
func UtilCleanupTestUserDatabases ¶
func UtilCleanupTestUserDatabases()
func UtilTestSetupThree ¶
func UtilTestSetupThree(callmePreStart func(peer *Peer)) (p0, p1, p2 *Peer, peers *LeadAndFollowList)
Types ¶
type BoltSaver ¶
type BoltSaver struct {
// contains filtered or unexported fields
}
func NewBoltSaver ¶
func (*BoltSaver) Compact ¶ added in v0.9.9
Compact does these steps to reduce the space fragmentation that happens in a bolt database.
- open a fresh .compressed bolt db file.
- read each object from the bolt db and write it to the fresh -> .compressed bolt db file.
- close the both files.
- rename the .compressed file to be the original db file name. os.Rename is atomic.
- re-open the newly compact-ed db file
The routines below were adapted from the compaction code in https://github.com/boltdb/bolt/blob/master/cmd/bolt/main.go which is used under the following MIT license.
The MIT License (MIT)
Copyright (c) 2013 Ben Johnson ¶
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
INVAR: b.db must be already open.
func (*BoltSaver) InitDbIfNeeded ¶
type Inventory ¶
func NewInventory ¶
func NewInventory() *Inventory
type LeadAndFollowList ¶
type Peer ¶
type Peer struct { Halt *idem.Halter LeadAndFollowBchan *bchan.Bchan MemberGainedBchan *bchan.Bchan MemberLostBchan *bchan.Bchan LeadStatus leadFlag GservCfg *gserv.ServerConfig Whoami string // as a host SshClientLoginUsername string SshClientPrivateKeyPath string SshClientClientKnownHostsPath string SshdReady chan bool SshClientAllowsNewSshdServer bool TestAllowOneshotConnect bool SkipEncryption bool // contains filtered or unexported fields }
Peer serves as a member of a replication cluster. One peer will be elected lead. The others will be followers. All peers will run a background receive session.
func NewPeer ¶
NewPeer should be given the same cmdflags as a hnatsd/gnatsd process.
"-routes=nats://localhost:9229 -cluster=nats://localhost:9230 -p 4223"
We auto-append "-health" if not provided, since that is essential for our peering network.
Each node needs its own -cluster address and -p port, and to form a cluster, the -routes of subsequent nodes need to point at one of the -cluster of an earlier started node.
After NewPeer() and before calling Start(), the Peer.SetGrpcPorts() method should be called to establish which port(s) to listen on.
If we save every 30 seconds, then numSetsBeforeCompact int64 == 1000 means compact the boltdb after every ~8 hours of writes.
func (*Peer) BackgroundReceiveBcastSetAndWriteToBolt ¶
func (peer *Peer) BackgroundReceiveBcastSetAndWriteToBolt()
func (*Peer) ClientInitiateBcastGet ¶
func (*Peer) GetFollowSess ¶
func (*Peer) GetGrpcAddr ¶
func (*Peer) GetGrpcPorts ¶
func (*Peer) GetPeerList ¶
func (peer *Peer) GetPeerList(timeout time.Duration) (*LeadAndFollowList, error)
func (*Peer) LeadTransferCheckpoint ¶
LeadTransferCheckpoint is the raison d'etre of the peer libary. This function is called called when we've just generated a checkpoint and need to propagate it out to our followers. It is safe to be called by followers, but that should be avoided since the background peer will already be checkpointing when it receives data from the lead.
The first time the lead calls this, it should be with chkptData == nil. This will signal to a new lead that it should get the most recent checkpoint from the cluster (including ourselves) so that new leads taking over don't ignore the very recent state from any previous lead.
func (*Peer) ServerHandleBcastGet ¶
pull the local boltdb version of the key and send it back. User nats if only the key's metadata was requested. Use gRPC for sending the large file.
func (*Peer) SetFollowSess ¶
func (*Peer) SetGrpcPorts ¶
func (*Peer) StartBackgroundCheckpointRecv ¶
StartBackroundCheckpointdRecv will keep a peer running in the background and always accepting and writing checkpoints (as long as we are not lead when they are received). Track these by their timestamps, and if we have a new one (recognized by a more recent timestamp), then save it to disk (this dedups if we get multiples of the same).
The host address bound will be taken from peer.serverOpts.Host which corresponds to the --addr in the nats options.
After NewPeer() and before calling this routine, the Peer.SetGrpcPorts() method should be called to establish which port(s) to listen on.
func (*Peer) StartPeriodicClusterAgentLocQueries ¶
func (peer *Peer) StartPeriodicClusterAgentLocQueries()