cluster

package
v0.0.0-...-b84de06 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 9, 2020 License: Apache-2.0 Imports: 52 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var ConfigSchema = config.Schema{
	"backups.compression_algorithm":  {Default: "gzip", Validator: validateCompression},
	"cluster.offline_threshold":      {Type: config.Int64, Default: offlineThresholdDefault(), Validator: offlineThresholdValidator},
	"cluster.images_minimal_replica": {Type: config.Int64, Default: "3", Validator: imageMinimalReplicaValidator},
	"cluster.max_voters":             {Type: config.Int64, Default: "3", Validator: maxVotersValidator},
	"cluster.max_standby":            {Type: config.Int64, Default: "2", Validator: maxStandByValidator},
	"core.https_allowed_headers":     {},
	"core.https_allowed_methods":     {},
	"core.https_allowed_origin":      {},
	"core.https_allowed_credentials": {Type: config.Bool},
	"core.proxy_http":                {},
	"core.proxy_https":               {},
	"core.proxy_ignore_hosts":        {},
	"core.trust_password":            {Hidden: true, Setter: passwordSetter},
	"core.trust_ca_certificates":     {Type: config.Bool},
	"candid.api.key":                 {},
	"candid.api.url":                 {},
	"candid.domains":                 {},
	"candid.expiry":                  {Type: config.Int64, Default: "3600"},
	"images.auto_update_cached":      {Type: config.Bool, Default: "true"},
	"images.auto_update_interval":    {Type: config.Int64, Default: "6"},
	"images.compression_algorithm":   {Default: "gzip", Validator: validateCompression},
	"images.remote_cache_expiry":     {Type: config.Int64, Default: "10"},
	"maas.api.key":                   {},
	"maas.api.url":                   {},
	"rbac.agent.url":                 {},
	"rbac.agent.username":            {},
	"rbac.agent.private_key":         {},
	"rbac.agent.public_key":          {},
	"rbac.api.expiry":                {Type: config.Int64, Default: "3600"},
	"rbac.api.key":                   {},
	"rbac.api.url":                   {},
	"rbac.expiry":                    {Type: config.Int64, Default: "3600"},

	"storage.lvm_fstype":           {Setter: deprecatedStorage, Default: "ext4"},
	"storage.lvm_mount_options":    {Setter: deprecatedStorage, Default: "discard"},
	"storage.lvm_thinpool_name":    {Setter: deprecatedStorage, Default: "LXDThinPool"},
	"storage.lvm_vg_name":          {Setter: deprecatedStorage},
	"storage.lvm_volume_size":      {Setter: deprecatedStorage, Default: "10GiB"},
	"storage.zfs_pool_name":        {Setter: deprecatedStorage},
	"storage.zfs_remove_snapshots": {Setter: deprecatedStorage, Type: config.Bool},
	"storage.zfs_use_refquota":     {Setter: deprecatedStorage, Type: config.Bool},
}

ConfigSchema defines available server configuration keys.

View Source 
var ErrNotLeader = fmt.Errorf("Not leader")

ErrNotLeader signals that a node not the leader.

View Source 
var SchemaVersion = cluster.SchemaVersion

SchemaVersion holds the version of the cluster database schema.

Functions

func Accept 

func Accept(state *state.State, gateway *Gateway, name, address string, schema, api, arch int) ([]db.RaftNode, error)

Accept a new node and add it to the cluster.

This instance must already be clustered.

Return an updated list raft database nodes (possibly including the newly accepted node).

func Assign 

func Assign(state *state.State, gateway *Gateway, nodes []db.RaftNode) error

Assign a new role to the local dqlite node.

func Bootstrap 

func Bootstrap(state *state.State, gateway *Gateway, name string) error

Bootstrap turns a non-clustered LXD instance into the first (and leader) node of a new LXD cluster.

This instance must already have its cluster.https_address set and be listening on the associated network address.

func ConfigGetBool 

func ConfigGetBool(cluster *db.Cluster, key string) (bool, error)

ConfigGetBool is a convenience for loading the cluster configuration and returning the value of a particular boolean key.

It's a deprecated API meant to be used by call sites that are not interacting with the database in a transactional way.

func ConfigGetInt64 

func ConfigGetInt64(cluster *db.Cluster, key string) (int64, error)

ConfigGetInt64 is a convenience for loading the cluster configuration and returning the value of a particular key.

It's a deprecated API meant to be used by call sites that are not interacting with the database in a transactional way.

func ConfigGetString 

func ConfigGetString(cluster *db.Cluster, key string) (string, error)

ConfigGetString is a convenience for loading the cluster configuration and returning the value of a particular key.

It's a deprecated API meant to be used by call sites that are not interacting with the database in a transactional way.

func Connect 

func Connect(address string, cert *shared.CertInfo, notify bool) (lxd.InstanceServer, error)

Connect is a convenience around lxd.ConnectLXD that configures the client with the correct parameters for node-to-node communication.

If 'notify' switch is true, then the user agent will be set to the special value 'lxd-cluster-notifier', which can be used in some cases to distinguish between a regular client request and an internal cluster request.

func ConnectIfInstanceIsRemote 

func ConnectIfInstanceIsRemote(cluster *db.Cluster, project, name string, cert *shared.CertInfo, instanceType instancetype.Type) (lxd.InstanceServer, error)

ConnectIfInstanceIsRemote figures out the address of the node which is running the container with the given name. If it's not the local node will connect to it and return the connected client, otherwise it will just return nil.

func ConnectIfVolumeIsRemote 

func ConnectIfVolumeIsRemote(cluster *db.Cluster, poolID int64, volumeName string, volumeType int, cert *shared.CertInfo) (lxd.InstanceServer, error)

ConnectIfVolumeIsRemote figures out the address of the node on which the volume with the given name is defined. If it's not the local node will connect to it and return the connected client, otherwise it will just return nil.

If there is more than one node with a matching volume name, an error is returned.

func Count 

func Count(state *state.State) (int, error)

Count is a convenience for checking the current number of nodes in the cluster.

func DqliteLog 

func DqliteLog(l client.LogLevel, format string, a ...interface{})

DqliteLog redirects dqlite's logs to our own logger

func Enabled 

func Enabled(node *db.Node) (bool, error)

Enabled is a convenience that returns true if clustering is enabled on this node.

func Events 

func Events(endpoints *endpoints.Endpoints, cluster *db.Cluster, f func(int64, api.Event)) (task.Func, task.Schedule)

Events starts a task that continuously monitors the list of cluster nodes and maintains a pool of websocket connections against all of them, in order to get notified about events.

Whenever an event is received the given callback is invoked.

func Handover 

func Handover(state *state.State, gateway *Gateway, address string) (string, []db.RaftNode, error)

Handover looks for a non-voter member that can be promoted to replace a the member with the given address, which is shutting down. It returns the address of such member along with an updated list of nodes, with the ne role set.

It should be called only by the current leader.

func HasConnectivity 

func HasConnectivity(cert *shared.CertInfo, address string) bool

HasConnectivity probes the member with the given address for connectivity.

func HeartbeatNode 

func HeartbeatNode(taskCtx context.Context, address string, cert *shared.CertInfo, heartbeatData *APIHeartbeat) error

HeartbeatNode performs a single heartbeat request against the node with the given address.

func HeartbeatTask 

func HeartbeatTask(gateway *Gateway) (task.Func, task.Schedule)

HeartbeatTask returns a task function that performs leader-initiated heartbeat checks against all LXD nodes in the cluster.

It will update the heartbeat timestamp column of the nodes table accordingly, and also notify them of the current list of database nodes.

func Join 

func Join(state *state.State, gateway *Gateway, cert *shared.CertInfo, name string, raftNodes []db.RaftNode) error

Join makes a non-clustered LXD node join an existing cluster.

It's assumed that Accept() was previously called against the leader node, which handed the raft server ID.

The cert parameter must contain the keypair/CA material of the cluster being joined.

func Leave 

func Leave(state *state.State, gateway *Gateway, name string, force bool) (string, error)

Leave a cluster.

If the force flag is true, the node will leave even if it still has containers and images.

The node will only leave the raft cluster, and won't be removed from the database. That's done by Purge().

Upon success, return the address of the leaving node.

This function must be called by the cluster leader.

func List 

func List(state *state.State, gateway *Gateway) ([]api.ClusterMember, error)

List the nodes of the cluster.

func ListDatabaseNodes 

func ListDatabaseNodes(database *db.Node) ([]string, error)

ListDatabaseNodes returns a list of database node names.

func MaybeUpdate 

func MaybeUpdate(state *state.State) error

MaybeUpdate Check this node's version and possibly run LXD_CLUSTER_UPDATE.

func MigrateToDqlite10 

func MigrateToDqlite10(dir string) error

MigrateToDqlite10 migrates the on-disk global database format to dqlite 1.0.

func NotifyUpgradeCompleted 

func NotifyUpgradeCompleted(state *state.State, cert *shared.CertInfo) error

NotifyUpgradeCompleted sends a notification to all other nodes in the cluster that any possible pending database update has been applied, and any nodes which was waiting for this node to be upgraded should re-check if it's okay to move forward.

func Purge 

func Purge(cluster *db.Cluster, name string) error

Purge removes a node entirely from the cluster database.

func Rebalance 

func Rebalance(state *state.State, gateway *Gateway) (string, []db.RaftNode, error)

Rebalance the raft cluster, trying to see if we have a spare online node that we can promote to voter node if we are below membershipMaxRaftVoters, or to standby if we are below membershipMaxStandBys.

If there's such spare node, return its address as well as the new list of raft nodes.

func Recover 

func Recover(database *db.Node) error

Recover attempts data recovery on the cluster database.

func RemoveRaftNode 

func RemoveRaftNode(gateway *Gateway, address string) error

RemoveRaftNode removes a raft node from the raft configuration.

func ResolveTarget 

func ResolveTarget(cluster *db.Cluster, target string) (string, error)

ResolveTarget is a convenience for handling the value ?targetNode query parameter. It returns the address of the given node, or the empty string if the given node is the local one.

func SetupTrust 

func SetupTrust(cert, targetAddress, targetCert, targetPassword string) error

SetupTrust is a convenience around InstanceServer.CreateCertificate that adds the given client certificate to the trusted pool of the cluster at the given address, using the given password.

Types

type APIHeartbeat 

type APIHeartbeat struct {
	sync.Mutex // Used to control access to Members maps.
	Members    map[int64]APIHeartbeatMember
	Version    APIHeartbeatVersion
	Time       time.Time

	// Indicates if heartbeat contains a fresh set of node states.
	// This can be used to indicate to the receiving node that the state is fresh enough to
	// trigger node refresh activies (such as forkdns).
	FullStateList bool
}

APIHeartbeat contains data sent to nodes in heartbeat.

func (*APIHeartbeat) Send 

func (hbState *APIHeartbeat) Send(ctx context.Context, cert *shared.CertInfo, localAddress string, nodes []db.NodeInfo, delay bool)

Send sends heartbeat requests to the nodes supplied and updates heartbeat state.

func (*APIHeartbeat) Update 

func (hbState *APIHeartbeat) Update(fullStateList bool, raftNodes []db.RaftNode, allNodes []db.NodeInfo, offlineThreshold time.Duration)

Update updates an existing APIHeartbeat struct with the raft and all node states supplied. If allNodes provided is an empty set then this is considered a non-full state list.

type APIHeartbeatMember 

type APIHeartbeatMember struct {
	ID            int64     // ID field value in nodes table.
	Address       string    // Host and Port of node.
	RaftID        uint64    // ID field value in raft_nodes table, zero if non-raft node.
	RaftRole      int       // Node role in the raft cluster, from the raft_nodes table
	Raft          bool      // Deprecated, use non-zero RaftID instead to indicate raft node.
	LastHeartbeat time.Time // Last time we received a successful response from node.
	Online        bool      // Calculated from offline threshold and LastHeatbeat time.
	// contains filtered or unexported fields
}

APIHeartbeatMember contains specific cluster node info.

type APIHeartbeatVersion 

type APIHeartbeatVersion struct {
	Schema        int
	APIExtensions int
}

APIHeartbeatVersion contains max versions for all nodes in cluster.

type Config 

type Config struct {
	// contains filtered or unexported fields
}

Config holds cluster-wide configuration values.

func ConfigLoad 

func ConfigLoad(tx *db.ClusterTx) (*Config, error)

ConfigLoad loads a new Config object with the current cluster configuration values fetched from the database.

func (*Config) AutoUpdateInterval 

func (c *Config) AutoUpdateInterval() time.Duration

AutoUpdateInterval returns the configured images auto update interval.

func (*Config) CandidServer 

func (c *Config) CandidServer() (string, string, int64, string)

CandidServer returns all the Candid settings needed to connect to a server.

func (*Config) Dump 

func (c *Config) Dump() map[string]interface{}

Dump current configuration keys and their values. Keys with values matching their defaults are omitted.

func (*Config) HTTPSAllowedCredentials 

func (c *Config) HTTPSAllowedCredentials() bool

HTTPSAllowedCredentials returns the relevant CORS setting.

func (*Config) HTTPSAllowedHeaders 

func (c *Config) HTTPSAllowedHeaders() string

HTTPSAllowedHeaders returns the relevant CORS setting.

func (*Config) HTTPSAllowedMethods 

func (c *Config) HTTPSAllowedMethods() string

HTTPSAllowedMethods returns the relevant CORS setting.

func (*Config) HTTPSAllowedOrigin 

func (c *Config) HTTPSAllowedOrigin() string

HTTPSAllowedOrigin returns the relevant CORS setting.

func (*Config) ImagesMinimalReplica 

func (c *Config) ImagesMinimalReplica() int64

ImagesMinimalReplica returns the numbers of nodes for cluster images replication

func (*Config) MAASController 

func (c *Config) MAASController() (string, string)

MAASController the configured MAAS url and key, if any.

func (*Config) MaxStandBy 

func (c *Config) MaxStandBy() int64

MaxStandBy returns the maximum number of standby members in a cluster that will be assigned the stand-by role.

func (*Config) MaxVoters 

func (c *Config) MaxVoters() int64

MaxVoters returns the maximum number of members in a cluster that will be assigned the voter role.

func (*Config) OfflineThreshold 

func (c *Config) OfflineThreshold() time.Duration

OfflineThreshold returns the configured heartbeat threshold, i.e. the number of seconds before after which an unresponsive node is considered offline..

func (*Config) Patch 

func (c *Config) Patch(patch map[string]interface{}) (map[string]string, error)

Patch changes only the configuration keys in the given map.

Return what has actually changed.

func (*Config) ProxyHTTP 

func (c *Config) ProxyHTTP() string

ProxyHTTP returns the configured HTTP proxy, if any.

func (*Config) ProxyHTTPS 

func (c *Config) ProxyHTTPS() string

ProxyHTTPS returns the configured HTTPS proxy, if any.

func (*Config) ProxyIgnoreHosts 

func (c *Config) ProxyIgnoreHosts() string

ProxyIgnoreHosts returns the configured ignore-hosts proxy setting, if any.

func (*Config) RBACServer 

func (c *Config) RBACServer() (string, string, int64, string, string, string, string)

RBACServer returns all the Candid settings needed to connect to a server.

func (*Config) RemoteCacheExpiry 

func (c *Config) RemoteCacheExpiry() int64

RemoteCacheExpiry returns the configured expiration value for remote images expiration.

func (*Config) Replace 

func (c *Config) Replace(values map[string]interface{}) (map[string]string, error)

Replace the current configuration with the given values.

Return what has actually changed.

func (*Config) TrustCACertificates 

func (c *Config) TrustCACertificates() bool

TrustCACertificates returns whether client certificates are checked against a CA.

func (*Config) TrustPassword 

func (c *Config) TrustPassword() string

TrustPassword returns the LXD trust password for authenticating clients.

type Gateway 

type Gateway struct {

	// Used for the heartbeat handler
	Cluster           *db.Cluster
	HeartbeatNodeHook func(*APIHeartbeat)
	// contains filtered or unexported fields
}

Gateway mediates access to the dqlite cluster using a gRPC SQL client, and possibly runs a dqlite replica on this LXD node (if we're configured to do so).

func NewGateway 

func NewGateway(db *db.Node, cert *shared.CertInfo, options ...Option) (*Gateway, error)

NewGateway creates a new Gateway for managing access to the dqlite cluster.

When a new gateway is created, the node-level database is queried to check what kind of role this node plays and if it's exposed over the network. It will initialize internal data structures accordingly, for example starting a local dqlite server if this node is a database node.

After creation, the Daemon is expected to expose whatever http handlers the HandlerFuncs method returns and to access the dqlite cluster using the dialer returned by the DialFunc method.

func (*Gateway) Context 

func (g *Gateway) Context() context.Context

Context returns a cancellation context to pass to dqlite.NewDriver as option.

This context gets cancelled by Gateway.Kill() and at that point any connection failure won't be retried.

func (*Gateway) DemoteOfflineNode 

func (g *Gateway) DemoteOfflineNode(raftID uint64) error

DemoteOfflineNode force demoting an offline node.

func (*Gateway) DialFunc 

func (g *Gateway) DialFunc() client.DialFunc

DialFunc returns a dial function that can be used to connect to one of the dqlite nodes.

func (*Gateway) HandlerFuncs 

func (g *Gateway) HandlerFuncs(nodeRefreshTask func(*APIHeartbeat)) map[string]http.HandlerFunc

HandlerFuncs returns the HTTP handlers that should be added to the REST API endpoint in order to handle database-related requests.

There are two handlers, one for the /internal/raft endpoint and the other for /internal/db, which handle respectively raft and gRPC-SQL requests.

These handlers might return 404, either because this LXD node is a non-clustered node not available over the network or because it is not a database node part of the dqlite cluster.

func (*Gateway) IsDqliteNode 

func (g *Gateway) IsDqliteNode() bool

IsDqliteNode returns true if this gateway is running a dqlite node.

func (*Gateway) Kill 

func (g *Gateway) Kill()

Kill is an API that the daemon calls before it actually shuts down and calls Shutdown(). It will abort any ongoing or new attempt to establish a SQL gRPC connection with the dialer (typically for running some pre-shutdown queries).

func (*Gateway) LeaderAddress 

func (g *Gateway) LeaderAddress() (string, error)

LeaderAddress returns the address of the current raft leader.

func (*Gateway) NodeStore 

func (g *Gateway) NodeStore() client.NodeStore

NodeStore returns a dqlite server store that can be used to lookup the addresses of known database nodes.

func (*Gateway) Reset 

func (g *Gateway) Reset(cert *shared.CertInfo) error

Reset the gateway, shutting it down and starting against from scratch using the given certificate.

This is used when disabling clustering on a node.

func (*Gateway) Shutdown 

func (g *Gateway) Shutdown() error

Shutdown this gateway, stopping the gRPC server and possibly the raft factory.

func (*Gateway) Snapshot 

func (g *Gateway) Snapshot() error

Snapshot can be used to manually trigger a RAFT snapshot

func (*Gateway) Sync 

func (g *Gateway) Sync()

Sync dumps the content of the database to disk. This is useful for inspection purposes, and it's also needed by the activateifneeded command so it can inspect the database in order to decide whether to activate the daemon or not.

func (*Gateway) TransferLeadership 

func (g *Gateway) TransferLeadership() error

TransferLeadership attempts to transfer leadership to another node.

func (*Gateway) WaitUpgradeNotification 

func (g *Gateway) WaitUpgradeNotification()

WaitUpgradeNotification waits for a notification from another node that all nodes in the cluster should now have been upgraded and have matching schema and API versions.

type Notifier 

type Notifier func(hook func(lxd.InstanceServer) error) error

Notifier is a function that invokes the given function against each node in the cluster excluding the invoking one.

func NewNotifier 

func NewNotifier(state *state.State, cert *shared.CertInfo, policy NotifierPolicy) (Notifier, error)

NewNotifier builds a Notifier that can be used to notify other peers using the given policy.

type NotifierPolicy 

type NotifierPolicy int

NotifierPolicy can be used to tweak the behavior of NewNotifier in case of some nodes are down. 

const (
	NotifyAll    NotifierPolicy = iota // Requires that all nodes are up.
	NotifyAlive                        // Only notifies nodes that are alive
	NotifyTryAll                       // Attempt to notify all nodes regardless of state.
)

Possible notifcation policies.

type Option 

type Option func(*options)

Option to be passed to NewGateway to customize the resulting instance.

func Latency 

func Latency(latency float64) Option

Latency is a coarse grain measure of how fast/reliable network links are. This is used to tweak the various timeouts parameters of the raft algorithm. See the raft.Config structure for more details. A value of 1.0 means use the default values from hashicorp's raft package. Values closer to 0 reduce the values of the various timeouts (useful when running unit tests in-memory).

func LogLevel 

func LogLevel(level string) Option

LogLevel sets the logging level for messages emitted by dqlite and raft.

Source Files

View all Source files

Directories

Path Synopsis
Package raft contains the subset of hashicorp/raft needed to perform data migrations to dqlite 1.0.
 Package raft contains the subset of hashicorp/raft needed to perform data migrations to dqlite 1.0.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.