cluster

package
v1.29.0-rc.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 10, 2025 License: BSD-3-Clause Imports: 36 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Config added in v1.25.2 

type Config struct {
	// WorkDir is the directory RAFT will use to store config & snapshot
	WorkDir string
	// NodeID is this node id
	NodeID string
	// Host is this node host name
	Host string
	// RaftPort is used by internal RAFT communication
	RaftPort int
	// RPCPort is used by weaviate internal gRPC communication
	RPCPort int
	// RaftRPCMessageMaxSize is the maximum message sized allowed on the internal RPC communication
	// TODO: Remove Raft prefix to avoid confusion between RAFT and RPC.
	RaftRPCMessageMaxSize int

	// NodeNameToPortMap maps server names to port numbers
	NodeNameToPortMap map[string]int

	// HeartbeatTimeout specifies the time in follower state without contact
	// from a leader before we attempt an election.
	HeartbeatTimeout time.Duration
	// ElectionTimeout specifies the time in candidate state without contact
	// from a leader before we attempt an election.
	ElectionTimeout time.Duration

	// SnapshotThreshold controls how many outstanding logs there must be before
	// we perform a snapshot. This is to prevent excessive snapshotting by
	// replaying a small set of logs instead. The value passed here is the initial
	// setting used. This can be tuned during operation using ReloadConfig.
	SnapshotThreshold uint64

	// SnapshotInterval controls how often we check if we should perform a
	// snapshot. We randomly stagger between this value and 2x this value to avoid
	// the entire cluster from performing a snapshot at once. The value passed
	// here is the initial setting used. This can be tuned during operation using
	// ReloadConfig.
	SnapshotInterval time.Duration

	// BootstrapTimeout is the time a node will notify other node that it is ready to bootstrap a cluster if it can't
	// find a an existing cluster to join
	BootstrapTimeout time.Duration
	// BootstrapExpect is the number of nodes this cluster expect to receive a notify from to start bootstrapping a
	// cluster
	BootstrapExpect int

	// ConsistencyWaitTimeout is the duration we will wait for a schema version to land on that node
	ConsistencyWaitTimeout time.Duration
	NodeToAddressResolver  resolver.NodeToAddress
	// NodeSelector is the memberlist interface to RAFT
	NodeSelector cluster.NodeSelector
	Logger       *logrus.Logger
	Voter        bool

	// MetadataOnlyVoters configures the voters to store metadata exclusively, without storing any other data
	MetadataOnlyVoters bool

	// DB is the interface to the weaviate database. It is necessary so that schema changes are reflected to the DB
	DB schema.Indexer
	// Parser parses class field after deserialization
	Parser schema.Parser
	// LoadLegacySchema is responsible for loading old schema from boltDB
	LoadLegacySchema schema.LoadLegacySchema
	// SaveLegacySchema is responsible for loading new schema into boltDB
	SaveLegacySchema schema.SaveLegacySchema
	// IsLocalHost only required when running Weaviate from the console in localhost
	IsLocalHost bool

	// SentryEnabled configures the sentry integration to add internal middlewares to rpc client/server to set spans &
	// capture traces
	SentryEnabled bool

	// EnableOneNodeRecovery enables the actually one node recovery logic to avoid it running all the time when
	// unnecessary
	EnableOneNodeRecovery bool
	// ForceOneNodeRecovery will force the single node recovery routine to run. This is useful if the cluster has
	// committed wrong peer configuration entry that makes it unable to obtain a quorum to start.
	// WARNING: This should be run on *actual* one node cluster only.
	ForceOneNodeRecovery bool

	EnableFQDNResolver bool
	FQDNResolverTLD    string

	// 	AuthzController to manage RBAC commands and apply it to casbin
	AuthzController authorization.Controller
}

type Raft added in v1.25.2 

type Raft struct {
	// contains filtered or unexported fields
}

Raft abstracts away the Raft store, providing clients with an interface that encompasses all query & write operations. It ensures that these operations are executed on the current leader, regardless of the specific leader in the cluster. If current node is the leader, then changes will be applied on the local node and bypass any networking requests.

func NewRaft added in v1.25.2 

func NewRaft(selector cluster.NodeSelector, store *Store, client client) *Raft

func (*Raft) AddClass added in v1.25.2 

func (s *Raft) AddClass(ctx context.Context, cls *models.Class, ss *sharding.State) (uint64, error)

func (*Raft) AddProperty added in v1.25.2 

func (s *Raft) AddProperty(ctx context.Context, class string, props ...*models.Property) (uint64, error)

func (*Raft) AddRolesForUser added in v1.28.0 

func (s *Raft) AddRolesForUser(user string, roles []string) error

func (*Raft) AddTenants added in v1.25.2 

func (s *Raft) AddTenants(ctx context.Context, class string, req *cmd.AddTenantsRequest) (uint64, error)

func (*Raft) Close added in v1.25.2 

func (s *Raft) Close(ctx context.Context) (err error)

func (*Raft) DeleteClass added in v1.25.2 

func (s *Raft) DeleteClass(ctx context.Context, name string) (uint64, error)

func (*Raft) DeleteRoles added in v1.28.0 

func (s *Raft) DeleteRoles(names ...string) error

func (*Raft) DeleteTenants added in v1.25.2 

func (s *Raft) DeleteTenants(ctx context.Context, class string, req *cmd.DeleteTenantsRequest) (uint64, error)

func (*Raft) Execute added in v1.25.2 

func (s *Raft) Execute(ctx context.Context, req *cmd.ApplyRequest) (uint64, error)

func (*Raft) GetRoles added in v1.28.0 

func (s *Raft) GetRoles(names ...string) (map[string][]authorization.Policy, error)

func (*Raft) GetRolesForUser added in v1.28.0 

func (s *Raft) GetRolesForUser(user string) (map[string][]authorization.Policy, error)

func (*Raft) GetUsersForRole added in v1.28.0 

func (s *Raft) GetUsersForRole(role string) ([]string, error)

func (*Raft) HasPermission added in v1.28.0 

func (s *Raft) HasPermission(roleName string, permission *authorization.Policy) (bool, error)

HasPermission returns consistent permissions check by asking the leader

func (*Raft) Join added in v1.25.2 

func (s *Raft) Join(ctx context.Context, id, addr string, voter bool) error

func (*Raft) LeaderWithID added in v1.25.2 

func (s *Raft) LeaderWithID() (string, string)

LeaderWithID is used to return the current leader address and ID of the cluster. It may return empty strings if there is no current leader or the leader is unknown.

func (*Raft) Open added in v1.25.2 

func (s *Raft) Open(ctx context.Context, db schema.Indexer) error

Open opens this store service and marked as such. It constructs a new Raft node using the provided configuration. If there is any old state, such as snapshots, logs, peers, etc., all of those will be restored

func (*Raft) Query added in v1.25.2 

func (s *Raft) Query(ctx context.Context, req *cmd.QueryRequest) (*cmd.QueryResponse, error)

Query receives a QueryRequest and ensure it is executed on the leader and returns the related QueryResponse If any error happens it returns it

func (*Raft) QueryClassVersions added in v1.27.11 

func (s *Raft) QueryClassVersions(classes ...string) (map[string]uint64, error)

QueryClassVersions returns the current version of the requested classes.

func (*Raft) QueryReadOnlyClasses added in v1.25.2 

func (s *Raft) QueryReadOnlyClasses(classes ...string) (map[string]versioned.Class, error)

QueryReadOnlyClass will verify that class is non empty and then build a Query that will be directed to the leader to ensure we will read the class with strong consistency

func (*Raft) QuerySchema added in v1.25.2 

func (s *Raft) QuerySchema() (models.Schema, error)

QuerySchema build a Query to read the schema that will be directed to the leader to ensure we will read the class with strong consistency

func (*Raft) QueryShardOwner added in v1.25.2 

func (s *Raft) QueryShardOwner(class, shard string) (string, uint64, error)

QueryShardOwner build a Query to read the tenants of a given class that will be directed to the leader to ensure we will read the tenant with strong consistency and return the shard owner node

func (*Raft) QueryShardingState added in v1.25.2 

func (s *Raft) QueryShardingState(class string) (*sharding.State, uint64, error)

QueryShardingState build a Query to read the sharding state of a given class. The request will be directed to the leader to ensure we will read the shard state with strong consistency and return the state and it's version.

func (*Raft) QueryTenants added in v1.25.2 

func (s *Raft) QueryTenants(class string, tenants []string) ([]*models.TenantResponse, uint64, error)

QueryTenants build a Query to read the tenants of a given class that will be directed to the leader to ensure we will read the class with strong consistency

func (*Raft) QueryTenantsShards added in v1.25.2 

func (s *Raft) QueryTenantsShards(class string, tenants ...string) (map[string]string, uint64, error)

QueryTenantsShards build a Query to read the tenants and their activity status of a given class. The request will be directed to the leader to ensure we will read the tenant with strong consistency and return the shard owner node

func (*Raft) Ready added in v1.25.2 

func (s *Raft) Ready() bool

func (*Raft) Remove added in v1.25.2 

func (s *Raft) Remove(ctx context.Context, id string) error

func (*Raft) RemovePermissions added in v1.28.0 

func (s *Raft) RemovePermissions(role string, permissions []*authorization.Policy) error

func (*Raft) RestoreClass added in v1.25.2 

func (s *Raft) RestoreClass(ctx context.Context, cls *models.Class, ss *sharding.State) (uint64, error)

func (*Raft) RevokeRolesForUser added in v1.28.0 

func (s *Raft) RevokeRolesForUser(user string, roles ...string) error

func (*Raft) SchemaReader added in v1.25.2 

func (s *Raft) SchemaReader() schema.SchemaReader

func (*Raft) Stats added in v1.25.2 

func (s *Raft) Stats() map[string]any

func (*Raft) StorageCandidates added in v1.25.13 

func (s *Raft) StorageCandidates() []string

StorageCandidates return the nodes in the raft configuration or memberlist storage nodes based on the current configuration of the cluster if it does have MetadataVoterOnly nodes.

func (*Raft) StoreSchemaV1 added in v1.25.2 

func (s *Raft) StoreSchemaV1() error

func (*Raft) UpdateClass added in v1.25.2 

func (s *Raft) UpdateClass(ctx context.Context, cls *models.Class, ss *sharding.State) (uint64, error)

func (*Raft) UpdateShardStatus added in v1.25.2 

func (s *Raft) UpdateShardStatus(ctx context.Context, class, shard, status string) (uint64, error)

func (*Raft) UpdateTenants added in v1.25.2 

func (s *Raft) UpdateTenants(ctx context.Context, class string, req *cmd.UpdateTenantsRequest) (uint64, error)

func (*Raft) UpdateTenantsProcess added in v1.26.0 

func (s *Raft) UpdateTenantsProcess(ctx context.Context, class string, req *cmd.TenantProcessRequest) (uint64, error)

func (*Raft) UpsertRolesPermissions added in v1.28.0 

func (s *Raft) UpsertRolesPermissions(roles map[string][]authorization.Policy) error

func (*Raft) WaitUntilDBRestored added in v1.25.2 

func (s *Raft) WaitUntilDBRestored(ctx context.Context, period time.Duration, close chan struct{}) error

type Response added in v1.25.2 

type Response struct {
	Error   error
	Version uint64
}

type Service 

type Service struct {
	*Raft
	// contains filtered or unexported fields
}

Service class serves as the primary entry point for the Raft layer, managing and coordinating the key functionalities of the distributed consensus protocol.

func New 

func New(cfg Config, svrMetrics *monitoring.GRPCServerMetrics) *Service

New returns a Service configured with cfg. The service will initialize internals gRPC api & clients to other cluster nodes. Raft store will be initialized and ready to be started. To start the service call Open().

func (*Service) Close 

func (c *Service) Close(ctx context.Context) error

Close closes the raft service and frees all allocated ressources. Internal RAFT store will be closed and if leadership is assumed it will be transferred to another node. gRPC server and clients will also be closed.

func (*Service) LeaderWithID 

func (c *Service) LeaderWithID() (string, string)

LeaderWithID is used to return the current leader address and ID of the cluster. It may return empty strings if there is no current leader or the leader is unknown.

func (*Service) Open 

func (c *Service) Open(ctx context.Context, db schema.Indexer) error

Open internal RPC service to handle node communication, bootstrap the Raft node, and restore the database state

func (*Service) Ready 

func (c *Service) Ready() bool

Ready returns or not whether the node is ready to accept requests.

func (*Service) StorageCandidates added in v1.25.13 

func (c *Service) StorageCandidates() []string

type Store added in v1.25.2 

type Store struct {
	// contains filtered or unexported fields
}

Store is the implementation of RAFT on this local node. It will handle the local schema and RAFT operations (startup, bootstrap, snapshot, etc...). It ensures that a raft cluster is setup with remote node on start (either recovering from old state, or bootstrap itself based on the provided configuration).

func NewFSM added in v1.25.2 

func NewFSM(cfg Config, reg prometheus.Registerer) Store

func (*Store) Apply added in v1.25.2 

func (st *Store) Apply(l *raft.Log) interface{}

Apply is called once a log entry is committed by a majority of the cluster. Apply should apply the log to the FSM. Apply must be deterministic and produce the same result on all peers in the cluster. The returned value is returned to the client as the ApplyFuture.Response.

func (*Store) Close added in v1.25.2 

func (st *Store) Close(ctx context.Context) error

func (*Store) Execute added in v1.25.2 

func (st *Store) Execute(req *api.ApplyRequest) (uint64, error)

func (*Store) ID added in v1.25.2 

func (st *Store) ID() string

func (*Store) IsLeader added in v1.25.2 

func (st *Store) IsLeader() bool

IsLeader returns whether this node is the leader of the cluster

func (*Store) IsVoter added in v1.25.2 

func (st *Store) IsVoter() bool

func (*Store) Join added in v1.25.2 

func (st *Store) Join(id, addr string, voter bool) error

Join adds the given peer to the cluster. This operation must be executed on the leader, otherwise, it will fail with ErrNotLeader. If the cluster has not been opened yet, it will return ErrNotOpen.

func (*Store) LastAppliedCommand added in v1.25.11 

func (st *Store) LastAppliedCommand() (uint64, error)

func (*Store) Leader added in v1.25.2 

func (st *Store) Leader() string

Leader is used to return the current leader address. It may return empty strings if there is no current leader or the leader is unknown.

func (*Store) LeaderWithID added in v1.25.2 

func (st *Store) LeaderWithID() (raft.ServerAddress, raft.ServerID)

func (*Store) Notify added in v1.25.2 

func (st *Store) Notify(id, addr string) (err error)

Notify signals this Store that a node is ready for bootstrapping at the specified address. Bootstrapping will be initiated once the number of known nodes reaches the expected level, which includes this node.

func (*Store) Open added in v1.25.2 

func (st *Store) Open(ctx context.Context) (err error)

Open opens this store and marked as such. It constructs a new Raft node using the provided configuration. If there is any old state, such as snapshots, logs, peers, etc., all of those will be restored.

func (*Store) Query added in v1.25.2 

func (st *Store) Query(req *cmd.QueryRequest) (*cmd.QueryResponse, error)

func (*Store) Ready added in v1.25.2 

func (st *Store) Ready() bool

func (*Store) Remove added in v1.25.2 

func (st *Store) Remove(id string) error

Remove removes this peer from the cluster

func (*Store) Restore added in v1.25.2 

func (st *Store) Restore(rc io.ReadCloser) error

Restore is used to restore an FSM from a snapshot. It is not called concurrently with any other command. The FSM must discard all previous state before restoring the snapshot.

func (*Store) SchemaReader added in v1.25.2 

func (st *Store) SchemaReader() schema.SchemaReader

SchemaReader returns a SchemaReader from the underlying schema manager using a wait function that will make it wait for a raft log entry to be applied in the FSM Store before authorizing the read to continue.

func (*Store) SetDB added in v1.25.2 

func (st *Store) SetDB(db schema.Indexer)

func (*Store) Snapshot added in v1.25.2 

func (st *Store) Snapshot() (raft.FSMSnapshot, error)

Snapshot returns an FSMSnapshot used to: support log compaction, to restore the FSM to a previous state, or to bring out-of-date followers up to a recent log index.

The Snapshot implementation should return quickly, because Apply can not be called while Snapshot is running. Generally this means Snapshot should only capture a pointer to the state, and any expensive IO should happen as part of FSMSnapshot.Persist.

Apply and Snapshot are always called from the same thread, but Apply will be called concurrently with FSMSnapshot.Persist. This means the FSM should be implemented to allow for concurrent updates while a snapshot is happening.

func (*Store) Stats added in v1.25.2 

func (st *Store) Stats() map[string]any

Stats returns internal statistics from this store, for informational/debugging purposes only.

The statistics directly from raft are nested under the "raft" key. If the raft statistics are not yet available, then the "raft" key will not exist. See https://pkg.go.dev/github.com/hashicorp/raft#Raft.Stats for the default raft stats.

The values of "leader_address" and "leader_id" are the respective address/ID for the current leader of the cluster. They may be empty strings if there is no current leader or the leader is unknown.

The value of "ready" indicates whether this store is ready, see Store.Ready.

The value of "is_voter" indicates whether this store is a voter, see Store.IsVoter.

The value of "open" indicates whether this store is open, see Store.open.

The value of "bootstrapped" indicates whether this store has completed bootstrapping, see Store.bootstrapped.

The value of "candidates" is a map[string]string of the current candidates IDs/addresses, see Store.candidates.

The value of "last_store_log_applied_index" is the index of the last applied command found when the store was opened, see Store.lastAppliedIndexToDB.

The value of "last_applied_index" is the index of the latest update to the store, see Store.lastAppliedIndex.

The value of "db_loaded" indicates whether the DB has finished loading, see Store.dbLoaded.

Since this is for information/debugging we want to avoid enforcing unnecessary restrictions on what can go in these stats, thus we're returning map[string]any. However, any values added to this map should be able to be JSON encoded.

func (*Store) StoreSchemaV1 added in v1.25.2 

func (st *Store) StoreSchemaV1() error

StoreSchemaV1() is responsible for saving new schema (RAFT) to boltDB

func (*Store) WaitForAppliedIndex added in v1.25.2 

func (st *Store) WaitForAppliedIndex(ctx context.Context, period time.Duration, version uint64) error

WaitForAppliedIndex waits until the update with the given version is propagated to this follower node

func (*Store) WaitToRestoreDB added in v1.25.2 

func (st *Store) WaitToRestoreDB(ctx context.Context, period time.Duration, close chan struct{}) error

WaitToLoadDB waits for the DB to be loaded. The DB might be first loaded after RAFT is in a healthy state, which is when the leader has been elected and there is consensus on the log.

Source Files

View all Source files

Directories

Path Synopsis
proto
api

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.