scaler

Documentation

Index

• Variables
• type BackUpper
• type Scaler
  • func New(cl cluster, source BackUpper, c client, logger logrus.FieldLogger, ...) *Scaler
  • func (s *Scaler) LocalScaleOut(ctx context.Context, className string, dist ShardDist) error
  • func (s *Scaler) Scale(ctx context.Context, className string, updated sharding.Config, ...) (*sharding.State, error)
  • func (s *Scaler) SetSchemaManager(sm SchemaManager)
• type SchemaManager
• type ShardDist

Variables

var (
	// ErrUnresolvedName cannot resolve the host address of a node
	ErrUnresolvedName = errors.New("cannot resolve node name")
)

Types

type BackUpper

type BackUpper interface {
	// ShardsBackup returns class backup descriptor for a list of shards
	ShardsBackup(_ context.Context, id, class string, shards []string) (backup.ClassDescriptor, error)
	// ReleaseBackup releases the backup specified by its id
	ReleaseBackup(ctx context.Context, id, className string) error
}

BackUpper is used to back up shards of a specific class

type Scaler

type Scaler struct {
	// contains filtered or unexported fields
}

Scaler scales out/in class replicas.

It scales out a class by replicating its shards on new replicas

func New

func New(cl cluster, source BackUpper,
	c client, logger logrus.FieldLogger, persistenceRoot string,
) *Scaler

New returns a new instance of Scaler

func (*Scaler) LocalScaleOut

func (s *Scaler) LocalScaleOut(ctx context.Context,
	className string, dist ShardDist,
) error

LocalScaleOut syncs local shards with new replicas.

This is the meat&bones of this implementation. For each shard, we're roughly doing the following:

• Create shards backup, so the shards are safe to copy
• Figure out the copy targets (i.e. each node that is part of the after state, but wasn't part of the before state yet)
• Create an empty shard on the target node
• Copy over all files from the backup
• ReInit the shard to recognize the copied files
• Release the single-shard backup

func (*Scaler) Scale

func (s *Scaler) Scale(ctx context.Context, className string,
	updated sharding.Config, prevReplFactor, newReplFactor int64,
) (*sharding.State, error)

Scale increase/decrease class replicas.

It returns the updated sharding state if successful. The caller must then make sure to broadcast that state to all nodes as part of the "update" transaction.

func (*Scaler) SetSchemaManager

func (s *Scaler) SetSchemaManager(sm SchemaManager)

type SchemaManager

type SchemaManager interface {
	CopyShardingState(class string) *sharding.State
}

SchemaManager is used by the scaler to get and update sharding states

type ShardDist

type ShardDist map[string][]string

ShardDist shard distribution over nodes