datastore

package
v1.82.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 17, 2020 License: MIT Imports: 12 Imported by: 0

Documentation

Overview

Package datastore provides data models and datastore persistence abstractions for tracking the state of repository replicas.

See original design discussion: https://gitlab.com/gitlab-org/gitaly/issues/1495

Index

Constants

This section is empty.

Variables

View Source 
var ErrInvalidReplTarget = errors.New("targetStorage repository fails preconditions for replication")

ErrInvalidReplTarget indicates a targetStorage repository cannot be chosen because it fails preconditions for being replicatable

View Source 
var (
	// ErrPrimaryNotSet indicates the primary has not been set in the datastore
	ErrPrimaryNotSet = errors.New("primary is not set")
)
View Source 
var ErrReplicasMissing = errors.New("repository missing secondary replicas")

ErrReplicasMissing indicates the repository does not have any backup replicas

Functions

func CheckPostgresVersion added in v1.76.0 

func CheckPostgresVersion(conf config.Config) error

CheckPostgresVersion checks the server version of the Postgres DB specified in conf. This is a diagnostic for the Praefect Postgres rollout. https://gitlab.com/gitlab-org/gitaly/issues/1755

func Migrate added in v1.81.0 

func Migrate(conf config.Config) (int, error)

Migrate will apply all pending SQL migrations

Types

type ChangeType 

type ChangeType int

ChangeType indicates what kind of change the replication is propagating 

const (
	// UpdateRepo is when a replication updates a repository in place
	UpdateRepo ChangeType = iota + 1
	// DeleteRepo is when a replication deletes a repo
	DeleteRepo
)

type Datastore 

type Datastore interface {
	ReplJobsDatastore
	ReplicasDatastore
}

Datastore is a data persistence abstraction for all of Praefect's persistence needs

type JobState 

type JobState uint8

JobState is an enum that indicates the state of a job 

const (
	// JobStatePending is the initial job state when it is not yet ready to run
	// and may indicate recovery from a failure prior to the ready-state
	JobStatePending JobState = 1 << iota
	// JobStateReady indicates the job is now ready to proceed
	JobStateReady
	// JobStateInProgress indicates the job is being processed by a worker
	JobStateInProgress
	// JobStateComplete indicates the job is now complete
	JobStateComplete
	// JobStateCancelled indicates the job was cancelled. This can occur if the
	// job is no longer relevant (e.g. a node is moved out of a repository)
	JobStateCancelled
)

type MemoryDatastore 

type MemoryDatastore struct {
	// contains filtered or unexported fields
}

MemoryDatastore is a simple datastore that isn't persisted to disk. It is only intended for early beta requirements and as a reference implementation for the eventual SQL implementation

func NewInMemory 

func NewInMemory(cfg config.Config) *MemoryDatastore

NewInMemory returns an initialized in-memory datastore

func (*MemoryDatastore) AddReplica 

func (md *MemoryDatastore) AddReplica(relativePath, nodeStorage string) error

AddReplica adds a secondary to a repository of a repository relative path

func (*MemoryDatastore) CreateReplicaReplJobs 

func (md *MemoryDatastore) CreateReplicaReplJobs(relativePath string, change ChangeType) ([]uint64, error)

CreateReplicaReplJobs creates a replication job for each secondary that backs the specified repository. Upon success, the job IDs will be returned.

func (*MemoryDatastore) GetJobs 

func (md *MemoryDatastore) GetJobs(state JobState, targetNodeStorage string, count int) ([]ReplJob, error)

GetJobs is a more general method to retrieve jobs of a certain state from the datastore

func (*MemoryDatastore) GetPrimary 

func (md *MemoryDatastore) GetPrimary(relativePath string) (*models.Node, error)

GetPrimary gets the primary storage node for a repository of a repository relative path

func (*MemoryDatastore) GetReplicas 

func (md *MemoryDatastore) GetReplicas(relativePath string) ([]models.Node, error)

GetReplicas gets the secondaries for a repository based on the relative path

func (*MemoryDatastore) GetRepository 

func (md *MemoryDatastore) GetRepository(relativePath string) (*models.Repository, error)

GetRepository gets the repository for a repository relative path

func (*MemoryDatastore) GetStorageNode 

func (md *MemoryDatastore) GetStorageNode(nodeStorage string) (models.Node, error)

GetStorageNode gets all storage nodes

func (*MemoryDatastore) GetStorageNodes 

func (md *MemoryDatastore) GetStorageNodes() ([]models.Node, error)

GetStorageNodes gets all storage nodes

func (*MemoryDatastore) PickAPrimary 

func (md *MemoryDatastore) PickAPrimary(virtualStorage string) (*models.Node, error)

PickAPrimary returns the primary configured in the config file

func (*MemoryDatastore) RemoveReplica 

func (md *MemoryDatastore) RemoveReplica(relativePath, nodeStorage string) error

RemoveReplica removes a secondary from a repository of a repository relative path

func (*MemoryDatastore) SetPrimary 

func (md *MemoryDatastore) SetPrimary(relativePath, nodeStorage string) error

SetPrimary sets the primary storagee node for a repository of a repository relative path

func (*MemoryDatastore) UpdateReplJob 

func (md *MemoryDatastore) UpdateReplJob(jobID uint64, newState JobState) error

UpdateReplJob updates an existing replication job's state

type ReplJob 

type ReplJob struct {
	Change                 ChangeType
	ID                     uint64            // autoincrement ID
	TargetNode, SourceNode models.Node       // which node to replicate to?
	Repository             models.Repository // source for replication
	State                  JobState
}

ReplJob is an instance of a queued replication job. A replication job is meant for updating the repository so that it is synced with the primary copy. Scheduled indicates when a replication job should be performed.

type ReplJobsDatastore 

type ReplJobsDatastore interface {
	// GetJobs fetches a list of chronologically ordered replication
	// jobs for the given storage replica. The returned list will be at most
	// count-length.
	GetJobs(flag JobState, nodeStorage string, count int) ([]ReplJob, error)

	// CreateReplicaJobs will create replication jobs for each secondary
	// replica of a repository known to the datastore. A set of replication job
	// ID's for the created jobs will be returned upon success.
	CreateReplicaReplJobs(relativePath string, change ChangeType) ([]uint64, error)

	// UpdateReplJob updates the state of an existing replication job
	UpdateReplJob(jobID uint64, newState JobState) error
}

ReplJobsDatastore represents the behavior needed for fetching and updating replication jobs from the datastore

type ReplicasDatastore 

type ReplicasDatastore interface {
	PickAPrimary(virtualStorage string) (*models.Node, error)

	GetReplicas(relativePath string) ([]models.Node, error)

	GetStorageNode(nodeStorage string) (models.Node, error)

	GetStorageNodes() ([]models.Node, error)

	GetPrimary(relativePath string) (*models.Node, error)

	SetPrimary(relativePath, nodeStorage string) error

	AddReplica(relativePath string, nodeStorage string) error

	RemoveReplica(relativePath, nodeStorage string) error

	GetRepository(relativePath string) (*models.Repository, error)
}

ReplicasDatastore manages accessing and setting which secondary replicas backup a repository

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.