housekeeping

package
v16.9.8 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 9, 2024 License: MIT Imports: 26 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// RepackObjectsStrategyIncrementalWithUnreachable performs an incremental repack by writing
	// all loose objects into a new packfile, regardless of their reachability. The loose
	// objects will be deleted.
	RepackObjectsStrategyIncrementalWithUnreachable = RepackObjectsStrategy("incremental_with_unreachable")
	// RepackObjectsStrategyFullWithCruft performs a full repack by writing all reachable
	// objects into a new packfile. Unreachable objects will be written into a separate cruft
	// packfile.
	RepackObjectsStrategyFullWithCruft = RepackObjectsStrategy("full_with_cruft")
	// RepackObjectsStrategyFullWithUnreachable performs a full repack by writing all reachable
	// objects into a new packfile. Packed unreachable objects will be appended to the packfile
	// and redundant loose object files will be deleted.
	//
	// Note that this will not include unreachable loose objects, but only packed loose objects.
	// git-repack(1) does not currently expose an option to make it include all objects.
	// Combined with geometric repacks though this is acceptable as the geometric strategy will
	// include all loose objects.
	RepackObjectsStrategyFullWithUnreachable = RepackObjectsStrategy("full_with_unreachable")
	// RepackObjectsStrategyGeometric performs an geometric repack. This strategy will repack
	// packfiles so that the resulting pack structure forms a geometric sequence in the number
	// of objects. Loose objects will get soaked up as part of the repack regardless of their
	// reachability.
	RepackObjectsStrategyGeometric = RepackObjectsStrategy("geometric")
)
View Source 
const (
	// WorktreesPrefix is a subdirectory in the repository where Gitaly creates worktrees.
	WorktreesPrefix = "worktrees"
	// GitlabWorktreePrefix is a subdirectory in the repository where Gitaly creates worktrees.
	GitlabWorktreePrefix = "gitlab-worktree"
)
View Source 
const (
	// FullRepackCooldownPeriod is the cooldown period that needs to pass since the last full
	// repack before we consider doing another full repack.
	FullRepackCooldownPeriod = 5 * 24 * time.Hour
)

Variables

This section is empty.

Functions

func CleanupWorktrees 

func CleanupWorktrees(ctx context.Context, repo *localrepo.Repo) error

CleanupWorktrees cleans up stale and disconnected worktrees for the given repository.

func GetRepackGitConfig 

func GetRepackGitConfig(ctx context.Context, repo storage.Repository, bitmap bool) []git.ConfigPair

GetRepackGitConfig returns configuration suitable for Git commands which write new packfiles.

func PerformFullRepackingWithUnreachable added in v16.9.0 

func PerformFullRepackingWithUnreachable(ctx context.Context, repo *localrepo.Repo, cfg RepackObjectsConfig) error

PerformFullRepackingWithUnreachable performs a full repacking task using git-repack(1) command. This will omit packing objects part of alternates.

func PerformGeometricRepacking added in v16.9.0 

func PerformGeometricRepacking(ctx context.Context, repo *localrepo.Repo, cfg RepackObjectsConfig) error

PerformGeometricRepacking performs geometric repacking task using git-repack(1) command. It allows us to merge multiple packfiles without having to rewrite all packfiles into one. This new "geometric" strategy tries to ensure that existing packfiles in the repository form a geometric sequence where each successive packfile contains at least n times as many objects as the preceding packfile. If the sequence isn't maintained, Git will determine a slice of packfiles that it must repack to maintain the sequence again. With this process, we can limit the number of packfiles that exist in the repository without having to repack all objects into a single packfile regularly. This repacking does not take reachability into account. For more information, https://about.gitlab.com/blog/2023/11/02/rearchitecting-git-object-database-mainentance-for-scale/#geometric-repacking

func PerformRepack added in v16.9.0 

func PerformRepack(ctx context.Context, repo *localrepo.Repo, cfg RepackObjectsConfig, opts ...git.Option) error

PerformRepack performs `git-repack(1)` command on a repository with some pre-built configs.

func PruneObjects 

func PruneObjects(ctx context.Context, repo *localrepo.Repo, cfg PruneObjectsConfig) error

PruneObjects prunes loose objects from the repository that are already packed or which are unreachable and older than the configured expiry date.

func RepackObjects 

func RepackObjects(ctx context.Context, repo *localrepo.Repo, cfg RepackObjectsConfig) error

RepackObjects repacks objects in the given repository and updates the commit-graph. The way objects are repacked is determined via the RepackObjectsConfig.

func ValidateRepacking added in v16.9.0 

func ValidateRepacking(cfg RepackObjectsConfig) (bool, error)

ValidateRepacking validates the input repacking config. This function any validating error and if the configuration is for full repack.

func WriteCommitGraph 

func WriteCommitGraph(ctx context.Context, repo *localrepo.Repo, cfg WriteCommitGraphConfig) error

WriteCommitGraph updates the commit-graph in the given repository. The commit-graph is updated incrementally, except in the case where it doesn't exist yet or in case it is detected that the commit-graph is missing bloom filters.

Types

type CleanStaleDataConfig 

type CleanStaleDataConfig struct {
	// contains filtered or unexported fields
}

CleanStaleDataConfig is the configuration for running CleanStaleData. It is used to define the different types of cleanups we want to run.

func DefaultStaleDataCleanup 

func DefaultStaleDataCleanup() CleanStaleDataConfig

DefaultStaleDataCleanup is the default configuration for CleanStaleData which contains all the cleanup functions.

func OnlyStaleReferenceLockCleanup 

func OnlyStaleReferenceLockCleanup(gracePeriod time.Duration) CleanStaleDataConfig

OnlyStaleReferenceLockCleanup returns a config which only contains a stale reference lock cleaner with the provided grace period.

type EagerOptimizationStrategy 

type EagerOptimizationStrategy struct {
	// contains filtered or unexported fields
}

EagerOptimizationStrategy is a strategy that will eagerly perform optimizations. All of the data structures will be optimized regardless of whether they already are in an optimal state or not.

func NewEagerOptimizationStrategy 

func NewEagerOptimizationStrategy(info stats.RepositoryInfo) EagerOptimizationStrategy

NewEagerOptimizationStrategy creates a new EagerOptimizationStrategy.

func (EagerOptimizationStrategy) ShouldPruneObjects 

func (s EagerOptimizationStrategy) ShouldPruneObjects(context.Context) (bool, PruneObjectsConfig)

ShouldPruneObjects always instructs the caller to prune objects, unless the repository is an object pool.

func (EagerOptimizationStrategy) ShouldRepackObjects 

func (s EagerOptimizationStrategy) ShouldRepackObjects(ctx context.Context) (bool, RepackObjectsConfig)

ShouldRepackObjects always instructs the caller to repack objects. The strategy will always be to repack all objects into a single packfile. The bitmap will be written in case the repository does not have any alternates.

func (EagerOptimizationStrategy) ShouldRepackReferences 

func (s EagerOptimizationStrategy) ShouldRepackReferences(context.Context) bool

ShouldRepackReferences always instructs the caller to repack references.

func (EagerOptimizationStrategy) ShouldWriteCommitGraph 

func (s EagerOptimizationStrategy) ShouldWriteCommitGraph(context.Context) (bool, WriteCommitGraphConfig)

ShouldWriteCommitGraph always instructs the caller to write the commit-graph. The strategy will always be to completely rewrite the commit-graph chain.

type HeuristicalOptimizationStrategy 

type HeuristicalOptimizationStrategy struct {
	// contains filtered or unexported fields
}

HeuristicalOptimizationStrategy is an optimization strategy that is based on a set of heuristics.

func NewHeuristicalOptimizationStrategy 

func NewHeuristicalOptimizationStrategy(gitVersion git.Version, info stats.RepositoryInfo) HeuristicalOptimizationStrategy

NewHeuristicalOptimizationStrategy constructs a heuristicalOptimizationStrategy for the given repository info. It derives all data from the repository so that the heuristics used by this repository can be decided without further disk reads.

func (HeuristicalOptimizationStrategy) ShouldPruneObjects 

func (s HeuristicalOptimizationStrategy) ShouldPruneObjects(context.Context) (bool, PruneObjectsConfig)

ShouldPruneObjects determines whether the repository has stale objects that should be pruned. Object pools are never pruned to not lose data in them, but otherwise we prune when we've found enough stale objects that might in fact get pruned.

func (HeuristicalOptimizationStrategy) ShouldRepackObjects 

func (s HeuristicalOptimizationStrategy) ShouldRepackObjects(ctx context.Context) (bool, RepackObjectsConfig)

ShouldRepackObjects checks whether the repository's objects need to be repacked. This uses a set of heuristics that scales with the size of the object database: the larger the repository, the less frequent does it get a full repack.

func (HeuristicalOptimizationStrategy) ShouldRepackReferences 

func (s HeuristicalOptimizationStrategy) ShouldRepackReferences(context.Context) bool

ShouldRepackReferences determines whether the repository's references need to be repacked based on heuristics. The more references there are, the more loose referencos may exist until they are packed again.

func (HeuristicalOptimizationStrategy) ShouldWriteCommitGraph 

func (s HeuristicalOptimizationStrategy) ShouldWriteCommitGraph(ctx context.Context) (bool, WriteCommitGraphConfig)

ShouldWriteCommitGraph determines whether we need to write the commit-graph and how it should be written.

type Manager 

type Manager interface {
	// CleanStaleData removes any stale data in the repository as per the provided configuration.
	CleanStaleData(context.Context, *localrepo.Repo, CleanStaleDataConfig) error
	// OptimizeRepository optimizes the repository's data structures such that it can be more
	// efficiently served.
	OptimizeRepository(context.Context, *localrepo.Repo, ...OptimizeRepositoryOption) error
	// AddPackRefsInhibitor allows clients to block housekeeping from running git-pack-refs(1).
	AddPackRefsInhibitor(ctx context.Context, repoPath string) (bool, func(), error)
}

Manager is a housekeeping manager. It is supposed to handle housekeeping tasks for repositories such as the cleanup of unneeded files and optimizations for the repository's data structures.

type OptimizationStrategy 

type OptimizationStrategy interface {
	// ShouldRepackObjects determines whether the repository needs to be repacked and, if so,
	// how it should be done.
	ShouldRepackObjects(context.Context) (bool, RepackObjectsConfig)
	// ShouldPruneObjects determines whether the repository has stale objects that should be
	// pruned and, if so, how it should be done.
	ShouldPruneObjects(context.Context) (bool, PruneObjectsConfig)
	// ShouldRepackReferences determines whether the repository's references need to be
	// repacked.
	ShouldRepackReferences(context.Context) bool
	// ShouldWriteCommitGraph determines whether we need to write the commit-graph and how it
	// should be written.
	ShouldWriteCommitGraph(context.Context) (bool, WriteCommitGraphConfig)
}

OptimizationStrategy is an interface to determine which parts of a repository should be optimized.

type OptimizationStrategyConstructor 

type OptimizationStrategyConstructor func(stats.RepositoryInfo) OptimizationStrategy

OptimizationStrategyConstructor is a constructor for an OptimizationStrategy that is being informed by the passed-in RepositoryInfo.

type OptimizeRepositoryConfig 

type OptimizeRepositoryConfig struct {
	StrategyConstructor OptimizationStrategyConstructor
}

OptimizeRepositoryConfig is the configuration used by OptimizeRepository that is computed by applying all the OptimizeRepositoryOption modifiers.

type OptimizeRepositoryOption 

type OptimizeRepositoryOption func(cfg *OptimizeRepositoryConfig)

OptimizeRepositoryOption is an option that can be passed to OptimizeRepository.

func WithOptimizationStrategyConstructor 

func WithOptimizationStrategyConstructor(strategyConstructor OptimizationStrategyConstructor) OptimizeRepositoryOption

WithOptimizationStrategyConstructor changes the constructor for the optimization strategy.that is used to determine which parts of the repository will be optimized. By default the HeuristicalOptimizationStrategy is used.

type PruneObjectsConfig 

type PruneObjectsConfig struct {
	// ExpireBefore controls the grace period after which unreachable objects shall be pruned.
	// An unreachable object must be older than the given date in order to be considered for
	// deletion.
	ExpireBefore time.Time
}

PruneObjectsConfig determines which objects should be pruned in PruneObjects.

type RepackObjectsConfig 

type RepackObjectsConfig struct {
	// Strategy determines the strategy with which to repack objects.
	Strategy RepackObjectsStrategy
	// WriteBitmap determines whether reachability bitmaps should be written or not. There is no
	// reason to set this to `false`, except for legacy compatibility reasons with existing RPC
	// behaviour
	WriteBitmap bool
	// WriteMultiPackIndex determines whether a multi-pack index should be written or not.
	WriteMultiPackIndex bool
	// CruftExpireBefore determines the cutoff date before which unreachable cruft objects shall
	// be expired and thus deleted.
	CruftExpireBefore time.Time
}

RepackObjectsConfig is configuration for RepackObjects.

type RepackObjectsStrategy 

type RepackObjectsStrategy string

RepackObjectsStrategy defines how objects shall be repacked.

type RepositoryManager 

type RepositoryManager struct {
	// contains filtered or unexported fields
}

RepositoryManager is an implementation of the Manager interface.

func NewManager 

func NewManager(promCfg gitalycfgprom.Config, logger log.Logger, txManager transaction.Manager) *RepositoryManager

NewManager creates a new RepositoryManager.

func (*RepositoryManager) AddPackRefsInhibitor added in v16.1.0 

func (m *RepositoryManager) AddPackRefsInhibitor(ctx context.Context, repoPath string) (successful bool, _ func(), err error)

AddPackRefsInhibitor exposes the internal function addPackRefsInhibitor on the RepositoryManager level. This can then be used by other clients to block housekeeping from running git-pack-refs(1).

func (*RepositoryManager) CleanStaleData 

func (m *RepositoryManager) CleanStaleData(ctx context.Context, repo *localrepo.Repo, cfg CleanStaleDataConfig) error

CleanStaleData removes any stale data in the repository as per the provided configuration.

func (*RepositoryManager) Collect 

func (m *RepositoryManager) Collect(metrics chan<- prometheus.Metric)

Collect is used to collect Prometheus metrics.

func (*RepositoryManager) Describe 

func (m *RepositoryManager) Describe(descs chan<- *prometheus.Desc)

Describe is used to describe Prometheus metrics.

func (*RepositoryManager) OptimizeRepository 

func (m *RepositoryManager) OptimizeRepository(
	ctx context.Context,
	repo *localrepo.Repo,
	opts ...OptimizeRepositoryOption,
) error

OptimizeRepository performs optimizations on the repository. Whether optimizations are performed or not depends on a set of heuristics.

type WriteCommitGraphConfig 

type WriteCommitGraphConfig struct {
	// ReplaceChain causes WriteCommitGraph to rewrite the complete commit-graph chain. This is
	// a lot more expensive than the default, incremental update of the commit-graph chains but
	// may be required in certain cases to fix up commit-graphs.
	ReplaceChain bool
}

WriteCommitGraphConfig contains configuration that can be passed to WriteCommitGraph to alter its default behaviour.

func WriteCommitGraphConfigForRepository 

func WriteCommitGraphConfigForRepository(ctx context.Context, repo *localrepo.Repo) (WriteCommitGraphConfig, error)

WriteCommitGraphConfigForRepository returns the optimal default-configuration for the given repo. By default, the configuration will ask for an incremental commit-graph update. If the preexisting commit-graph is missing bloom filters though then the whole commit-graph chain will be rewritten.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.