maintenance

package
v0.17.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 16, 2024 License: Apache-2.0 Imports: 28 Imported by: 2

Documentation

Overview

Package maintenance manages automatic repository maintenance.

Index

Constants

View Source 
const (
	TaskSnapshotGarbageCollection    = "snapshot-gc"
	TaskDeleteOrphanedBlobsQuick     = "quick-delete-blobs"
	TaskDeleteOrphanedBlobsFull      = "full-delete-blobs"
	TaskRewriteContentsQuick         = "quick-rewrite-contents"
	TaskRewriteContentsFull          = "full-rewrite-contents"
	TaskDropDeletedContentsFull      = "full-drop-deleted-content"
	TaskIndexCompaction              = "index-compaction"
	TaskExtendBlobRetentionTimeFull  = "extend-blob-retention-time"
	TaskCleanupLogs                  = "cleanup-logs"
	TaskEpochAdvance                 = "advance-epoch"
	TaskEpochDeleteSupersededIndexes = "delete-superseded-epoch-indexes"
	TaskEpochCleanupMarkers          = "cleanup-epoch-markers"
	TaskEpochGenerateRange           = "generate-epoch-range-index"
	TaskEpochCompactSingle           = "compact-single-epoch"
)

Task IDs.

Variables

View Source 
var (
	// SafetyNone has safety parameters which allow full garbage collection without unnecessary
	// delays, but it is safe only if no other kopia clients are running and storage backend is
	// strongly consistent.
	SafetyNone = SafetyParameters{
		BlobDeleteMinAge:                 0,
		DropContentFromIndexExtraMargin:  0,
		MarginBetweenSnapshotGC:          0,
		MinContentAgeSubjectToGC:         0,
		RewriteMinAge:                    0,
		SessionExpirationAge:             0,
		RequireTwoGCCycles:               false,
		DisableEventualConsistencySafety: true,
	}

	// SafetyFull has default safety parameters which allow safe GC concurrent with snapshotting
	// by other Kopia clients.
	SafetyFull = SafetyParameters{
		BlobDeleteMinAge:                24 * time.Hour,
		DropContentFromIndexExtraMargin: time.Hour,
		MarginBetweenSnapshotGC:         4 * time.Hour,
		MinContentAgeSubjectToGC:        24 * time.Hour,
		RewriteMinAge:                   2 * time.Hour,
		SessionExpirationAge:            96 * time.Hour,
		RequireTwoGCCycles:              true,
		MinRewriteToOrphanDeletionDelay: time.Hour,
	}
)

Supported safety levels.

Functions

func CheckExtendRetention added in v0.14.0 

func CheckExtendRetention(ctx context.Context, blobCfg format.BlobStorageConfiguration, p *Params) error

CheckExtendRetention verifies if extension can be enabled due to maintenance and blob parameters.

func CleanupLogs added in v0.9.0 

func CleanupLogs(ctx context.Context, rep repo.DirectRepositoryWriter, opt LogRetentionOptions) ([]blob.Metadata, error)

CleanupLogs deletes old logs blobs beyond certain age, total size or count.

func DeleteUnreferencedBlobs 

func DeleteUnreferencedBlobs(ctx context.Context, rep repo.DirectRepositoryWriter, opt DeleteUnreferencedBlobsOptions, safety SafetyParameters) (int, error)

DeleteUnreferencedBlobs deletes o was created after maintenance startederenced by index entries.

func DropDeletedContents 

func DropDeletedContents(ctx context.Context, rep repo.DirectRepositoryWriter, dropDeletedBefore time.Time, safety SafetyParameters) error

DropDeletedContents rewrites indexes while dropping deleted contents above certain age.

func ExtendBlobRetentionTime added in v0.14.0 

func ExtendBlobRetentionTime(ctx context.Context, rep repo.DirectRepositoryWriter, opt ExtendBlobRetentionTimeOptions) (int, error)

ExtendBlobRetentionTime extends the retention time of all relevant blobs managed by storage engine with Object Locking enabled.

func HasParams 

func HasParams(ctx context.Context, rep repo.Repository) (bool, error)

HasParams determines whether repository-wide maintenance parameters have been set.

func IsOwnedByThisUser added in v0.8.4 

func IsOwnedByThisUser(ctx context.Context, rep repo.Repository) (bool, error)

IsOwnedByThisUser determines whether current user is the maintenance owner.

func ReportRun 

func ReportRun(ctx context.Context, rep repo.DirectRepositoryWriter, taskType TaskType, s *Schedule, run func() error) error

ReportRun reports timing of a maintenance run and persists it in repository.

func RewriteContents 

func RewriteContents(ctx context.Context, rep repo.DirectRepositoryWriter, opt *RewriteContentsOptions, safety SafetyParameters) error

RewriteContents rewrites contents according to provided criteria and creates new blobs and index entries to point at them.

func Run 

func Run(ctx context.Context, runParams RunParameters, safety SafetyParameters) error

Run performs maintenance activities for a repository.

func RunExclusive 

func RunExclusive(ctx context.Context, rep repo.DirectRepositoryWriter, mode Mode, force bool, cb func(ctx context.Context, runParams RunParameters) error) error

RunExclusive runs the provided callback if the maintenance is owned by local user and lock can be acquired. Lock is passed to the function, which ensures that every call to Run() is within the exclusive context.

func SetParams 

func SetParams(ctx context.Context, rep repo.RepositoryWriter, par *Params) error

SetParams sets the maintenance parameters.

func SetSchedule 

func SetSchedule(ctx context.Context, rep repo.DirectRepositoryWriter, s *Schedule) error

SetSchedule updates scheduled maintenance times.

func TimeToAttemptNextMaintenance added in v0.9.5 

func TimeToAttemptNextMaintenance(ctx context.Context, rep repo.DirectRepository) (time.Time, error)

TimeToAttemptNextMaintenance returns the time when we should attempt next maintenance. if the maintenance is not owned by this user, returns time.Time{}.

Types

type CycleParams 

type CycleParams struct {
	Enabled  bool          `json:"enabled"`
	Interval time.Duration `json:"interval"`
}

CycleParams specifies parameters for a maintenance cycle (quick or full).

type DeleteUnreferencedBlobsOptions 

type DeleteUnreferencedBlobsOptions struct {
	Parallel     int
	Prefix       blob.ID
	DryRun       bool
	NotAfterTime time.Time
}

DeleteUnreferencedBlobsOptions provides option for blob garbage collection algorithm.

type ExtendBlobRetentionTimeOptions added in v0.14.0 

type ExtendBlobRetentionTimeOptions struct {
	Parallel int
	DryRun   bool
}

ExtendBlobRetentionTimeOptions provides options for extending blob retention algorithm.

type LogRetentionOptions added in v0.9.0 

type LogRetentionOptions struct {
	MaxTotalSize int64            `json:"maxTotalSize"`
	MaxCount     int              `json:"maxCount"`
	MaxAge       time.Duration    `json:"maxAge"`
	DryRun       bool             `json:"-"`
	TimeFunc     func() time.Time `json:"-"`
}

LogRetentionOptions provides options for logs retention.

func (LogRetentionOptions) OrDefault added in v0.9.0 

func (o LogRetentionOptions) OrDefault() LogRetentionOptions

OrDefault returns default LogRetentionOptions.

type Mode 

type Mode string

Mode describes the mode of maintenance to perform. 

const (
	ModeNone  Mode = "none"
	ModeQuick Mode = "quick"
	ModeFull  Mode = "full"
	ModeAuto  Mode = "auto" // run either quick of full if required by schedule
)

Supported maintenance modes.

type NotOwnedError 

type NotOwnedError struct {
	Owner string
}

NotOwnedError is returned when maintenance cannot run because it is owned by another user.

func (NotOwnedError) Error 

func (e NotOwnedError) Error() string

type Params 

type Params struct {
	Owner string `json:"owner"`

	QuickCycle CycleParams `json:"quick"`
	FullCycle  CycleParams `json:"full"`

	LogRetention LogRetentionOptions `json:"logRetention"`

	ExtendObjectLocks bool `json:"extendObjectLocks"`
}

Params is a JSON-serialized maintenance configuration stored in a repository.

func DefaultParams 

func DefaultParams() Params

DefaultParams represents default values of maintenance parameters.

func GetParams 

func GetParams(ctx context.Context, rep repo.Repository) (*Params, error)

GetParams returns repository-wide maintenance parameters.

type RewriteContentsOptions 

type RewriteContentsOptions struct {
	Parallel       int
	ContentIDs     []content.ID
	ContentIDRange content.IDRange
	PackPrefix     blob.ID
	ShortPacks     bool
	FormatVersion  int
	DryRun         bool
}

RewriteContentsOptions provides options for RewriteContents.

type RunInfo 

type RunInfo struct {
	Start   time.Time `json:"start"`
	End     time.Time `json:"end"`
	Success bool      `json:"success,omitempty"`
	Error   string    `json:"error,omitempty"`
}

RunInfo represents information about a single run of a maintenance task.

type RunParameters 

type RunParameters struct {
	Mode Mode

	Params *Params

	// timestamp of the last update of maintenance schedule blob
	MaintenanceStartTime time.Time
	// contains filtered or unexported fields
}

RunParameters passes essential parameters for maintenance. It is generated by RunExclusive and can't be create outside of its package and is required to ensure all maintenance tasks run under an exclusive lock.

type SafetyParameters added in v0.8.0 

type SafetyParameters struct {
	// Do not rewrite contents younger than this age.
	RewriteMinAge time.Duration

	// Snapshot GC: MinContentAgeSubjectToGC is the minimum age of content to be subject to garbage collection.
	MinContentAgeSubjectToGC time.Duration

	// MarginBetweenSnapshotGC is the minimal amount of time that must pass between snapshot
	// GC cycles to allow all in-flight snapshots during earlier GC to be flushed and
	// visible to a following GC. The uploader will automatically create a checkpoint every 45 minutes,
	// so ~1 hour should be enough but we're setting this to a higher conservative value for extra safety.
	MarginBetweenSnapshotGC time.Duration

	// RequireTwoGCCycles indicates that two GC cycles are required.
	RequireTwoGCCycles bool

	// DisableEventualConsistencySafety disables wait time to allow settling of eventually-consistent writes in blob stores.
	DisableEventualConsistencySafety bool

	// DropContentFromIndexExtraMargin is the amount of margin time before dropping deleted contents from indices.
	DropContentFromIndexExtraMargin time.Duration

	// Blob GC: Delete unused blobs above this age.
	BlobDeleteMinAge time.Duration

	// Blob GC: Drop incomplete session blobs above this age.
	SessionExpirationAge time.Duration

	// Minimum time that must pass after content rewrite before we delete orphaned blobs.
	MinRewriteToOrphanDeletionDelay time.Duration
}

SafetyParameters specifies timing parameters that affect safety of maintenance.

type Schedule 

type Schedule struct {
	NextFullMaintenanceTime  time.Time `json:"nextFullMaintenance"`
	NextQuickMaintenanceTime time.Time `json:"nextQuickMaintenance"`

	Runs map[TaskType][]RunInfo `json:"runs"`
}

Schedule keeps track of scheduled maintenance times.

func GetSchedule 

func GetSchedule(ctx context.Context, rep repo.DirectRepository) (*Schedule, error)

GetSchedule gets the scheduled maintenance times.

func (*Schedule) ReportRun 

func (s *Schedule) ReportRun(taskType TaskType, info RunInfo)

ReportRun adds the provided run information to the history and discards oldest entried.

type TaskType added in v0.8.0 

type TaskType string

TaskType identifies the type of a maintenance task.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.