tasks

Documentation

Index

• func IssueNewPasswordForPeer(ctx context.Context, cfg keppel.Configuration, db *keppel.DB, ...) (resultErr error)
• type Janitor
  • func NewJanitor(cfg keppel.Configuration, fd keppel.FederationDriver, sd keppel.StorageDriver, ...) *Janitor
  • func (j *Janitor) AbandonedUploadCleanupJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) AccountFederationAnnouncementJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) BlobMountSweepJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) BlobSweepJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) BlobValidationJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) CheckTrivySecurityStatusJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) DeleteAccountsJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) DisableJitter()
  • func (j *Janitor) EnforceManagedAccountsJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) ManifestGarbageCollectionJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) ManifestSyncJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) ManifestValidationJob(registerer prometheus.Registerer) jobloop.Job
  • func (j *Janitor) OverrideGenerateStorageID(generateStorageID func() string) *Janitor
  • func (j *Janitor) OverrideTimeNow(timeNow func() time.Time) *Janitor
  • func (j *Janitor) StorageSweepJob(registerer prometheus.Registerer) jobloop.Job

Functions

func IssueNewPasswordForPeer

func IssueNewPasswordForPeer(ctx context.Context, cfg keppel.Configuration, db *keppel.DB, tx *gorp.Transaction, peer models.Peer) (resultErr error)

IssueNewPasswordForPeer issues a new replication password for the given peer.

The `tx` argument can be given if the caller already has a transaction open for this operation. This is useful because it is the caller's responsibility to lock the database row for the peer to prevent concurrent issuances for the same peer by different keppel-api instances.

Types

type Janitor

type Janitor struct {
	// contains filtered or unexported fields
}

Janitor contains the toolbox of the keppel-janitor process.

func NewJanitor

func NewJanitor(cfg keppel.Configuration, fd keppel.FederationDriver, sd keppel.StorageDriver, icd keppel.InboundCacheDriver, db *keppel.DB, amd keppel.AccountManagementDriver, auditor audittools.Auditor) *Janitor

NewJanitor creates a new Janitor.

func (*Janitor) AbandonedUploadCleanupJob

func (j *Janitor) AbandonedUploadCleanupJob(registerer prometheus.Registerer) jobloop.Job

AbandonedUploadCleanupJob is a job. Each task finds an upload that has not been updated for more than a day, and cleans it up.

func (*Janitor) AccountFederationAnnouncementJob

func (j *Janitor) AccountFederationAnnouncementJob(registerer prometheus.Registerer) jobloop.Job

AccountFederationAnnouncementJob is a job. Each task finds an account that has not been announced to the FederationDriver in more than an hour, and announces it. If no accounts need to be announced, sql.ErrNoRows is returned to instruct the caller to slow down.

func (*Janitor) BlobMountSweepJob

func (j *Janitor) BlobMountSweepJob(registerer prometheus.Registerer) jobloop.Job

BlobMountSweepJob is a job. Each task finds one repo where blob mounts need to be garbage-collected, and performs the GC. This entails a marking of all blob mounts that are not used by any manifest, and a sweeping of all blob mounts that were marked in the previous pass and which are still not used by any manifest.

This staged mark-and-sweep ensures that we don't remove fresh blob mounts that were just created, but where the manifest has not yet been pushed.

Blob mounts are sweeped in each repo at most once per hour.

func (*Janitor) BlobSweepJob

func (j *Janitor) BlobSweepJob(registerer prometheus.Registerer) jobloop.Job

BlobSweepJob is a job. Each task finds one account where blobs need to be garbage-collected, and performs the GC. This entails a marking of all blobs that are not mounted in any repo, and a sweeping of all blobs that were marked in the previous pass and which are still not mounted anywhere.

This staged mark-and-sweep ensures that we don't remove fresh blobs that were just pushed and have not been mounted anywhere.

Blobs are sweeped in each account at most once per hour.

func (*Janitor) BlobValidationJob

func (j *Janitor) BlobValidationJob(registerer prometheus.Registerer) jobloop.Job

BlobValidationJob is a job. Each task validates a blob that has not been validated for more than 7 days.

func (*Janitor) CheckTrivySecurityStatusJob

func (j *Janitor) CheckTrivySecurityStatusJob(registerer prometheus.Registerer) jobloop.Job

func (*Janitor) DeleteAccountsJob

func (j *Janitor) DeleteAccountsJob(registerer prometheus.Registerer) jobloop.Job

EnforceManagedAccounts is a job. Each task creates newly discovered accounts from the driver.

func (*Janitor) DisableJitter

func (j *Janitor) DisableJitter()

DisableJitter replaces addJitter with a no-op for this Janitor.

func (*Janitor) EnforceManagedAccountsJob

func (j *Janitor) EnforceManagedAccountsJob(registerer prometheus.Registerer) jobloop.Job

EnforceManagedAccounts is a job. Each task creates newly discovered accounts from the driver.

func (*Janitor) ManifestGarbageCollectionJob

func (j *Janitor) ManifestGarbageCollectionJob(registerer prometheus.Registerer) jobloop.Job

ManifestGarbageCollectionJob is a job. Each task finds the a where GC has not been performed for more than an hour, and performs GC based on the GC policies configured on the repo's account.

func (*Janitor) ManifestSyncJob

func (j *Janitor) ManifestSyncJob(registerer prometheus.Registerer) jobloop.Job

ManifestSyncJob is a job. Each task finds a repository in a replica account where manifests have not been synced for more than an hour, and syncs its manifests. Syncing involves checking with the primary account which manifests have been deleted there, and replicating the deletions on our side.

func (*Janitor) ManifestValidationJob

func (j *Janitor) ManifestValidationJob(registerer prometheus.Registerer) jobloop.Job

ManifestValidationJob is a job. Each task validates a manifest that has not been validated for more than 24 hours.

func (*Janitor) OverrideGenerateStorageID

func (j *Janitor) OverrideGenerateStorageID(generateStorageID func() string) *Janitor

OverrideGenerateStorageID replaces keppel.GenerateStorageID with a test double.

func (*Janitor) OverrideTimeNow

func (j *Janitor) OverrideTimeNow(timeNow func() time.Time) *Janitor

OverrideTimeNow replaces time.Now with a test double.

func (*Janitor) StorageSweepJob

func (j *Janitor) StorageSweepJob(registerer prometheus.Registerer) jobloop.Job

SweepStorageJob is a job. Each task finds an account where the backing storage needs to be garbage-collected, and performs the GC. This entails a marking of all blobs and manifests that exist in the backing storage, but not in the database; and a sweeping of all items that were marked in the previous pass and which are still not entered in the database.

This staged mark-and-sweep ensures that we don't remove fresh blobs and manifests that were just pushed, but where the entry in the database is still being created.

The storage of each account is sweeped at most once every 6 hours.