prune

Documentation

Overview

Package prune contains functions that allow you to manipulate data on the storage.

OVERVIEW

Since the registry server stores metadata in the etcd database, there are situations of the desynchronization of the database and the registry storage.

1. If the data is removed from the etcd database, but not removed from the storage, this leads to a waste of space in the storage. At the moment, the hard pruning can help with this problem if you do not need the blobs anymore.

2. If the data is removed from the registry storage, but not removed from etcd database, this leads us to broken cluster. The images that are managed by registry can not be used. In some of these situations the best thing we can do is provide a tool which reports on what images are broken so the user can attempt to recover their system. In other cases, we can restore the data in the etcd database based on data from the storage.

HARD PRUNE

This mode allows you to delete blobs that are no longer referenced in the etcd database (garbage) from storage and reduce the used space on the storage.

RECOVERY

This mode is opposite to the HARD PRUNE. In this mode, we try to restore metadata from the data on the storage. There are two modes of this: check and check+recover.

This mode covers the following cases:

1. Show broken imagestream tags that refer to non-existent images on the storage. In this case, you can either delete such a tag, or re-push the image pointed to by this tag.

2. Show broken images that refer to non-existent blobs on the storage. In this case, you can re-push any imagestream tag that point to this image. Since the images are global, it's enough to re-push only one image to fix all the imagestream tags that refer to it.

3. Show and restore either entire imagestreams or individual imagestream tags inside it that exists on the storage, but removed from etcd database. Tag names will look like: lost-found-<IMAGE-DIGEST> because the name of the tag is not stored on the storage and can't be restored. You can make a new tag with a different name.

Note: labels and anotations that were assigned on the image or imagestream will not be restored because they were stored only in the etcd database, but not on the storage.

Index

• type DryRunPruner
  • func (p *DryRunPruner) DeleteBlob(ctx context.Context, dgst digest.Digest) error
  • func (p *DryRunPruner) DeleteManifestLink(ctx context.Context, svc distribution.ManifestService, reponame string, ...) error
  • func (p *DryRunPruner) DeleteRepository(ctx context.Context, reponame string) error
• type DryRunRestore
  • func (r *DryRunRestore) BrokenImage(image imageapiv1.Image, err error) error
  • func (r *DryRunRestore) BrokenImageStreamTag(is imageapiv1.ImageStream, event imageapiv1.NamedTagEventList, pos int, ...) error
  • func (r *DryRunRestore) ImageStreamTag(imageStream imagestream.ImageStream, image *imageapiv1.Image, tagName string) error
• type ErrImage
  • func (err ErrImage) Error() string
• type Fsck
  • func (r *Fsck) Database(namespace string) error
  • func (r *Fsck) Storage(namespace string) error
• type Pruner
• type RegistryPruner
  • func (p *RegistryPruner) DeleteBlob(ctx context.Context, dgst digest.Digest) error
  • func (p *RegistryPruner) DeleteManifestLink(ctx context.Context, svc distribution.ManifestService, reponame string, ...) error
  • func (p *RegistryPruner) DeleteRepository(ctx context.Context, reponame string) error
• type Restore
• type StorageRestore
  • func (r *StorageRestore) ImageStreamTag(imageStream imagestream.ImageStream, image *imageapiv1.Image, tagName string) error
• type Summary
  • func Prune(ctx context.Context, registry distribution.Namespace, ...) (Summary, error)

Types

type DryRunPruner

type DryRunPruner struct{}

DryRunPruner prints information about each object that going to remove.

func (*DryRunPruner) DeleteBlob

func (p *DryRunPruner) DeleteBlob(ctx context.Context, dgst digest.Digest) error

func (*DryRunPruner) DeleteManifestLink

func (p *DryRunPruner) DeleteManifestLink(ctx context.Context, svc distribution.ManifestService, reponame string, dgst digest.Digest) error

func (*DryRunPruner) DeleteRepository

func (p *DryRunPruner) DeleteRepository(ctx context.Context, reponame string) error

type DryRunRestore

type DryRunRestore struct{}

DryRunRestore prints information about each object

func (*DryRunRestore) BrokenImage

func (r *DryRunRestore) BrokenImage(image imageapiv1.Image, err error) error

func (*DryRunRestore) BrokenImageStreamTag

func (r *DryRunRestore) BrokenImageStreamTag(is imageapiv1.ImageStream, event imageapiv1.NamedTagEventList, pos int, err error) error

BrokenImageStreamTag prints information about broken imagestream tag

func (*DryRunRestore) ImageStreamTag

func (r *DryRunRestore) ImageStreamTag(imageStream imagestream.ImageStream, image *imageapiv1.Image, tagName string) error

ImageStreamTag prints information about imagestream tag which could be restored

type ErrImage

type ErrImage struct {
	Digest  digest.Digest
	Problem error
}

ErrImage defines the error associated with the digest

func (ErrImage) Error

func (err ErrImage) Error() string

Error implements the error interface

type Fsck

type Fsck struct {
	Ctx        context.Context
	Client     client.Interface
	Registry   distribution.Namespace
	ServerAddr string
	Restore    Restore
}

Fsck validates or recovers database based on storage

func (*Fsck) Database

func (r *Fsck) Database(namespace string) error

Database checks metadata in the database

func (*Fsck) Storage

func (r *Fsck) Storage(namespace string) error

Storage restores metadata based on the storage

type Pruner

type Pruner interface {
	DeleteRepository(ctx context.Context, reponame string) error
	DeleteManifestLink(ctx context.Context, svc distribution.ManifestService, reponame string, dgst digest.Digest) error
	DeleteBlob(ctx context.Context, dgst digest.Digest) error
}

Pruner defines a common set of operations for pruning

type RegistryPruner

type RegistryPruner struct {
	StorageDriver driver.StorageDriver
}

RegistryPruner deletes objects.

func (*RegistryPruner) DeleteBlob

func (p *RegistryPruner) DeleteBlob(ctx context.Context, dgst digest.Digest) error

DeleteBlob removes a blob from the storage

func (*RegistryPruner) DeleteManifestLink

func (p *RegistryPruner) DeleteManifestLink(ctx context.Context, svc distribution.ManifestService, reponame string, dgst digest.Digest) error

DeleteManifestLink removes a manifest link from the storage

func (*RegistryPruner) DeleteRepository

func (p *RegistryPruner) DeleteRepository(ctx context.Context, reponame string) error

DeleteRepository removes a repository directory from the storage

type Restore

type Restore interface {
	BrokenImage(image imageapiv1.Image, err error) error
	BrokenImageStreamTag(is imageapiv1.ImageStream, tag imageapiv1.NamedTagEventList, pos int, err error) error
	ImageStreamTag(imageStream imagestream.ImageStream, image *imageapiv1.Image, tagName string) error
}

Restore defines a common set of operations for database and storage validation

type StorageRestore

type StorageRestore struct {
	DryRunRestore

	Ctx    context.Context
	Client client.Interface
}

func (*StorageRestore) ImageStreamTag

func (r *StorageRestore) ImageStreamTag(imageStream imagestream.ImageStream, image *imageapiv1.Image, tagName string) error

type Summary

type Summary struct {
	Blobs     int
	DiskSpace int64
}

Summary is cumulative information about what was pruned.

func Prune

func Prune(ctx context.Context, registry distribution.Namespace, registryClient client.RegistryClient, pruner Pruner) (Summary, error)

Prune removes blobs which are not used by Images in OpenShift.

On error, the Summary will contain what was deleted so far.

TODO(dmage): remove layer links to a blob if the blob is removed or it doesn't belong to the ImageStream. TODO(dmage): keep young blobs (docker/distribution#2297).