imo

README

Image Overlay Kit

The imo library provides a way to handle incremental image updates in container registries. It allows users to pull the difference between two image versions and push this differential update to a registry.

This can significantly reduce the amount of data transferred, especially useful in air-gapped (disconnected) environments.

Key Types and Functions

Incremental

The Incremental type offers methods to retrieve (via Pull) or transmit (via Push) the differences between two images. These differences are determined based on the base and final images. It is crucial to ensure that the destination registry, when pushing these differences, possesses all the layers not encompassed in the incremental difference." You can use PushVet() for checking.

Methods

• PushVet: Verifies if all layers not included in the incremental difference exist in the destination registry. It returns an error if any layer is missing.
• Push: Pushes the incremental difference stored in an OCI-archive tarball to the destination registry. It fails if the remote registry lacks any layers not included in the incremental difference.
• Pull: Pulls the incremental difference between two images. It returns an io.ReadCloser from which an OCI-archive tarball can be read. The caller is responsible for closing the reader.
• New: Returns a new Incremental object. With this object, callers can calculate or send the incremental difference between two images.

Usage

Pulling Image Differences

The imo library allows you to pull the difference between two container images as a tarball. Here's how you can do it:

package main

import (
	"context"
	"io"
	"os"

	"github.com/ricardomaraschini/imo"
)

func pull() {
	// create a new incremental puller
	inc := imo.New(
		imo.WithReporterWriter(os.Stdout),
		imo.WithBaseAuth("user", "pass"),
		imo.WithFinalAuth("user2", "pass2"),
	)

	// pull the differential update
	diff, err := inc.Pull(
		context.Background(),
		"docker.io/myaccount/myapp:v1.0.0",
		"docker.io/myaccount/myapp:v2.0.0",
	)
	if err != nil {
		panic(err)
	}
	defer diff.Close()

	// save the differential update to a file
	fp, err := os.Create("difference.tar")
	if err != nil {
		panic(err)
	}
	defer fp.Close()

	if _, err := io.Copy(fp, diff); err != nil {
		panic(err)
	}
}

Pushing Image Differences

You can also use imo to push a differential update to a container registry:

package main

import (
	"context"
	"os"

	"github.com/ricardomaraschini/imo"
)

func push() {
	// create a new incremental pusher
	inc := imo.New(
		imo.WithReporterWriter(os.Stdout),
		imo.WithPushAuth("user", "pass"),
	)

	// check if the remote registry has all needed layers.
	if err := imo.PushVet(
		context.Background(),
		"difference.tar",
		"myregistry.io/myaccount/app:v1.0.0",
	); err != nil {
		panic(err)
	}

	// push the differential update to the registry
	if err := inc.Push(
		context.Background(),
		"difference.tar",
		"myregistry.io/myaccount/app:v2.0.0",,
	); err != nil {
		panic(err)
	}
}

Documentation

Index

• type Authentications
• type Incremental
  • func New(opts ...Option) *Incremental
  • func (inc *Incremental) Pull(ctx context.Context, base, final string) (io.ReadCloser, error)
  • func (inc *Incremental) Push(ctx context.Context, src, dst string) error
  • func (inc *Incremental) PushVet(ctx context.Context, src, dst string) error
• type ManifestsIndex
  • func NewManifestsIndex(sysctx *types.SystemContext) *ManifestsIndex
  • func (m *ManifestsIndex) FetchManifests(ctx context.Context, from types.ImageReference) error
  • func (m *ManifestsIndex) HasLayer(dgst digest.Digest) bool
  • func (m *ManifestsIndex) Manifests() []manifest.Manifest
• type Option
  • func WithAllArchitectures() Option
  • func WithBaseAuth(user, pass string) Option
  • func WithFinalAuth(user, pass string) Option
  • func WithPushAuth(user, pass string) Option
  • func WithReporterWriter(reporter io.Writer) Option
  • func WithTempDir(dir string) Option
• type RemoveOnClose
  • func (r RemoveOnClose) Close() error
• type Writer
  • func NewWriter(ctx context.Context, from types.ImageReference, to types.ImageReference, ...) (*Writer, error)
  • func NewWriterFromScratch(ctx context.Context, to types.ImageReference, sysctx *types.SystemContext) (*Writer, error)
  • func (i *Writer) NewImageDestination(ctx context.Context, sys *types.SystemContext) (types.ImageDestination, error)

Types

type Authentications

type Authentications struct {
	BaseAuth  *types.DockerAuthConfig
	FinalAuth *types.DockerAuthConfig
	PushAuth  *types.DockerAuthConfig
}

Authentications holds the all the necessary authentications for the incremental operations. BaseAuth is the authentication for the base image, FinalAuth is the authentication for the final image and PushAuth is the authentication for the destination registry. For example, let's suppose we want to get an incremental difference between an imaged hosted on X registry and an image hosted on Y registry and late on we want to push the difference to Z registry. In this case: - BaseAuth is the authentication for the X registry. - FinalAuth is the authentication for the Y registry. - PushAuth is the authentication for the Z registry.

type Incremental

type Incremental struct {
	// contains filtered or unexported fields
}

Incremental provides tooling about getting (Pull) or sending (Push) the difference between two images. The difference is calculated between the base and final images. When Pushing the difference to a destination registry it is important to note that the other layers (the ones not included in the 'difference') exist.

func New

func New(opts ...Option) *Incremental

New returns a new Incremental object. With Incremental objects callers can calculate the incremental difference between two images (Pull) or send the incremental towards a destination (Push).

func (*Incremental) Pull

func (inc *Incremental) Pull(ctx context.Context, base, final string) (io.ReadCloser, error)

Pull pulls the incremental difference between two images. Returns an ReaderCloser from where can be read as an oci-archive tarball. The caller is responsible for closing the reader. If 'base' is equal to 'scratch' then we do not compare the layers of the final image with the layers of the base image. In this case, the returned tarball contains all the layers of the final image.

func (*Incremental) Push

func (inc *Incremental) Push(ctx context.Context, src, dst string) error

Push pushes the incremental difference stored in the oci-archive tarball pointed by src to the destination registry pointed by to. Be aware that if the remote registry does not contain one or more of the layers not included in the incremental difference the push will fail.

func (*Incremental) PushVet

func (inc *Incremental) PushVet(ctx context.Context, src, dst string) error

PushVet verifies if all the layers not included in the incremental difference exist in the destination registry. If not, it returns an error.

type ManifestsIndex

type ManifestsIndex struct {
	// contains filtered or unexported fields
}

ManifestsIndex is an entity that indexes multiple manifests that are part of the same image. Provide tooling around the manifests.

func NewManifestsIndex

func NewManifestsIndex(sysctx *types.SystemContext) *ManifestsIndex

NewManifestsIndex creates a new ManifestsIndex. A ManifestsIndex is an entity capable of indexing all layers that are present in the manifests of an image. Uses the provided SystemContext to access the remote manifests.

func (*ManifestsIndex) FetchManifests

func (m *ManifestsIndex) FetchManifests(ctx context.Context, from types.ImageReference) error

FetchManifests gets the manifests from the source image and indexes all the layers that are present in the manifests in the internal 'index' map. Users can then call 'HasLayer' to check if a layer is present in the source image.

func (*ManifestsIndex) HasLayer

func (m *ManifestsIndex) HasLayer(dgst digest.Digest) bool

HasLayer returns true if the layer is referred by any of the indexed manifests.

func (*ManifestsIndex) Manifests

func (m *ManifestsIndex) Manifests() []manifest.Manifest

Manifests returns the manifests that were fetched for the image.

type Option

type Option func(*Incremental)

Option is a functional option for the Incremental type.

func WithAllArchitectures

func WithAllArchitectures() Option

WithAllArchitectures sets the selection to include all the architectures.

func WithBaseAuth

func WithBaseAuth(user, pass string) Option

WithBaseAuth sets the authentication for the registry from where we are going to pull the "base" image. If we are comparing images v1 and v2 this is the auth for v1 registry.

func WithFinalAuth

func WithFinalAuth(user, pass string) Option

WithFinalAuth sets the authentication for the registry from where we are going to pull the "latest" image. If we are comparing images v1 and v2 this is the auth for v2 registry.

func WithPushAuth

func WithPushAuth(user, pass string) Option

WithPushAuth sets the authentication for the registry where we are going to push the incremental difference. If we have previously compared images v1 and v2 and we are going to push the difference to v3 this is the auth for v3 registry.

func WithReporterWriter

func WithReporterWriter(reporter io.Writer) Option

WithReportWriter sets the report writer for the Incremental type. By default the report writer is io.Discard.

func WithTempDir

func WithTempDir(dir string) Option

WithTempDir sets the temporary directory where we are going to store the diff while the user decides what to do with it. By default this is os.TempDir().

type RemoveOnClose

type RemoveOnClose struct {
	*os.File
	// contains filtered or unexported fields
}

RemoveOnClose is a wrapper around a file that removes the file when closed.

func (RemoveOnClose) Close

func (r RemoveOnClose) Close() error

Close removes the file and closes the underlying file.

type Writer

type Writer struct {
	types.ImageReference
	// contains filtered or unexported fields
}

Writer provides a tool to copy only the layers that are not already present in a different version of the same image.

func NewWriter

func NewWriter(ctx context.Context, from types.ImageReference, to types.ImageReference, sysctx *types.SystemContext) (*Writer, error)

NewWriter is capable of providing an incremental copy of an image using 'from' as base and storing the result in 'to'.

func NewWriterFromScratch

func NewWriterFromScratch(ctx context.Context, to types.ImageReference, sysctx *types.SystemContext) (*Writer, error)

NewWriterFromScratch uses the "scratch" image as base and stores the result in 'to'. This is useful to create a new image from scratch.

func (*Writer) NewImageDestination

func (i *Writer) NewImageDestination(ctx context.Context, sys *types.SystemContext) (types.ImageDestination, error)

NewImageDestination returns a handler used to write.