remote

package
v0.20.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 18, 2024 License: Apache-2.0 Imports: 29 Imported by: 4

README

remote

GoDoc

The remote package implements a client for accessing a registry, per the OCI distribution spec.

It leans heavily on the lower level transport package, which handles the authentication handshake and structured errors.

Usage

package main

import (
	"github.com/schidstorm/go-containerregistry/pkg/authn"
	"github.com/schidstorm/go-containerregistry/pkg/name"
	"github.com/schidstorm/go-containerregistry/pkg/v1/remote"
)

func main() {
	ref, err := name.ParseReference("gcr.io/google-containers/pause")
	if err != nil {
		panic(err)
	}

	img, err := remote.Image(ref, remote.WithAuthFromKeychain(authn.DefaultKeychain))
	if err != nil {
		panic(err)
	}

	// do stuff with img
}

Structure

Background

There are a lot of confusingly similar terms that come up when talking about images in registries.

Anatomy of an image

In general...

  • A tag refers to an image manifest.
  • An image manifest references a config file and an orderered list of compressed layers by sha256 digest.
  • A config file references an ordered list of uncompressed layers by sha256 digest and contains runtime configuration.
  • The sha256 digest of the config file is the image id for the image.

For example, an image with two layers would look something like this:

image anatomy

Anatomy of an index

In the normal case, an index is used to represent a multi-platform image. This was the original use case for a manifest list.

image index anatomy

It is possible for an index to reference another index, per the OCI image-spec. In theory, both an image and image index can reference arbitrary things via descriptors, e.g. see the image layout example, which references an application/xml file from an image index.

That could look something like this:

strange image index anatomy

Using a recursive index like this might not be possible with all registries, but this flexibility allows for some interesting applications, e.g. the OCI Artifacts effort.

Anatomy of an image upload

The structure of an image requires a delicate ordering when uploading an image to a registry. Below is a (slightly simplified) figure that describes how an image is prepared for upload to a registry and how the data flows between various artifacts:

upload

Note that:

  • A config file references the uncompressed layer contents by sha256.
  • A manifest references the compressed layer contents by sha256 and the size of the layer.
  • A manifest references the config file contents by sha256 and the size of the file.

It follows that during an upload, we need to upload layers before the config file, and we need to upload the config file before the manifest.

Sometimes, we know all of this information ahead of time, (e.g. when copying from remote.Image), so the ordering is less important.

In other cases, e.g. when using a stream.Layer, we can't compute anything until we have already uploaded the layer, so we need to be careful about ordering.

Caveats

schema 1

This package does not support schema 1 images, see #377, however, it's possible to do something useful with them via remote.Get, which doesn't try to interpret what is returned by the registry.

crane.Copy takes advantage of this to implement support for copying schema 1 images, see here.

Documentation

Overview

Package remote provides facilities for reading/writing v1.Images from/to a remote image registry.

Index

Constants

This section is empty.

Variables

View Source 
var DefaultTransport http.RoundTripper = &http.Transport{
	Proxy: http.ProxyFromEnvironment,
	DialContext: (&net.Dialer{
		Timeout:   30 * time.Second,
		KeepAlive: 30 * time.Second,
	}).DialContext,
	ForceAttemptHTTP2:     true,
	MaxIdleConns:          100,
	IdleConnTimeout:       90 * time.Second,
	TLSHandshakeTimeout:   10 * time.Second,
	ExpectContinueTimeout: 1 * time.Second,

	MaxIdleConnsPerHost: 50,
}

DefaultTransport is based on http.DefaultTransport with modifications documented inline below.

View Source 
var ErrSchema1 = errors.New("see https://github.com/schidstorm/go-containerregistry/issues/377")

ErrSchema1 indicates that we received a schema1 manifest from the registry. This library doesn't have plans to support this legacy image format: https://github.com/schidstorm/go-containerregistry/issues/377

Functions

func Catalog 

func Catalog(ctx context.Context, target name.Registry, options ...Option) ([]string, error)

Catalog calls /_catalog, returning the list of repositories on the registry.

func CatalogPage 

func CatalogPage(target name.Registry, last string, n int, options ...Option) ([]string, error)

CatalogPage calls /_catalog, returning the list of repositories on the registry.

func CheckPushPermission 

func CheckPushPermission(ref name.Reference, kc authn.Keychain, t http.RoundTripper) error

CheckPushPermission returns an error if the given keychain cannot authorize a push operation to the given ref.

This can be useful to check whether the caller has permission to push an image before doing work to construct the image.

TODO(#412): Remove the need for this method.

func Delete 

func Delete(ref name.Reference, options ...Option) error

Delete removes the specified image reference from the remote registry. 

func Head(ref name.Reference, options ...Option) (*v1.Descriptor, error)

Head returns a v1.Descriptor for the given reference by issuing a HEAD request.

Note that the server response will not have a body, so any errors encountered should be retried with Get to get more details.

func Image 

func Image(ref name.Reference, options ...Option) (v1.Image, error)

Image provides access to a remote image reference.

func Index 

func Index(ref name.Reference, options ...Option) (v1.ImageIndex, error)

Index provides access to a remote index reference.

func Layer 

func Layer(ref name.Digest, options ...Option) (v1.Layer, error)

Layer reads the given blob reference from a registry as a Layer. A blob reference here is just a punned name.Digest where the digest portion is the digest of the blob to be read and the repository portion is the repo where that blob lives.

func List 

func List(repo name.Repository, options ...Option) ([]string, error)

List calls /tags/list for the given repository, returning the list of tags in the "tags" property.

func ListWithContext deprecated 

func ListWithContext(ctx context.Context, repo name.Repository, options ...Option) ([]string, error)

ListWithContext calls List with the given context.

Deprecated: Use List and WithContext. This will be removed in a future release.

func MultiWrite 

func MultiWrite(todo map[name.Reference]Taggable, options ...Option) (rerr error)

MultiWrite writes the given Images or ImageIndexes to the given refs, as efficiently as possible, by deduping shared layer blobs while uploading them in parallel.

func Put 

func Put(ref name.Reference, t Taggable, options ...Option) error

Put adds a manifest from the given Taggable via PUT /v1/.../manifest/<ref>

Notable implementations of Taggable are v1.Image, v1.ImageIndex, and remote.Descriptor.

If t implements MediaType, we will use that for the Content-Type, otherwise we will default to types.DockerManifestSchema2.

Put does not attempt to write anything other than the manifest, so callers should ensure that all blobs or manifests that are referenced by t exist in the target registry.

func Referrers 

func Referrers(d name.Digest, options ...Option) (v1.ImageIndex, error)

Referrers returns a list of descriptors that refer to the given manifest digest.

The subject manifest doesn't have to exist in the registry for there to be descriptors that refer to it.

func Tag 

func Tag(tag name.Tag, t Taggable, options ...Option) error

Tag adds a tag to the given Taggable via PUT /v2/.../manifests/<tag>

Notable implementations of Taggable are v1.Image, v1.ImageIndex, and remote.Descriptor.

If t implements MediaType, we will use that for the Content-Type, otherwise we will default to types.DockerManifestSchema2.

Tag does not attempt to write anything other than the manifest, so callers should ensure that all blobs or manifests that are referenced by t exist in the target registry.

func WithNondistributable 

func WithNondistributable(o *options) error

WithNondistributable includes non-distributable (foreign) layers when writing images, see: https://github.com/opencontainers/image-spec/blob/master/layer.md#non-distributable-layers

The default behaviour is to skip these layers

func Write 

func Write(ref name.Reference, img v1.Image, options ...Option) (rerr error)

Write pushes the provided img to the specified image reference.

func WriteIndex 

func WriteIndex(ref name.Reference, ii v1.ImageIndex, options ...Option) (rerr error)

WriteIndex pushes the provided ImageIndex to the specified image reference. WriteIndex will attempt to push all of the referenced manifests before attempting to push the ImageIndex, to retain referential integrity.

func WriteLayer 

func WriteLayer(repo name.Repository, layer v1.Layer, options ...Option) (rerr error)

WriteLayer uploads the provided Layer to the specified repo.

Types

type Backoff 

type Backoff = retry.Backoff

Backoff is an alias of retry.Backoff to expose this configuration option to consumers of this lib

type Catalogger 

type Catalogger struct {
	// contains filtered or unexported fields
}

func (*Catalogger) HasNext 

func (l *Catalogger) HasNext() bool

func (*Catalogger) Next 

func (l *Catalogger) Next(ctx context.Context) (*Catalogs, error)

type Catalogs 

type Catalogs struct {
	Repos []string `json:"repositories"`
	Next  string   `json:"next,omitempty"`
}

type Descriptor 

type Descriptor struct {
	v1.Descriptor

	Manifest []byte
	// contains filtered or unexported fields
}

Descriptor provides access to metadata about remote artifact and accessors for efficiently converting it into a v1.Image or v1.ImageIndex.

func Get 

func Get(ref name.Reference, options ...Option) (*Descriptor, error)

Get returns a remote.Descriptor for the given reference. The response from the registry is left un-interpreted, for the most part. This is useful for querying what kind of artifact a reference represents.

See Head if you don't need the response body.

func (*Descriptor) Image 

func (d *Descriptor) Image() (v1.Image, error)

Image converts the Descriptor into a v1.Image.

If the fetched artifact is already an image, it will just return it.

If the fetched artifact is an index, it will attempt to resolve the index to a child image with the appropriate platform.

See WithPlatform to set the desired platform.

func (*Descriptor) ImageIndex 

func (d *Descriptor) ImageIndex() (v1.ImageIndex, error)

ImageIndex converts the Descriptor into a v1.ImageIndex.

func (*Descriptor) RawManifest 

func (d *Descriptor) RawManifest() ([]byte, error)

RawManifest exists to satisfy the Taggable interface.

func (*Descriptor) Schema1 

func (d *Descriptor) Schema1() (v1.Image, error)

Schema1 converts the Descriptor into a v1.Image for v2 schema 1 media types.

The v1.Image returned by this method does not implement the entire interface because it would be inefficient. This exists mostly to make it easier to copy schema 1 images around or look at their filesystems. This is separate from Image() to avoid a backward incompatible change for callers expecting ErrSchema1.

type Lister 

type Lister struct {
	// contains filtered or unexported fields
}

func (*Lister) HasNext 

func (l *Lister) HasNext() bool

func (*Lister) Next 

func (l *Lister) Next(ctx context.Context) (*Tags, error)

type MountableLayer 

type MountableLayer struct {
	v1.Layer

	Reference name.Reference
}

MountableLayer wraps a v1.Layer in a shim that enables the layer to be "mounted" when published to another registry.

func (*MountableLayer) Descriptor 

func (ml *MountableLayer) Descriptor() (*v1.Descriptor, error)

Descriptor retains the original descriptor from an image manifest. See partial.Descriptor.

func (*MountableLayer) Exists 

func (ml *MountableLayer) Exists() (bool, error)

Exists is a hack. See partial.Exists.

type Option 

type Option func(*options) error

Option is a functional option for remote operations.

func Reuse 

func Reuse[I *Puller | *Pusher](i I) Option

Reuse takes a Puller or Pusher and reuses it for remote interactions rather than starting from a clean slate. For example, it will reuse token exchanges when possible and avoid sending redundant HEAD requests.

Reuse will take precedence over other options passed to most remote functions because most options deal with setting up auth and transports, which Reuse intetionally skips.

func WithAuth 

func WithAuth(auth authn.Authenticator) Option

WithAuth is a functional option for overriding the default authenticator for remote operations. It is an error to use both WithAuth and WithAuthFromKeychain in the same Option set.

The default authenticator is authn.Anonymous.

func WithAuthFromKeychain 

func WithAuthFromKeychain(keys authn.Keychain) Option

WithAuthFromKeychain is a functional option for overriding the default authenticator for remote operations, using an authn.Keychain to find credentials. It is an error to use both WithAuth and WithAuthFromKeychain in the same Option set.

The default authenticator is authn.Anonymous.

func WithContext 

func WithContext(ctx context.Context) Option

WithContext is a functional option for setting the context in http requests performed by a given function. Note that this context is used for _all_ http requests, not just the initial volley. E.g., for remote.Image, the context will be set on http requests generated by subsequent calls to RawConfigFile() and even methods on layers returned by Layers().

The default context is context.Background().

func WithFilter 

func WithFilter(key string, value string) Option

WithFilter sets the filter querystring for HTTP operations.

func WithJobs 

func WithJobs(jobs int) Option

WithJobs is a functional option for setting the parallelism of remote operations performed by a given function. Note that not all remote operations support parallelism.

The default value is 4.

func WithPageSize 

func WithPageSize(size int) Option

WithPageSize sets the given size as the value of parameter 'n' in the request.

To omit the `n` parameter entirely, use WithPageSize(0). The default value is 1000.

func WithPlatform 

func WithPlatform(p v1.Platform) Option

WithPlatform is a functional option for overriding the default platform that Image and Descriptor.Image use for resolving an index to an image.

The default platform is amd64/linux.

func WithProgress 

func WithProgress(updates chan<- v1.Update) Option

WithProgress takes a channel that will receive progress updates as bytes are written.

Sending updates to an unbuffered channel will block writes, so callers should provide a buffered channel to avoid potential deadlocks.

func WithRetryBackoff 

func WithRetryBackoff(backoff Backoff) Option

WithRetryBackoff sets the httpBackoff for retry HTTP operations.

func WithRetryPredicate 

func WithRetryPredicate(predicate retry.Predicate) Option

WithRetryPredicate sets the predicate for retry HTTP operations.

func WithRetryStatusCodes 

func WithRetryStatusCodes(codes ...int) Option

WithRetryStatusCodes sets which http response codes will be retried.

func WithTransport 

func WithTransport(t http.RoundTripper) Option

WithTransport is a functional option for overriding the default transport for remote operations. If transport.Wrapper is provided, this signals that the consumer does *not* want any further wrapping to occur. i.e. logging, retry and useragent

The default transport is DefaultTransport.

func WithUserAgent 

func WithUserAgent(ua string) Option

WithUserAgent adds the given string to the User-Agent header for any HTTP requests. This header will also include "go-containerregistry/${version}".

If you want to completely overwrite the User-Agent header, use WithTransport.

type Puller 

type Puller struct {
	// contains filtered or unexported fields
}

func NewPuller 

func NewPuller(options ...Option) (*Puller, error)

func (*Puller) Catalog 

func (p *Puller) Catalog(ctx context.Context, reg name.Registry) ([]string, error)

Catalog lists repos in a registry and handles pagination, returning the full list of repos.

func (*Puller) Catalogger 

func (p *Puller) Catalogger(ctx context.Context, reg name.Registry) (*Catalogger, error)

Catalogger lists repos in a registry and returns a Catalogger for paginating through the results.

func (*Puller) Get 

func (p *Puller) Get(ctx context.Context, ref name.Reference) (*Descriptor, error)

Get is like remote.Get, but avoids re-authenticating when possible.

func (*Puller) Head 

func (p *Puller) Head(ctx context.Context, ref name.Reference) (*v1.Descriptor, error)

Head is like remote.Head, but avoids re-authenticating when possible.

func (*Puller) Layer 

func (p *Puller) Layer(ctx context.Context, ref name.Digest) (v1.Layer, error)

Layer is like remote.Layer, but avoids re-authenticating when possible.

func (*Puller) List 

func (p *Puller) List(ctx context.Context, repo name.Repository) ([]string, error)

List lists tags in a repo and handles pagination, returning the full list of tags.

func (*Puller) Lister 

func (p *Puller) Lister(ctx context.Context, repo name.Repository) (*Lister, error)

Lister lists tags in a repo and returns a Lister for paginating through the results.

type Pusher 

type Pusher struct {
	// contains filtered or unexported fields
}

func NewPusher 

func NewPusher(options ...Option) (*Pusher, error)

func (*Pusher) Delete 

func (p *Pusher) Delete(ctx context.Context, ref name.Reference) error

func (*Pusher) Push 

func (p *Pusher) Push(ctx context.Context, ref name.Reference, t Taggable) error

func (*Pusher) Upload 

func (p *Pusher) Upload(ctx context.Context, repo name.Repository, l v1.Layer) error

type Taggable 

type Taggable interface {
	RawManifest() ([]byte, error)
}

Taggable is an interface that enables a manifest PUT (e.g. for tagging).

type Tags 

type Tags struct {
	Name string   `json:"name"`
	Tags []string `json:"tags"`
	Next string   `json:"next,omitempty"`
}

Source Files

View all Source files

Directories

Path Synopsis
Package transport provides facilities for setting up an authenticated http.RoundTripper given an Authenticator and base RoundTripper.
 Package transport provides facilities for setting up an authenticated http.RoundTripper given an Authenticator and base RoundTripper.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.