regclient

package module
v0.4.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 26, 2023 License: Apache-2.0 Imports: 31 Imported by: 16

README

regclient

GitHub Workflow Status GitHub Workflow Status GitHub Go Reference

Client interface for the registry API. This includes regctl for a command line interface to manage registries.

regctl demo

regclient Features

  • Provides a client interface to interacting with registries.
  • Images may be inspected without pulling the layers, allowing quick access to the image manifest and configuration.
  • Tags may be listed for a repository.
  • Repositories may be listed from a registry (if supported).
  • Copying an image only pulls layers when needed, allowing images to be quickly retagged or promoted across repositories.
  • Multi-platform images are supported, allowing all platforms to be copied between registries.
  • Digest tags used by projects like sigstore/cosign are supported, allowing signature, attestation, and SBOM metadata to be copied with the image.
  • Digests may be queried for a tag without pulling the manifest.
  • Rate limits may be queried from the registry without pulling an image (useful for Docker Hub).
  • Images may be imported and exported to both OCI and Docker formatted tar files.
  • OCI Layout is supported for copying images to and from a local directory.
  • Delete APIs have been provided for tags, manifests, and blobs (the tag deletion will only delete a single tag even if multiple tags point to the same digest).
  • Registry logins are imported from docker when available
  • Self signed, insecure, and http-only registries are all supported.
  • Requests will retry and fall back to chunked uploads when network issues are encountered.

regctl Features

regctl is a CLI interface to the regclient library. In addition to the features listed for regclient, regctl adds the following abilities:

  • Formatting output with templates.
  • Push and pull arbitrary artifacts.

regsync features

regsync is an image mirroring tool. It will copy images between two locations with the following additional features:

  • Uses a yaml configuration.
  • The regclient copy is used to only pull needed layers, supporting multi-platform, and additional metadata.
  • Can use user's docker configuration for registry credentials.
  • Ability to run on a cron schedule, one time synchronization, or only check for stale images.
  • Ability to backup previous target image before overwriting.
  • Ability to postpone mirror step when rate limit is below a threshold.
  • Ability to mirror multiple images concurrently.

regbot features

regbot is a scripting tool on top of the regclient API with the following features:

  • Runs user provided scripts based on a yaml configuration.
  • Scripts are written in Lua and executed directly in Go.
  • Can run on a cron schedule or a one time execution.
  • Dry-run option can be used for testing.
  • Built-in functions include:
    • Repository list
    • Tag list
    • Image manifest (either head or get, and optional resolving multi-platform reference)
    • Image config (this includes the creation time, labels, and other details shown in a docker image inspect)
    • Image ratelimit and a wait function to delay the script when ratelimit remaining is below a threshold
    • Image copy
    • Manifest delete
    • Tag delete

Development Status

This project is in active development. Various Go APIs may change, but efforts will be made to provide aliases and stubs for any removed API.

Building

git clone https://github.com/regclient/regclient.git
cd regclient
make

Downloading Binaries

Binaries are available on the releases page.

The latest release can be downloaded using curl (adjust "regctl" and "linux-amd64" for the desired command and your own platform):

curl -L https://github.com/regclient/regclient/releases/latest/download/regctl-linux-amd64 >regctl
chmod 755 regctl

Merges into the main branch also have binaries available as artifacts within GitHub Actions

Running as a Container

You can run regctl, regsync, and regbot in a container.

For regctl (include a -t for any commands that require a tty, e.g. registry login):

docker container run -i --rm --net host \
  -v regctl-conf:/home/appuser/.regctl/ \
  regclient/regctl:latest --help

For regsync:

docker container run -i --rm --net host \
  -v "$(pwd)/regsync.yml:/home/appuser/regsync.yml" \
  regclient/regsync:latest -c /home/appuser/regsync.yml check

For regbot:

docker container run -i --rm --net host \
  -v "$(pwd)/regbot.yml:/home/appuser/regbot.yml" \
  regclient/regbot:latest -c /home/appuser/regbot.yml once --dry-run

Or on Linux and Mac environments, you can run regctl as your own user and save configuration settings, use docker credentials, and use any docker certs:

docker container run -i --rm --net host \
  -u "$(id -u):$(id -g)" -e HOME -v $HOME:$HOME \
  -v /etc/docker/certs.d:/etc/docker/certs.d:ro \
  regclient/regctl:latest --help

And regctl can be packaged as a shell script with:

cat >regctl <<EOF
#!/bin/sh
opts=""
case "\$*" in
  "registry login"*) opts="-t";;
esac
docker container run \$opts -i --rm --net host \\
  -u "\$(id -u):\$(id -g)" -e HOME -v \$HOME:\$HOME \\
  -v /etc/docker/certs.d:/etc/docker/certs.d:ro \\
  regclient/regctl:latest "\$@"
EOF
chmod 755 regctl
./regctl --help

Images are also included with an alpine base, which are useful for CI pipelines that expect the container to include a /bin/sh. These alpine based images also include the ecr-login and gcr credential helpers.

Installing as a Docker CLI Plugin

To install regctl as a docker CLI plugin:

make plugin-user # install for the current user
make plugin-host # install for all users on the host (requires sudo)

Once installed as a plugin, you can access it from the docker CLI:

$ docker regctl --help
Utility for accessing docker registries
More details at https://github.com/regclient/regclient

Usage:
  regctl <cmd> [flags]
  regctl [command]

Available Commands:
  help        Help about any command
  image       manage images
  layer       manage image layers/blobs
  registry    manage registries
  tag         manage tags
...

Usage

See the project documentation.

Documentation

Overview

Package regclient is used to access OCI registries

Index

Constants

View Source 
const (
	// DefaultUserAgent sets the header on http requests
	DefaultUserAgent = "regclient/regclient"
	// DockerCertDir default location for docker certs
	DockerCertDir = "/etc/docker/certs.d"
	// DockerRegistry is the well known name of Docker Hub, "docker.io"
	DockerRegistry = config.DockerRegistry
	// DockerRegistryAuth is the name of Docker Hub seen in docker's config.json
	DockerRegistryAuth = config.DockerRegistryAuth
	// DockerRegistryDNS is the actual registry DNS name for Docker Hub
	DockerRegistryDNS = config.DockerRegistryDNS
)

Variables

This section is empty.

Functions

This section is empty.

Types

type ImageOpts 

type ImageOpts func(*imageOpt)

ImageOpts define options for the Image* commands

func ImageWithCheckBaseDigest added in v0.4.5 

func ImageWithCheckBaseDigest(d string) ImageOpts

ImageWithCheckBaseDigest provides a base digest to compare

func ImageWithCheckBaseRef added in v0.4.5 

func ImageWithCheckBaseRef(r string) ImageOpts

ImageWithCheckBaseRef provides a base reference to compare

func ImageWithCheckSkipConfig added in v0.4.5 

func ImageWithCheckSkipConfig() ImageOpts

ImageWithCheckSkipConfig skips the configuration check

func ImageWithChild added in v0.4.4 

func ImageWithChild() ImageOpts

ImageWithChild attempts to copy every manifest and blob even if parent manifests already exist.

func ImageWithDigestTags 

func ImageWithDigestTags() ImageOpts

ImageWithDigestTags looks for "sha-<digest>.*" tags in the repo to copy with any manifest. These are used by some artifact systems like sigstore/cosign.

func ImageWithForceRecursive 

func ImageWithForceRecursive() ImageOpts

ImageWithForceRecursive attempts to copy every manifest and blob even if parent manifests already exist.

func ImageWithIncludeExternal added in v0.4.3 

func ImageWithIncludeExternal() ImageOpts

ImageWithIncludeExternal attempts to copy every manifest and blob even if parent manifests already exist.

func ImageWithPlatform added in v0.4.5 

func ImageWithPlatform(p string) ImageOpts

ImageWithPlatform requests specific platforms from a manifest list. This is used by ImageCheckBase.

func ImageWithPlatforms 

func ImageWithPlatforms(p []string) ImageOpts

ImageWithPlatforms only copies specific platforms from a manifest list. This will result in a failure on many registries that validate manifests. Use the empty string to indicate images without a platform definition should be copied.

func ImageWithReferrers added in v0.4.3 

func ImageWithReferrers(rOpts ...scheme.ReferrerOpts) ImageOpts

ImageWithReferrers recursively includes images that refer to this.

type ManifestOpts added in v0.4.1 

type ManifestOpts func(*manifestOpt)

ManifestOpts define options for the Manifest* commands

func WithManifest added in v0.4.5 

func WithManifest(m manifest.Manifest) ManifestOpts

WithManifest passes a manifest to ManifestDelete.

func WithManifestCheckReferrers added in v0.4.5 

func WithManifestCheckReferrers() ManifestOpts

WithManifestCheckReferrers checks for referrers field on ManifestDelete.

func WithManifestChild added in v0.4.5 

func WithManifestChild() ManifestOpts

WithManifestChild for ManifestPut.

func WithManifestDesc added in v0.4.5 

func WithManifestDesc(d types.Descriptor) ManifestOpts

WithManifestDesc includes the descriptor for ManifestGet. This is used to automatically extract a Data field if available.

func WithManifestRequireDigest added in v0.4.6 

func WithManifestRequireDigest() ManifestOpts

WithManifestRequireDigest falls back from a HEAD to a GET request when digest headers aren't received.

type Opt 

type Opt func(*RegClient)

Opt functions are used to configure NewRegClient

func WithBlobSize 

func WithBlobSize(chunk, max int64) Opt

WithBlobSize overrides default blob sizes

func WithCertDir 

func WithCertDir(path ...string) Opt

WithCertDir adds a path of certificates to trust similar to Docker's /etc/docker/certs.d

func WithConfigHost 

func WithConfigHost(configHost config.Host) Opt

WithConfigHost adds config host settings

func WithConfigHosts 

func WithConfigHosts(configHosts []config.Host) Opt

WithConfigHosts adds a list of config host settings

func WithDockerCerts 

func WithDockerCerts() Opt

WithDockerCerts adds certificates trusted by docker in /etc/docker/certs.d

func WithDockerCreds 

func WithDockerCreds() Opt

WithDockerCreds adds configuration from users docker config with registry logins This changes the default value from the config file, and should be added after the config file is loaded

func WithFS 

func WithFS(fs rwfs.RWFS) Opt

WithFS overrides the backing filesystem (used by ocidir)

func WithLog 

func WithLog(log *logrus.Logger) Opt

WithLog overrides default logrus Logger

func WithRetryDelay 

func WithRetryDelay(delayInit, delayMax time.Duration) Opt

WithRetryDelay specifies the time permitted for retry delays

func WithRetryLimit 

func WithRetryLimit(retryLimit int) Opt

WithRetryLimit specifies the number of retries for non-fatal errors

func WithUserAgent 

func WithUserAgent(ua string) Opt

WithUserAgent specifies the User-Agent http header

type RegClient 

type RegClient struct {
	// contains filtered or unexported fields
}

RegClient is used to access OCI distribution-spec registries

func New 

func New(opts ...Opt) *RegClient

New returns a registry client

func (*RegClient) BlobCopy 

func (rc *RegClient) BlobCopy(ctx context.Context, refSrc ref.Ref, refTgt ref.Ref, d types.Descriptor) error

BlobCopy copies a blob between two locations If the blob already exists in the target, the copy is skipped A server side cross repository blob mount is attempted

func (*RegClient) BlobDelete 

func (rc *RegClient) BlobDelete(ctx context.Context, r ref.Ref, d types.Descriptor) error

BlobDelete removes a blob from the registry This method should only be used to repair a damaged registry Typically a server side garbage collection should be used to purge unused blobs

func (*RegClient) BlobGet 

func (rc *RegClient) BlobGet(ctx context.Context, r ref.Ref, d types.Descriptor) (blob.Reader, error)

BlobGet retrieves a blob, returning a reader

func (*RegClient) BlobGetOCIConfig 

func (rc *RegClient) BlobGetOCIConfig(ctx context.Context, ref ref.Ref, d types.Descriptor) (blob.OCIConfig, error)

BlobGetOCIConfig retrieves an OCI config from a blob, automatically extracting the JSON

func (*RegClient) BlobHead 

func (rc *RegClient) BlobHead(ctx context.Context, r ref.Ref, d types.Descriptor) (blob.Reader, error)

BlobHead is used to verify if a blob exists and is accessible

func (*RegClient) BlobMount 

func (rc *RegClient) BlobMount(ctx context.Context, refSrc ref.Ref, refTgt ref.Ref, d types.Descriptor) error

BlobMount attempts to perform a server side copy/mount of the blob between repositories

func (*RegClient) BlobPut 

func (rc *RegClient) BlobPut(ctx context.Context, ref ref.Ref, d types.Descriptor, rdr io.Reader) (types.Descriptor, error)

BlobPut uploads a blob to a repository. This will attempt an anonymous blob mount first which some registries may support. It will then try doing a full put of the blob without chunking (most widely supported). If the full put fails, it will fall back to a chunked upload (useful for flaky networks).

func (*RegClient) Close 

func (rc *RegClient) Close(ctx context.Context, r ref.Ref) error

Close is used to free resources associated with a reference With ocidir, this may trigger a garbage collection process

func (*RegClient) ImageCheckBase added in v0.4.5 

func (rc *RegClient) ImageCheckBase(ctx context.Context, r ref.Ref, opts ...ImageOpts) error

ImageCheckBase returns nil if the base image is unchanged. A base image mismatch returns an error that wraps types.ErrMismatch.

func (*RegClient) ImageCopy 

func (rc *RegClient) ImageCopy(ctx context.Context, refSrc ref.Ref, refTgt ref.Ref, opts ...ImageOpts) error

ImageCopy copies an image This will retag an image in the same repository, only pushing and pulling the top level manifest On the same registry, it will attempt to use cross-repository blob mounts to avoid pulling blobs Blobs are only pulled when they don't exist on the target and a blob mount fails

func (*RegClient) ImageExport 

func (rc *RegClient) ImageExport(ctx context.Context, r ref.Ref, outStream io.Writer) error

ImageExport exports an image to an output stream. The format is compatible with "docker load" if a single image is selected and not a manifest list. The ref must include a tag for exporting to docker (defaults to latest), and may also include a digest. The export is also formatted according to OCI layout which supports multi-platform images. <https://github.com/opencontainers/image-spec/blob/master/image-layout.md> A tar file will be sent to outStream.

Resulting filesystem: oci-layout: created at top level, can be done at the start index.json: created at top level, single descriptor with org.opencontainers.image.ref.name annotation pointing to the tag manifest.json: created at top level, based on every layer added, only works for a single arch image blobs/$algo/$hash: each content addressable object (manifest, config, or layer), created recursively

func (*RegClient) ImageImport 

func (rc *RegClient) ImageImport(ctx context.Context, ref ref.Ref, rs io.ReadSeeker) error

ImageImport pushes an image from a tar file to a registry

func (*RegClient) ManifestDelete 

func (rc *RegClient) ManifestDelete(ctx context.Context, r ref.Ref, opts ...ManifestOpts) error

ManifestDelete removes a manifest, including all tags pointing to that registry The reference must include the digest to delete (see TagDelete for deleting a tag) All tags pointing to the manifest will be deleted

func (*RegClient) ManifestGet 

func (rc *RegClient) ManifestGet(ctx context.Context, r ref.Ref, opts ...ManifestOpts) (manifest.Manifest, error)

ManifestGet retrieves a manifest

func (*RegClient) ManifestHead 

func (rc *RegClient) ManifestHead(ctx context.Context, r ref.Ref, opts ...ManifestOpts) (manifest.Manifest, error)

ManifestHead queries for the existence of a manifest and returns metadata (digest, media-type, size)

func (*RegClient) ManifestPut 

func (rc *RegClient) ManifestPut(ctx context.Context, r ref.Ref, m manifest.Manifest, opts ...ManifestOpts) error

ManifestPut pushes a manifest Any descriptors referenced by the manifest typically need to be pushed first

func (*RegClient) ReferrerList added in v0.4.3 

func (rc *RegClient) ReferrerList(ctx context.Context, r ref.Ref, opts ...scheme.ReferrerOpts) (referrer.ReferrerList, error)

ReferrerList retrieves a manifest

func (*RegClient) RepoList 

func (rc *RegClient) RepoList(ctx context.Context, hostname string, opts ...scheme.RepoOpts) (*repo.RepoList, error)

RepoList returns a list of repositories on a registry Note the underlying "_catalog" API is not supported on many cloud registries

func (*RegClient) TagDelete 

func (rc *RegClient) TagDelete(ctx context.Context, r ref.Ref) error

TagDelete deletes a tag from the registry. Since there's no API for this, you'd want to normally just delete the manifest. However multiple tags may point to the same manifest, so instead you must: 1. Make a manifest, for this we put a few labels and timestamps to be unique. 2. Push that manifest to the tag. 3. Delete the digest for that new manifest that is only used by that tag.

func (*RegClient) TagList 

func (rc *RegClient) TagList(ctx context.Context, r ref.Ref, opts ...scheme.TagOpts) (*tag.List, error)

TagList returns a tag list from a repository

Source Files

View all Source files

Directories

Path Synopsis
cmd
regbot
regbot/internal/go2lua
Package go2lua converts between go and Lua data structures
 Package go2lua converts between go and Lua data structures
regbot/sandbox
Package sandbox defines the Lua sandbox used to run user scripts
 Package sandbox defines the Lua sandbox used to run user scripts
regctl
regsync
Package config is used for all regclient configuration settings
 Package config is used for all regclient configuration settings
internal
auth
Package auth is used for HTTP authentication
 Package auth is used for HTTP authentication
conffile
Package conffile wraps the read and write of configuration files
 Package conffile wraps the read and write of configuration files
diff
Package diff computes the efficient set of changes (insert/delete) between two arrays of strings
 Package diff computes the efficient set of changes (insert/delete) between two arrays of strings
httplink
Package httplink parses the Link header from HTTP responses according to RFC5988
 Package httplink parses the Link header from HTTP responses according to RFC5988
reghttp
Package reghttp is used for HTTP requests to a registry
 Package reghttp is used for HTTP requests to a registry
reqresp
Package reqresp is used to create mock web servers for testing
 Package reqresp is used to create mock web servers for testing
retryable
Package retryable is a legacy package, functionality has been moved to reghttp
 Package retryable is a legacy package, functionality has been moved to reghttp
rwfs
Package rwfs implements a read-write filesystem, extending fs.FS
 Package rwfs implements a read-write filesystem, extending fs.FS
timejson
Package timejson extends time methods with marshal/unmarshal for json
 Package timejson extends time methods with marshal/unmarshal for json
units
Package units is copied from https://github.com/docker/go-units
 Package units is copied from https://github.com/docker/go-units
version
Package version returns details on the Go and Git repo used in the build
 Package version returns details on the Go and Git repo used in the build
wraperr
Package wraperr packages an error with another wrapped error This allows errors.Is to work without directly injecting the string of the wrapped error
 Package wraperr packages an error with another wrapped error This allows errors.Is to work without directly injecting the string of the wrapped error
Package mod changes an image according to the requested modifications.
 Package mod changes an image according to the requested modifications.
pkg
archive
Package archive is used to read and write tar files
 Package archive is used to read and write tar files
template
Package template wraps a common set of templates around text/template
 Package template wraps a common set of templates around text/template
Package regclient is a legacy package, this has been moved to the top level regclient package
 Package regclient is a legacy package, this has been moved to the top level regclient package
blob
Package blob is a legacy package, this has been moved to the types/blob package
 Package blob is a legacy package, this has been moved to the types/blob package
config
Package config is a legacy package, this has been moved to the config package
 Package config is a legacy package, this has been moved to the config package
manifest
Package manifest is a legacy package, this has been moved to the types/manifest package
 Package manifest is a legacy package, this has been moved to the types/manifest package
types
Package types is a legacy package, using the top level types package is recommended
 Package types is a legacy package, using the top level types package is recommended
Package scheme defines the interface for various reference schemes
 Package scheme defines the interface for various reference schemes
ocidir
Package ocidir implements the OCI Image Layout scheme with a directory (not packed in a tar)
 Package ocidir implements the OCI Image Layout scheme with a directory (not packed in a tar)
reg
Package reg implements the OCI registry scheme used by most images (host:port/repo:tag)
 Package reg implements the OCI registry scheme used by most images (host:port/repo:tag)
Package types defines various types that have no other internal imports This allows them to be used between other packages without creating import loops
 Package types defines various types that have no other internal imports This allows them to be used between other packages without creating import loops
blob
Package blob is the underlying type for pushing and pulling blobs
 Package blob is the underlying type for pushing and pulling blobs
docker
Package docker defines the common types for all docker schemas
 Package docker defines the common types for all docker schemas
docker/schema1
Package schema1 defines the manifest and json marshal/unmarshal for docker schema1
 Package schema1 defines the manifest and json marshal/unmarshal for docker schema1
docker/schema2
Package schema2 contains structs for Docker schema v2 manifests.
 Package schema2 contains structs for Docker schema v2 manifests.
manifest
Package manifest abstracts the various types of supported manifests.
 Package manifest abstracts the various types of supported manifests.
oci
Package oci defiles OCI image-spec types
 Package oci defiles OCI image-spec types
oci/v1
Package v1 defiles version 1 of OCI image-spec types
 Package v1 defiles version 1 of OCI image-spec types
platform
Package platform handles the parsing and comparing of the image platform (e.g.
 Package platform handles the parsing and comparing of the image platform (e.g.
ref
Package ref is used to define references References default to remote registry references (registry:port/repo:tag) Schemes can be included in front of the reference for different reference types
 Package ref is used to define references References default to remote registry references (registry:port/repo:tag) Schemes can be included in front of the reference for different reference types
referrer
Package referrer is used for responses to the referrers to a manifest
 Package referrer is used for responses to the referrers to a manifest
repo
Package repo handles a list of repositories from a registry
 Package repo handles a list of repositories from a registry
tag
Package tag is used for wrapping tag lists
 Package tag is used for wrapping tag lists

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.