reimage

package module
v0.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 8, 2023 License: Apache-2.0 Imports: 43 Imported by: 0

README

Re-image

A tool for mass update of images for kubernetes manifests.

This is a Work In Progress, YMMV, configuration and settings may change.

  • Works with helm post-renderer, or arbitrary k8s manifests
  • Check images used by Deployments, StatefulSets, DaemonSets, Cronjobs and Job (or arbitrary objects using jsonpath queries):
    • Exist (prevents deploy of manifests with bad references)
    • Remap tags (e.g latest) to a tag for the explicit digest they currently map to
    • Optionally copy images from third party repositories to known repository
    • Check that image scanning:
      • has completed (with time limited check retry)
      • images have no CVEs above a certain score (with overridable ignore list)
  • Can create GCP BinAuthz attestations for discovered image digests

This is intended to:

  • Prevent deploying assets with un-pullable images
  • Localise images for faster start times
  • Potentially improve availability by reducing runtime third party service dependencies (e.g. Dockerhub)
  • Help with compliance by pulling all images from registries with image scanning
  • Help with the use of in-cluster binary authorization

Renaming / Copying

$ cat manifest.yml | reimage \
  -rename-ignore 'docker.example.com/registry/spitfire/.+$' \
  -rename-remote-path 'docker.example.com/registry/spitfire/imported' \
  > manifest-out.yaml

This will update all the image fields of standard resource types in manifest.yml. All the images matching rename-ignore will be left alone, all other images will be update to exist under the -rename-remote-path repository. The remote path is templateable using -rename-force-digest.

If the images do not exist in the remote repository they will be copied to the new location. If the images have been copied previously this can be disabled by using -no-copy (repository actions can be slow, so this can be a significant speed up).

If -rename-force-digest enabled, the image will first be renamed and copied as above, but will then be transferred into the full resolved digest form. The final mapping (see below), will be from the original name to the full resolved digest form of the renamed image. This ensure in-cluster images are guaranteed stable, and is also required for cluster with an enforced Grafeas/Kritis/BinAuthz image policy.

The following flags control renaming and copying

  -clobber
        allow overwriting remote images
  -no-copy
        disable copying of renamed images
  -rename-force-digest
        the final renamed image will be transformed to digest form before output
  -rename-ignore string
        ignore images matching this expression (default "^$")
  -rename-remote-path string
        template for remapping imported images
  -rename-template string
        template for remapping imported images (default "{{ .RemotePath }}/{{ .Registry }}/{{ .Repository }}:{{ .DigestHex }}")

Supporting Unknown K8S types

If you need to find images in non-standard k8s you can provide rules to reimage to help it find image fields. You can pass these rules using the -rules-config CLI flag.

- kind: ^Prometheus$                     # Regexp matching the k8s Kind of objects
  apiVersion: ^monitoring.coreos.com/v1$ # Regexp matching APIVersion of objects
  imageJSONP:
  - "$.spec.image"                       # JSONP queries that match image fields of a type

Stored Mappings

The mappings that result from the renaming of images can be written to a file, and/r directly to an OCI registry. You can then run reimage again, reading the exact set of mapping generated previously. This ensures that mappings of image tags to digests can be consistent between runs of reimage.

$ # write to a file
$ helm template -write-mappings-file mappings.json -remote-path example.com/registry/imported
$ # write to an image
$ helm template \
  -rename-remote-path 'docker.example.com/registry/spitfire/imported' \
  -write-mappings-img example.com/registry/imported/reimage-mapping:1234 \
  -remote-path example.com/registry/imported

The -mappings-only switches off the default yaml processing, and instead will apply any requested copying, vulnerability checking, and attestation against every image listed in the mappings file.

$ # read image from an image
$ helm template \
  -mappings-only
  -static-mappings-img example.com/registry/imported/reimage-mapping:1234 \
  -remote-path example.com/registry/imported

If vulnerability scanning (see below) is performed when the mappings are being written, the CVEs that exist in an image (but are below the max CVSS score, or explicitly ignored), are included in the image. This makes it easy to audit CVEs for a specific image.

The following flags control mappings usage

  -static-json-mappings-file string
        take all mappings from a mappings file
  -static-json-mappings-img string
        take all mapping from a mappings registry image
  -write-json-mappings-file string
        write final image mappings to a json file
  -write-json-mappings-img string
        write final image mapping to a registry image
  -mappings-only
        skip yaml processing, and image copying,  and just run checks and attestations from images in mappings

Grafeas Vulnerability Checking

Alternatively, reimage can execute any command compatible with trivy's image scanning JSON output to scan images.

alternatively trivy can check for Grafeas Discovery occurrences containing CVE checks for the discovered images. If discovery checking is enabled, but no completed discovery has occurred, reimage will wait for a configurable time. Vulnerability checking is disabled by default, and can be enabled by setting -vulncheck-max-cvss. If you want to scan, but ignore all CVEs, use -vulncheck-max-cvss 11

  -grafeas-parent string
        value for the parent of the grafeas client (e.g. "project/my-project-id" for GCP
  -trivy-command string
        the command to run to retrieve vulnerability scans in trivy's JSON format (the image id will be added as an additional arg (default "trivy image -f json")
  -vulncheck-method string
        force the vulnerability check method, (trivy or grafeas) (default "trivy")
  -vulncheck-ignore-cve-list string
        comma separated list of vulnerabilities to ignore
  -vulncheck-ignore-images string
        regexp of images to skip for CVE checks
  -vulncheck-max-cvss float
        maximum CVSS vulnerabitility score
  -vulncheck-timeout duration
        how long to wait for vulnerability scanning to complete (default 5m0s)

Grafeas Attestation

NOTE: At present, attestation support only works with Google Cloud BinAuthz attestors

reimage can add attestations for the images it has processed. For example, you can create an attestation that shows that images were required for our helm deploys

$  helmfile template --environment=staging | reimage \
     -grafeas-parent projects/my-registry \
     -binauthz-attestor projects/my-registry/attestors/helm-requires

Similarly you can attest that all images have gone to staging (and perhaps have passed integration testing).

$ reimage \
     -mappings-only
     -static-mappings-img example.com/registry/imported/reimage-mapping:1234 \
     -grafeas-parent projects/my-registry \
     -binauthz-attestor projects/my-registry/attestors/cleared-staging

`

Documentation

Overview

Package reimage provides tools for processing/updating the images listed in k8s manifests

Package reimage provides tools for processing/updating the images listed in k8s manifests

Package reimage provides tools for processing/updating the images listed in k8s manifests

Index

Constants

This section is empty.

Variables

View Source 
var (
	// DefaultTemplateStr is a sensible default for importing images
	DefaultTemplateStr = `{{ .RemotePath }}/{{ .Registry }}/{{ .Repository }}:{{ .DigestHex }}`

	// DefaultRulesConfig is a set of additional, non-core rules for known existing image
	// locations
	DefaultRulesConfig = []JSONImageFinderConfig{
		{
			Kind:       "^Prometheus$",
			APIVersion: `^monitoring\.coreos\.com/v1$`,
			ImageJSONP: []string{"$.spec.image"},
		},
	}

	// ErrDiscoveryNotFound is returned when no Vulnerability checking Discovery is associated with an image
	ErrDiscoveryNotFound = errors.New("discovery not found in response")

	// ErrDiscoverNotFinished is returned when Vulnerability checking did not complete in time
	ErrDiscoverNotFinished = errors.New("vulnerability checking not completed")

	// ErrAttestationNotFound is return if no attestations are present for a given image digest
	ErrAttestationNotFound = errors.New("attestation not found in response")
)
View Source 
var DefaultLogger = Logger(slog.Default())

DefaultLogger is a quick shortcut to the slog default logger

View Source 
var ErrSkip = errors.New("skip further processing")

Functions

func Process 

func Process(w io.Writer, r io.Reader, u Updater) error

Process runs the Updater for each kubernetes resource found in the file. Unknown field are converted to

Types

type EnsureRemapper added in v0.2.0 

type EnsureRemapper struct {
	NoClobber bool // If true, we'll refuse to overwrite remote images
	DryRun    bool // If true, don't perform the any actual copies

	Logger
}

EnsureRemapper is a mapper that will copy the original image reference to the latest, possibly remote, reference

func (*EnsureRemapper) ReMap added in v0.2.0 

func (t *EnsureRemapper) ReMap(h *History) error

ReMap copies the original reference to the latest, potentially remote reference

type GCPBinAuthzConcisePayload added in v0.2.0 

type GCPBinAuthzConcisePayload struct {
	DockerReference      string
	DockerManifestDigest string
}

GCPBinAuthzConcisePayload is a convenient wrapper around GCPBinAuthzPayload it with json.Marshal to a GCPBinAuthzPayload with correctly set Type

func (*GCPBinAuthzConcisePayload) MarshalJSON added in v0.2.0 

func (pl *GCPBinAuthzConcisePayload) MarshalJSON() ([]byte, error)

MarshalJSON marshals the provided type to JSON, but conforming to the structure of a GCPBinAuthzPayload

type GCPBinAuthzPayload added in v0.2.0 

type GCPBinAuthzPayload struct {
	Critical struct {
		Identity struct {
			DockerReference string `json:"docker-reference"`
		} `json:"identitiy"`
		Image struct {
			DockerManifestDigest string `json:"docker-manifest-digest"`
		} `json:"image"`
		Type string `json:"type"`
	} `json:"critical"`
}

GCPBinAuthzPayload is the mandated attestation note for signing Docker/OCI images for Google's Binauthz implementation

type GrafeasAttester added in v0.2.0 

type GrafeasAttester struct {
	Grafeas GrafeasClient
	Parent  string

	Keys    Keyer
	NoteRef string

	Logger
}

GrafeasAttester implements attestation creation and checking using Grafaes

func (*GrafeasAttester) Attest added in v0.2.0 

func (t *GrafeasAttester) Attest(ctx context.Context, dig name.Digest) error

Attest creates a NoteRef attestation for digest. It will skip this if one already exist

func (*GrafeasAttester) Check added in v0.2.0 

func (t *GrafeasAttester) Check(ctx context.Context, dig name.Digest) (bool, error)

Check confirms that a correctly signed attestation for NoteRef exists for the image digest

func (*GrafeasAttester) Get added in v0.2.0 

func (t *GrafeasAttester) Get(ctx context.Context, dig name.Digest, noteRef string) ([]*grafeaspb.AttestationOccurrence, error)

Get retrieves all the Attestation occurences for the given image that use the provided noteRef (or all if noteRef is "")

type GrafeasClient added in v0.2.0 

type GrafeasClient interface {
	ListOccurrences(ctx context.Context, req *grafeaspb.ListOccurrencesRequest, opts ...gax.CallOption) *grafeas.OccurrenceIterator
	CreateOccurrence(ctx context.Context, req *grafeaspb.CreateOccurrenceRequest, opts ...gax.CallOption) (*grafeaspb.Occurrence, error)
}

GrafeasClient still isn't mockable, need to wrap it

type GrafeasVulnGetter added in v0.7.0 

type GrafeasVulnGetter struct {
	Grafeas    GrafeasClient
	Parent     string
	RetryMax   int           // Max attempts to retrieve vulnerability discovery results
	RetryDelay time.Duration // Max time to wait for vulnerability discovery results

	Logger
}

GrafeasVulnGetter checks that images have been scanned, and checks that they do not contain unexpected vulnerabilities

func (*GrafeasVulnGetter) GetVulnerabilities added in v0.7.0 

func (vc *GrafeasVulnGetter) GetVulnerabilities(ctx context.Context, dig name.Digest) ([]ImageVulnerability, error)

Check waits for a completed vulnerability discovery, and then check that an image has no CVEs that violate the configured policy

type History 

type History struct {
	Refs      []name.Reference
	DigestStr string
}

History is the full set of updates performed so far

func NewHistory 

func NewHistory(ref name.Reference) *History

NewHistory starts a history for a given reference

func (*History) Add 

func (h *History) Add(ref name.Reference)

Add updates the history with a new reference mutation

func (*History) AddDigest added in v0.2.0 

func (h *History) AddDigest(ref name.Digest)

AddDigest sets the known image digest for the image being tracked by this history

func (*History) Latest 

func (h *History) Latest() name.Reference

Latest returns the most recent history update

func (*History) LatestDigest added in v0.2.0 

func (h *History) LatestDigest() (name.Digest, error)

LatestDigest constructs a digest name for the latest reference, and the original digest

func (*History) Original 

func (h *History) Original() name.Reference

Original returns the start of the mapping history

func (*History) OriginalDigest added in v0.2.0 

func (h *History) OriginalDigest() (name.Digest, error)

OriginalDigest looks through the history to find any previously looked up Digest of the original image. If none is found it is looked up and added to the history

type IgnoreRemapper added in v0.6.0 

type IgnoreRemapper struct {
	Ignore *regexp.Regexp
}

func (*IgnoreRemapper) ReMap added in v0.6.0 

func (t *IgnoreRemapper) ReMap(h *History) error

type ImageCheckError added in v0.2.0 

type ImageCheckError struct {
	Image   string
	MaxCVSS float32
	CVEs    map[string]float32
}

ImageCheckError is returned by Check if unwanted vulnerabilities are found

func (*ImageCheckError) Error added in v0.2.0 

func (ice *ImageCheckError) Error() string

type ImageSetters 

type ImageSetters []Setter

ImageSetters is list of one of more Setters

func (ImageSetters) Set 

func (ss ImageSetters) Set(img string)

Set all the image setters in the list to the provided image

type ImageVulnerability added in v0.7.0 

type ImageVulnerability struct {
	ID   string
	CVSS float32
}

type ImagesFinder 

type ImagesFinder interface {
	FindImages(obj *unstructured.Unstructured) (map[string]ImageSetters, error)
}

ImagesFinder specifies any mechanism for finding images within any k8s Unstructured data. Each entry in the map is an image name that was found. Calling the Set method on the map values will replace the discovered image name with a replacement.

func CompileJSONImageFinders 

func CompileJSONImageFinders(jmCfgs []JSONImageFinderConfig) (ImagesFinder, error)

CompileJSONImageFinders builds an ImagesFinder than can find image configuration strings from arbitrary unstructured K8S JSON objects, using JSONP queries

type JSONImageFinderConfig 

type JSONImageFinderConfig struct {
	Kind       string   `json:"kind" yaml:"kind"`             // regexp to match k8s kind
	APIVersion string   `json:"apiVersion" yaml:"apiVersion"` // regexp to match k8s apiVersion
	ImageJSONP []string `json:"imageJSONP" yaml:"imageJSONP"` // jsonP queries to find individual image fields
}

JSONImageFinderConfig describes the settings for finding arbitrary image fields in K8S types

type KMS added in v0.2.0 

type KMS struct {
	Client KMSClient
	Key    string
	// contains filtered or unexported fields
}

KMS uses Google Cloud KMS to sign and verify data. Only EC_SIGN_P256_SHA256 are supported at this time

func (*KMS) Sign added in v0.2.0 

func (ks *KMS) Sign(ctx context.Context, bs []byte) ([]byte, string, error)

Sign bs, returns the signature and key ID of the signing key

func (*KMS) Verify added in v0.2.0 

func (ks *KMS) Verify(ctx context.Context, bs []byte, data []byte) error

Verify the sig against the data

type KMSClient added in v0.2.0 

type KMSClient interface {
	AsymmetricSign(ctx context.Context, req *kmspb.AsymmetricSignRequest, opts ...gax.CallOption) (*kmspb.AsymmetricSignResponse, error)
	GetPublicKey(ctx context.Context, req *kmspb.GetPublicKeyRequest, opts ...gax.CallOption) (*kmspb.PublicKey, error)
}

KMSClient describes all the methods we require for a Google compatible signing service

type Keyer added in v0.2.0 

type Keyer interface {
	Sign(ctx context.Context, bs []byte) ([]byte, string, error)
	Verify(ctx context.Context, bs []byte, sig []byte) error
}

Keyer is an interface to a private key, for signing and verifying blobs

type Logger 

type Logger interface {
	Debug(msg string, args ...any)
	Info(msg string, args ...any)
}

Logger is a subset of the slog interface

type MultiRemapper 

type MultiRemapper []Remapper

MultiRemapper applies each remapper, passing results from one to the next.

func (MultiRemapper) ReMap 

func (t MultiRemapper) ReMap(h *History) error

ReMap applies each remapper, passing results from one to the next. An error is returned as soon as any remapper fails

type QualifiedImage added in v0.2.0 

type QualifiedImage struct {
	Tag         string   `json:"tag"`
	Digest      string   `json:"digest"`
	IgnoredCVEs []string `json:"ignoredCVEs,omitempty"`
	FoundCVEs   []string `json:"foundCVEs,omitempty"`
}

QualifiedImage describes an image tag, at a specific digest

type RecorderRemapper added in v0.2.0 

type RecorderRemapper struct {
	// contains filtered or unexported fields
}

RecorderRemapper records all remappings up as they are seen

func (*RecorderRemapper) Mappings added in v0.2.0 

func (r *RecorderRemapper) Mappings() (map[string]QualifiedImage, error)

Mappings returns the set of image original to final performed by all the remappers

func (*RecorderRemapper) ReMap added in v0.2.0 

func (r *RecorderRemapper) ReMap(h *History) error

ReMap records all remappings so far, should usuually be used as the final remapper

type Remapper 

type Remapper interface {
	ReMap(ref *History) error
}

A Remapper transforms OCI images references, and may perform side effects

type RenameRemapper added in v0.2.0 

type RenameRemapper struct {
	Ignore     *regexp.Regexp
	RemotePath string             // used for the .RemotePath value in the template
	RemoteTmpl *template.Template // template to build the final image string

	Logger
	// contains filtered or unexported fields
}

RenameRemapper is a Remapper implementation that can rename an image to a remote registry/repository path. The new path is built using RemoteTmpl, and the copy is performed using crane.Copy. reimage will then optionally copy the image to the new locatio

func (*RenameRemapper) ReMap added in v0.2.0 

func (t *RenameRemapper) ReMap(h *History) error

ReMap copies an image from the original registry to a given new destination registry

type RenameUpdater added in v0.2.0 

type RenameUpdater struct {
	Ignore                   *regexp.Regexp // Completely ignore images strings matching this regexp
	UnstructuredImagesFinder ImagesFinder
	Remapper                 Remapper
	ForceDigests             bool
}

RenameUpdater applies the Remapper to all images found in object passed to Update. For Objects of unknown types the UnstructuredImagesFinder is used. TODO(tcm): rename this thinger.

func (*RenameUpdater) Update added in v0.2.0 

func (s *RenameUpdater) Update(obj runtime.Object) error

Update applies the Remapper to all found images in the object

type RepoTemplateInput 

type RepoTemplateInput struct {
	RemotePath string // The request remote repository and registry prefix
	Digest     string // The digest of the image
	DigestAlgo string // The hash algorithm of the image digest
	DigestHex  string // The hex string of the digest hash
	Tag        string // The image tag (TODO(tcm): not used at the moment)
	Registry   string // The image registry
	Repository string // The image repository
}

RepoTemplateInput is the input provied to the RemoteTmpl of the RepoRemapper

type Setter 

type Setter func(img string)

A Setter is used for setting the string description of an image

type StaticRemapper added in v0.2.0 

type StaticRemapper struct {
	Mappings     map[string]QualifiedImage
	AllowMissing bool
}

StaticRemapper is a Remapper implementation that allows statically mapping incoming images to a pre-existing set of known target image names and digests

func NewStaticRemapper added in v0.2.0 

func NewStaticRemapper(mps map[string]QualifiedImage, confirmDigest bool) (*StaticRemapper, error)

NewStaticRemapper creates a StaticRemapper. If confirmDigest is true, the constructor will check that all target image tags still map to the currently referenced digest

func (*StaticRemapper) ReMap added in v0.2.0 

func (s *StaticRemapper) ReMap(h *History) error

ReMap looks up the incoming image in the provided mappings. If AllowMissing is false, attempts to look up images not in the static mappings will fail (if true, ReMap is a no-op)

type TrivyVulnGetter added in v0.7.0 

type TrivyVulnGetter struct {
	Command []string
}

func (*TrivyVulnGetter) GetVulnerabilities added in v0.7.0 

func (vc *TrivyVulnGetter) GetVulnerabilities(ctx context.Context, dig name.Digest) ([]ImageVulnerability, error)

type Updater 

type Updater interface {
	Update(obj runtime.Object) error
}

Updater is used by Process search for, and update, images in k8s objects

type VulnCheckResult added in v0.7.0 

type VulnCheckResult struct {
	Ignored []string // CVEs that were present, but explicitly ignored by the checker
	Found   []string // CVEs that were present, but under the max requested CVSS
}

VulnCheckResult is the result of a vulnerability check

type VulnChecker added in v0.7.0 

type VulnChecker struct {
	IgnoreImages  *regexp.Regexp // do not look for CVEs in images matching this pattern
	MaxCVSS       float32        // Maximum permitted CVSS score
	CVEIgnoreList []string       // CVEs to explicitly ignore
	Getter        VulnGetter     //

	Logger

	sync.Mutex
	// contains filtered or unexported fields
}

VulnChecker checks that images have been scanned, and checks that they do not contain unexpected vulnerabilities

func (*VulnChecker) Check added in v0.7.0 

func (vc *VulnChecker) Check(ctx context.Context, dig name.Digest) (*VulnCheckResult, error)

Check waits for a completed vulnerability discovery, and then check that an image has no CVEs that violate the configured policy

type VulnGetter added in v0.7.0 

type VulnGetter interface {
	GetVulnerabilities(ctx context.Context, dig name.Digest) ([]ImageVulnerability, error)
}

Source Files

View all Source files

Directories

Path Synopsis
cmd
reimage
Package main is the main reimage binary
 Package main is the main reimage binary

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.