inventory

package
v3.6.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 17, 2023 License: Apache-2.0 Imports: 31 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// Add represents those tags that are freely promotable, without fear of an
	// overwrite (we are only adding tags).
	Add TagOp = iota
	// Move represents those tags that conflict with existing digests, and so
	// should be moved to re-point to the digest that we want to promote as
	// defined in the manifest. It can be thought of a Delete followed by an
	// Add.
	Move = iota
	// Delete represents those tags that are not in the manifest and should thus
	// be removed and deleted. This is a kind of "demotion".
	Delete = iota
)

Variables

This section is empty.

Functions

func CheckOverlappingEdges 

func CheckOverlappingEdges(
	edges map[PromotionEdge]interface{},
) (map[PromotionEdge]interface{}, error)

CheckOverlappingEdges checks to ensure that all the edges taken together as a whole are consistent. It checks that there are no duplicate promotions desired to the same destination vertex (same destination PQIN). If the digests are the same for those edges, ignore because by definition the digests are cryptographically guaranteed to be the same thing (it doesn't matter if 2 different parties want the same exact image to become promoted to the same destination --- in many ways this is actually a good thing because it's a form of redundancy). However, return an error if the digests are different, because most likely this is at best just human error and at worst a malicious attack (someone trying to push an image to an endpoint they shouldn't own).

func EdgesToRegInvImage 

func EdgesToRegInvImage(
	edges map[PromotionEdge]interface{},
	destRegistry string,
) registry.RegInvImage

EdgesToRegInvImage takes the destination endpoints of all edges and converts their information to a RegInvImage type. It uses only those edges that are trying to promote to the given destination registry.

func FilterByTag 

func FilterByTag(rii registry.RegInvImage, filterTag string) registry.RegInvImage

FilterByTag removes all images in RegInvImage that do not match the filterTag.

func GetDeleteCmd 

func GetDeleteCmd(
	rc registry.Context,
	useServiceAccount bool,
	img image.Name,
	digest image.Digest,
	force bool,
) []string

GetDeleteCmd generates the cloud command used to delete images (used for garbage collection).

func GetTokenKeyDomainRepoPath 

func GetTokenKeyDomainRepoPath(registryName image.Registry) (key, domain, repoPath string)

GetTokenKeyDomainRepoPath splits a string by '/'. It's OK to do this because the RegistryName is already parsed against a Regex. (Maybe we should store the repo path separately when we do the initial parse...).

func IsSevereOccurrence 

func IsSevereOccurrence(
	vuln *grafeaspb.VulnerabilityOccurrence,
	severityThreshold int,
) bool

IsSevereOccurrence checks if a vulnerability is a high enough severity to fail the ImageVulnCheck.

func MkReadManifestListCmdReal 

func MkReadManifestListCmdReal(sc *SyncContext, gmlc *GCRManifestListContext) stream.Producer

MkReadManifestListCmdReal creates a stream.Producer which makes a real call over the network to read ManifestList information.

TODO: Consider replacing stream.Producer return type with a simple ([]byte, error) tuple instead.

func MkReadRepositoryCmdReal 

func MkReadRepositoryCmdReal(
	sc *SyncContext,
	rc registry.Context,
) stream.Producer

MkReadRepositoryCmdReal creates a stream.Producer which makes a real call over the network.

func ParseContainerParts 

func ParseContainerParts(s string) (
	registryName string,
	repo string,
	parseErr error,
)

ParseContainerParts splits up a registry name into its component pieces. Unfortunately it has some specialized logic around particular inputs; this could be removed in a future promoter manifest version which could force the user to provide these delineations for us.

TODO: Can we simplify this to not use switch/case/goto?

func SplitByKnownRegistries 

func SplitByKnownRegistries(
	r image.Registry,
	rcs []registry.Context,
) (image.Registry, image.Name, error)

SplitByKnownRegistries splits a registry name into a RegistryName and ImageName. The purpose of this function is to split a long image path into 2 pieces --- the repository and the image name. We can't just split by the last "/" all the time, because some manifests have an image with a "/" in it.

func SplitRegistryImagePath 

func SplitRegistryImagePath(
	registryImagePath RegistryImagePath,
	knownRegistries []image.Registry,
) (image.Registry, image.Name, error)

SplitRegistryImagePath takes an arbitrary image path, and splits it into its component parts, according to the knownRegistries field. E.g., consider "gcr.io/foo/a/b/c" as the registryImagePath. If "gcr.io/foo" is in knownRegistries, then we split it into "gcr.io/foo" and "a/b/c". But if we were given "gcr.io/foo/a", we would split it into "gcr.io/foo/a" and "b/c".

func ToFQIN 

func ToFQIN(registryName image.Registry, imageName image.Name, digest image.Digest) string

ToFQIN combines a RegistryName, ImageName, and Digest to form a fully-qualified image name (FQIN).

func ToLQIN 

func ToLQIN(registryName image.Registry, imageName image.Name) string

ToLQIN converts a RegistryName and ImageName to form a loosely-qualified image name (LQIN). Notice that it is missing tag information --- hence "loosely-qualified".

func ToPQIN 

func ToPQIN(registryName image.Registry, imageName image.Name, tag image.Tag) string

ToPQIN converts a RegistryName, ImageName, and Tag to form a partially-qualified image name (PQIN). It's less exact than a FQIN because the digest information is not used.

func ToPromotionEdges 

func ToPromotionEdges(mfests []schema.Manifest) (map[PromotionEdge]interface{}, error)

ToPromotionEdges converts a list of manifests to a set of edges we want to try promoting.

func ValidateRegistryImagePath 

func ValidateRegistryImagePath(rip RegistryImagePath) error

ValidateRegistryImagePath validates the RegistryImagePath.

Types

type CapturedRequests 

type CapturedRequests map[PromotionRequest]int

CapturedRequests holds a map of all PromotionRequests that were generated. It is used for both -dry-run and testing.

type CollectedLogs 

type CollectedLogs struct {
	Errors Errors
}

CollectedLogs holds all the Errors that are generated as the promoter runs.

type DigestMediaType 

type DigestMediaType map[image.Digest]cr.MediaType

DigestMediaType holds media information about a Digest.

type Error 

type Error struct {
	Context string
	Error   error
}

Error contains slightly more verbosity than a standard "error".

type Errors 

type Errors []Error

Errors is a slice of Errors.

type GCRManifestListContext 

type GCRManifestListContext struct {
	RegistryContext registry.Context
	ImageName       image.Name
	Tag             image.Tag
	Digest          image.Digest
}

GCRManifestListContext is used only for reading GCRManifestList information from GCR, in the function ReadGCRManifestLists.

type GCRPubSubPayload 

type GCRPubSubPayload struct {
	Action string `json:"action"`

	// The payload field is "digest", but really it is a FQIN like
	// "gcr.io/linusa/small@sha256:35f442d8d56cc7a2d4000f3417d71f44a730b900f3df440c09a9c40c42c40f86".
	FQIN string `json:"digest,omitempty"`

	// Similarly, the field "tag is always a PQIN.
	//
	// Example:
	// "gcr.io/linusa/small:a".
	PQIN string `json:"tag,omitempty"`

	Path string

	// Image digest, if any.
	Digest image.Digest

	// Tag, if any.
	Tag image.Tag
}

GCRPubSubPayload is the message payload sent to a Pub/Sub topic by a GCR.

func (*GCRPubSubPayload) Match 

func (payload *GCRPubSubPayload) Match(mfest *schema.Manifest) GcrPayloadMatch

Match checks whether a GCRPubSubPayload is mentioned in a Manifest. The degree of the match is reflected in the GcrPayloadMatch result.

func (*GCRPubSubPayload) PopulateExtraFields 

func (payload *GCRPubSubPayload) PopulateExtraFields() error

PopulateExtraFields takes the existing fields in GCRPubSubPayload and uses them to populate the extra convenience fields (these fields are derived from the FQIN and PQIN fields, and do not add any new information of their own). This is because the payload bundles up digests, tags, etc into a single string. Instead of dealing with them later on, we just break them up into the pieces we would like to use.

func (*GCRPubSubPayload) String 

func (payload *GCRPubSubPayload) String() string

Prettified prints the payload in a way that is stable and which hides extra fields which are redundant.

type GcrPayloadMatch 

type GcrPayloadMatch struct {
	// Path is true if the registry + image path (everything leading
	// up to either the digest or a tag) matches.
	PathMatch bool
	// Digest is set if the digest in the payload matches a digest in
	// the promoter manifest. This is ONLY matched if the path also matches.
	DigestMatch bool
	// Tag is ONLY matched if the digest also matches.
	TagMatch bool
	// Tag is only true if the digest matches, but the tag found in
	// the payload does NOT match what is found in the promoter manifest for the
	// digest. This can happen if somone manually tweaks a tag in GCR (assume
	// bad actor) to something other than what is specified in the promoter
	// manifest.
	TagMismatch bool
}

GcrPayloadMatch holds booleans for matching a GCRPubSubPayload against a promoter manifest.

type ImageRemovalCheck 

type ImageRemovalCheck struct {
	GitRepoPath    string
	MasterSHA      plumbing.Hash
	PullRequestSHA plumbing.Hash
	PullEdges      map[PromotionEdge]interface{}
}

ImageRemovalCheck implements the PreCheck interface and checks against pull requests that attempt to remove any images from the promoter manifests.

type ImageTag 

type ImageTag struct {
	Name image.Name
	Tag  image.Tag
}

ImageTag is a combination of the image.Name and Tag.

type ImageVulnCheck 

type ImageVulnCheck struct {
	SyncContext       *SyncContext
	PullEdges         map[PromotionEdge]interface{}
	SeverityThreshold int
	FakeVulnProducer  ImageVulnProducer
}

ImageVulnCheck implements the PreCheck interface and checks against images that have known vulnerabilities.

func MKImageVulnCheck 

func MKImageVulnCheck(
	syncContext *SyncContext,
	newPullEdges map[PromotionEdge]interface{},
	severityThreshold int,
	fakeVulnProducer ImageVulnProducer,
) *ImageVulnCheck

MKImageVulnCheck returns an instance of ImageVulnCheck which checks against images that have known vulnerabilities.

func (*ImageVulnCheck) Run 

func (check *ImageVulnCheck) Run() error

Run is a function of ImageVulnCheck and checks that none of the images to be promoted have any severe vulnerabilities.

type ImageVulnError 

type ImageVulnError struct {
	ImageName      image.Name
	Digest         image.Digest
	OccurrenceName string
	Vulnerability  *grafeaspb.VulnerabilityOccurrence
}

ImageVulnError contains ImageVulnCheck information on images that contain a vulnerability with a severity level at or above the defined threshold.

func (ImageVulnError) Error 

func (err ImageVulnError) Error() string

Error is a function of ImageVulnError and implements the error interface.

type ImageVulnProducer 

type ImageVulnProducer func(
	edge PromotionEdge,
) ([]*grafeaspb.Occurrence, error)

ImageVulnProducer is used by ImageVulnCheck to get the vulnerabilities for an image and allows for custom vulnerability producers for testing.

type MasterInventory 

type MasterInventory map[image.Registry]registry.RegInvImage

MasterInventory stores multiple RegInvImage elements, keyed by RegistryName.

type ParentDigest 

type ParentDigest map[image.Digest]image.Digest

ParentDigest holds a map of the digests of children to parent digests. It is a reverse mapping of ManifestLists, which point to all the child manifests.

type PopulateRequests 

type PopulateRequests func(
	*SyncContext,
	chan<- stream.ExternalRequest,
	*sync.WaitGroup)

PopulateRequests is a function that can generate requests used to fetch information about a Docker Registry, or to promote images. It basically generates the set of "gcloud ..." commands used to manipulate Docker Registries.

func MKPopulateRequestsForPromotionEdges 

func MKPopulateRequestsForPromotionEdges(
	toPromote map[PromotionEdge]interface{},
) PopulateRequests

MKPopulateRequestsForPromotionEdges takes in a map of PromotionEdges to promote and a PromotionContext and returns a PopulateRequests which can generate requests to be processed

type PreCheck 

type PreCheck interface {
	Run() error
}

PreCheck represents a check function to run against a pull request that modifies the promoter manifests before oking promotion of the changes.

Run runs the defined check and returns an error if the check fails, returns nil otherwise.

type ProcessRequest 

type ProcessRequest func(
	*SyncContext,
	chan stream.ExternalRequest,
	chan<- RequestResult,
	*sync.WaitGroup,
	*sync.Mutex)

ProcessRequest is the counterpart to PopulateRequests. It is a function that can take a request (generated by PopulateRequests) and process it. In the ictual implementation (e.g. in ReadDigestsAndTags()) it closes over some other local variables to record the change of state in the Docker Registry that was touched by processing the request.

func MkRequestCapturer 

func MkRequestCapturer(captured *CapturedRequests) ProcessRequest

MkRequestCapturer returns a function that simply records requests as they are captured (slurped out from the reqs channel).

type PromotionContext 

type PromotionContext func(
	image.Registry,
	image.Name,
	registry.Context,
	image.Name,
	image.Digest,
	image.Tag,
	TagOp,
) stream.Producer

PromotionContext holds all info required to create a stream that would produce a stream.Producer, as it relates to an intent to promote an image.

type PromotionEdge 

type PromotionEdge struct {
	SrcRegistry registry.Context
	SrcImageTag ImageTag

	Digest image.Digest

	DstRegistry registry.Context
	DstImageTag ImageTag
}

PromotionEdge represents a promotion "link" of an image repository between 2 registries.

func (*PromotionEdge) DstReference added in v3.4.3 

func (edge *PromotionEdge) DstReference() string

DstReference returns a reference pointing to the destination image

func (*PromotionEdge) SrcReference added in v3.4.3 

func (edge *PromotionEdge) SrcReference() string

SrcReference returns a reference pointing to the source image

func (*PromotionEdge) VertexProps 

func (edge *PromotionEdge) VertexProps(
	mi *MasterInventory,
) (d, s VertexProperty)

VertexProps determines the properties of each vertex (src and dst) in the edge, depending on the state of the world in the MasterInventory.

func (*PromotionEdge) VertexPropsFor 

func (edge *PromotionEdge) VertexPropsFor(
	rc *registry.Context,
	imageTag *ImageTag,
	mi *MasterInventory,
) VertexProperty

VertexPropsFor examines one of the two vertices (src or dst) of a PromotionEdge.

type PromotionRequest 

type PromotionRequest struct {
	TagOp          TagOp
	RegistrySrc    image.Registry
	RegistryDest   image.Registry
	ServiceAccount string
	ImageNameSrc   image.Name
	ImageNameDest  image.Name
	Digest         image.Digest
	DigestOld      image.Digest // Only for tag moves.
	Tag            image.Tag
}

PromotionRequest contains all the information required for any type of promotion (or demotion!) (involving any TagOp).

func (*PromotionRequest) PrettyValue 

func (pr *PromotionRequest) PrettyValue() string

PrettyValue is a prettified string representation of a PromotionRequest.

type RegistryImagePath 

type RegistryImagePath string

RegistryImagePath is the registry name and image name, without the tag. E.g. "gcr.io/foo/bar/baz/image".

type RequestResult 

type RequestResult struct {
	Context stream.ExternalRequest
	Errors  Errors
}

RequestResult contains information about the result of running a request (e.g., a "gcloud" command, or perhaps in the future, a REST call).

type RootRepo 

type RootRepo string

RootRepo is the toplevel Docker repository (e.g., gcr.io/foo (GCR domain name + GCP project name).

type SyncContext 

type SyncContext struct {
	sync.Mutex
	Threads           int
	Confirm           bool
	UseServiceAccount bool
	Inv               MasterInventory
	InvIgnore         []image.Name
	RegistryContexts  []registry.Context
	SrcRegistry       *registry.Context
	Tokens            map[RootRepo]gcloud.Token
	DigestMediaType   DigestMediaType
	ParentDigest      ParentDigest
	Logs              CollectedLogs
}

SyncContext is the main data structure for performing the promotion.

func MakeSyncContext 

func MakeSyncContext(
	mfests []schema.Manifest,
	threads int,
	confirm, useSvcAcc bool,
) (*SyncContext, error)

MakeSyncContext creates a SyncContext.

func (*SyncContext) ClearRepository 

func (sc *SyncContext) ClearRepository(
	regName image.Registry,
	mkProducer func(registry.Context, image.Name, image.Digest) stream.Producer,
	customProcessRequest *ProcessRequest,
)

ClearRepository wipes out all Docker images from a registry! Use with caution.

TODO: Maybe split this into 2 parts, so that each part can be unit-tested separately (deletion of manifest lists vs deletion of other media types).

func (*SyncContext) ExecRequests 

func (sc *SyncContext) ExecRequests(
	populateRequests PopulateRequests,
	processRequest ProcessRequest,
) error

ExecRequests uses the Worker Pool pattern, where MaxConcurrentRequests determines the number of workers to spawn.

func (*SyncContext) FilterPromotionEdges 

func (sc *SyncContext) FilterPromotionEdges(
	edges map[PromotionEdge]interface{}, readRepos bool,
) (nedges map[PromotionEdge]interface{}, gotClean bool, err error)

FilterPromotionEdges generates all "edges" that we want to promote.

func (*SyncContext) GetPromotionCandidates 

func (sc *SyncContext) GetPromotionCandidates(edges map[PromotionEdge]interface{}) (
	map[PromotionEdge]interface{},
	bool,
)

This filters out those edges from ToPromotionEdges (found in []Manifest), to only those PromotionEdges that makes sense to keep around. For example, we want to remove all edges that have already been promoted.

func (*SyncContext) IgnoreFromPromotion 

func (sc *SyncContext) IgnoreFromPromotion(regName image.Registry)

IgnoreFromPromotion works by building up a new Inv type of those images that should NOT be bothered to be Promoted; these get ignored in the Promote() step later down the pipeline.

func (*SyncContext) LogJSONSummary 

func (sc *SyncContext) LogJSONSummary()

LogJSONSummary logs the SyncContext's Logs as a prettified JSON.

func (*SyncContext) PopulateTokens 

func (sc *SyncContext) PopulateTokens() error

PopulateTokens populates the SyncContext's Tokens map with actual usable access tokens.

func (*SyncContext) PrintCapturedRequests 

func (sc *SyncContext) PrintCapturedRequests(capReqs *CapturedRequests)

PrintCapturedRequests pretty-prints all given PromotionRequests.

func (*SyncContext) Promote 

func (sc *SyncContext) Promote(
	edges map[PromotionEdge]interface{},
	customProcessRequest *ProcessRequest,
) error

Promote performs container image promotion by realizing the intent in the Manifest.

func (*SyncContext) ReadGCRManifestLists 

func (sc *SyncContext) ReadGCRManifestLists(
	mkProducer func(*SyncContext, *GCRManifestListContext) stream.Producer,
)

ReadGCRManifestLists reads all manifest lists and populates the ParentDigest field of the SyncContext. ParentDigest is a map of values of the form map[ChildDigest]ParentDigest; and so, if a digest has an entry in this map, it is referenced by a parent DockerManifestList.

TODO: Combine this function with ReadRegistries().

func (*SyncContext) ReadRegistries 

func (sc *SyncContext) ReadRegistries(
	toRead []registry.Context,
	recurse bool,
	mkProducer func(*SyncContext, registry.Context) stream.Producer,
)

ReadRegistries reads all images in all registries in the SyncContext Each registry is composed of a image repositories, which can be recursive.

To summarize: a docker *registry* is a set of *repositories*. It just so happens that to end-users, repositores resemble a tree structure because they are delineated by familiar filesystem-like "directory" paths.

We use the term "registry" to mean the "root repository" in this program, but to be technically correct, for gcr.io/google-containers/foo/bar/baz:

  • gcr.io is the registry
  • gcr.io/google-containers is the toplevel repository (or "root" repo)
  • gcr.io/google-containers/foo is a child repository
  • gcr.io/google-containers/foo/bar is a child repository
  • gcr.io/google-containers/foo/bar/baz is a child repository

It may or may not be the case that the child repository is empty. E.g., if only one image gcr.io/google-containers/foo/bar/baz:1.0 exists in the entire registry, the foo/ and bar/ subdirs are empty repositories.

The root repo, or "registry" in the loose sense, is what we care about. This is because in GCR, each root repo is given its own service account and credentials that extend to all child repos. And also in GCR, the name of the root repo is the same as the name of the GCP project that hosts it.

NOTE: Repository names may overlap with image names. e.g., it may be in the example above that there are images named gcr.io/google-containers/foo:2.0 and gcr.io/google-containers/foo/baz:2.0.

func (*SyncContext) ReadRegistriesGGCR 

func (sc *SyncContext) ReadRegistriesGGCR(
	toRead []registry.Context, recurse bool,
) error

func (*SyncContext) RemoveChildDigestEntries 

func (sc *SyncContext) RemoveChildDigestEntries(rii registry.RegInvImage) registry.RegInvImage

RemoveChildDigestEntries removes all tagless images in RegInvImage that are referenced by ManifestLists in the Registries.

func (*SyncContext) RunChecks 

func (sc *SyncContext) RunChecks(preChecks []PreCheck) error

RunChecks runs defined PreChecks in order to check the promotion.

func (*SyncContext) ValidateEdge 

func (sc *SyncContext) ValidateEdge(edge *PromotionEdge) error

ValidateEdge checks to see if there are any malformed edges. Currently this check is limited to detecting attempted tag moves in the destination registry.

func (*SyncContext) ValidateEdges 

func (sc *SyncContext) ValidateEdges(edges map[PromotionEdge]interface{}) error

ValidatesEdges runs ValidateEdge for each given edge, collecting all errors found.

type TagOp 

type TagOp int

TagOp is an enum that describes the various types of tag-modifying operations. These actions are a bit more low-level, and currently support 3 operations: adding, moving, and deleting.

func (*TagOp) PrettyValue 

func (op *TagOp) PrettyValue() string

PrettyValue is a prettified string representation of a TagOp.

type VertexProperty 

type VertexProperty struct {
	// Pqin means that the entire path, including the registry name, image
	// name, and tag, in that combination, exists.
	PqinExists      bool
	DigestExists    bool
	PqinDigestMatch bool
	BadDigest       image.Digest
	OtherTags       registry.TagSlice
}

VertexProperty describes the properties of an Edge, with respect to the state of the world.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.