manifest

package
v0.0.0-...-3c039c4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 24, 2025 License: BSD-3-Clause Imports: 13 Imported by: 0

Documentation

Overview

Package manifest defines structure of YAML files with target definitions.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BuildStep 

type BuildStep struct {

	// Dest specifies a location to put the result into.
	//
	// Applies to `copy`, `go_build` and `go_gae_bundle` steps.
	//
	// Usually prefixed with "${contextdir}/" to indicate it is relative to
	// the context directory.
	//
	// Optional in the original YAML, always populated after Manifest is parsed.
	// See individual *BuildStep structs for defaults.
	Dest string `yaml:"dest,omitempty"`

	// Cwd is a working directory to run the command in.
	//
	// Applies to `run` and `go_build` steps.
	//
	// Default is ${inputsdir}, ${contextdir} or ${manifestdir}, whichever is set.
	Cwd string `yaml:"cwd,omitempty"`

	CopyBuildStep        `yaml:",inline"` // copy a file or directory into the output
	GoBuildStep          `yaml:",inline"` // build go binary using "go build"
	RunBuildStep         `yaml:",inline"` // run a command that modifies the checkout
	GoGAEBundleBuildStep `yaml:",inline"` // bundle Go source code for GAE
	// contains filtered or unexported fields
}

BuildStep is one local build operation.

It takes a local checkout and produces one or more output files put into the context directory.

This struct is a "case class" with union of all supported build step kinds. The chosen "case" is returned by Concrete() method.

func (*BuildStep) Concrete 

func (bs *BuildStep) Concrete() ConcreteBuildStep

Concrete returns a pointer to some concrete populated *BuildStep.

type CloudBuildBuilder 

type CloudBuildBuilder struct {
	// Project is Google Cloud Project name that hosts the Cloud Build instance.
	Project string `yaml:"project"`

	// Pool is a private worker pool to run builds on (if any).
	Pool *WorkerPool `yaml:"pool,omitempty"`

	// Executable is a name of the container to execute as Cloud Build step.
	//
	// May include ":<tag>" suffix. Its working directory will be "/workspace" and
	// it will be populated with files from the context directory, as prepared by
	// the cloudbuildhelper.
	Executable string `yaml:"executable"`

	// Args is command line arguments to pass to the executable.
	//
	// Following tokens will be substituted by the cloudbuildhelper before passing
	// the command to Cloud Build:
	//   * ${CBH_DOCKER_IMAGE}: the full image name (including the registry prefix
	//     and ":<tag>" suffix) that the Cloud Build step should build.
	//   * ${CBH_DOCKER_LABELS}: list of ["--label", "k=v", "--label", "k=v", ...]
	//     pairs with all labels to pass to `docker build`. Can be passed directly
	//     to `docker build`.
	Args []string `yaml:"args"`

	// PushesExplicitly, if true, indicates the step pushes the final image to
	// the registry itself, instead of relying on Cloud Build to do the push.
	//
	// Note that PushesExplicitly == true somewhat reduces security properties of
	// the build, since the pushed image can theoretically be replaced with
	// something else between the moment it was pushed by the builder and
	// cloudbuildhelper resolves its tag into a SHA256 digest. When pushing via
	// Cloud Build (i.e. PushesExplicitly == false) Cloud Build reports the image
	// SHA256 digest directly (based on its local docker cache) and the tag is
	// not used at all.
	PushesExplicitly bool `yaml:"pushes_explicitly"`

	// Timeout is how long the Cloud Build build is allowed to run.
	//
	// Default timeout is ten minutes. Specified as a duration in seconds,
	// terminated by 's'. Example: "3.5s".
	Timeout string `yaml:"timeout"`
}

CloudBuildBuilder contains a configuration of Cloud Build infrastructure.

It has a name. Individual targets can specify what configuration they want to use via `builder` field of `cloudbuild` section.

type CloudBuildConfig 

type CloudBuildConfig struct {
	Builder string `yaml:"builder"` // what named configuration to use
}

CloudBuildConfig contains target-specific Cloud Build configuration.

It is used to select/tweak a named configuration preset specified in `cloudbuild` field of `infra` section.

type ConcreteBuildStep 

type ConcreteBuildStep interface {
	Kind() string   // used by -restrict-build-steps
	String() string // used for human logs only, doesn't have to encode all details
	// contains filtered or unexported methods
}

ConcreteBuildStep is implemented by various *BuildStep structs.

type CopyBuildStep 

type CopyBuildStep struct {
	// Copy is a path to copy files from.
	//
	// Should start with either "${contextdir}/", "${inputsdir}/" or
	// "${manifestdir}/" to indicate the root path.
	//
	// Can either be a directory or a file. Whatever it is, it will be put into
	// the output as Dest. By default Dest is "${contextdir}/<basename of Copy>"
	// (i.e. we copy Copy into the root of the context dir).
	Copy string `yaml:"copy,omitempty"`

	// Ignore is a list of patterns of files and directories to *not* copy.
	//
	// The format of patterns is the same as used in .gitignore.
	Ignore []string `yaml:"ignore,omitempty"`
}

CopyBuildStep indicates we want to copy a file or directory.

Doesn't materialize copies on disk, just puts them directly into the output file set.

func (*CopyBuildStep) Kind 

func (s *CopyBuildStep) Kind() string

func (*CopyBuildStep) String 

func (s *CopyBuildStep) String() string

type GoBuildStep 

type GoBuildStep struct {
	// GoBinary specifies a go command binary to build.
	//
	// This is a path (relative to GOPATH) to some 'main' package. It will be
	// built roughly as:
	//
	//  $ CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build <go_binary> -o <dest>
	//
	// Where <dest> is taken from Dest and it must be under the context directory.
	// It is set to "${contextdir}/<go package name>" by default.
	GoBinary string `yaml:"go_binary,omitempty"`
}

GoBuildStep indicates we want to build a go command binary.

Doesn't materialize the build output on disk, just puts it directly into the output file set.

func (*GoBuildStep) Kind 

func (s *GoBuildStep) Kind() string

func (*GoBuildStep) String 

func (s *GoBuildStep) String() string

type GoGAEBundleBuildStep 

type GoGAEBundleBuildStep struct {
	// GoGAEBundle is path to GAE module YAML.
	GoGAEBundle string `yaml:"go_gae_bundle,omitempty"`
	// ModulesMode is an opt-in into bundling as a Go module.
	ModulesMode bool `yaml:"go_gae_bundle_as_module,omitempty"`
}

GoGAEBundleBuildStep can be used to prepare a tarball with Go GAE app source.

Given a path to a GAE service yaml (that should reside in a directory with some `main` go package), it:

  • Copies all files in the main package directory (including all non-go files) to `_gopath/src/<its import path>`
  • Copies all *.go code with transitive dependencies to `_gopath/src/`.
  • Makes `Dest` a symlink pointing to `_gopath/src/<import path>`.

If `go_gae_bundle_as_module` is True, it instead:

  • Copies all files in the main package directory (including all non-go files) to `_gomod/<its import path relative to module root>`
  • Copies all *.go code with transitive dependencies from the main module to `_gomod/<import path relative to module root>`.
  • Copies all *.go code with transitive dependencies from other modules to `_gomod/vendor/<import path>`.
  • Generates `_gomod/go.mod` and `_gomod/vendor/modules.txt` based on the original `go.mod` and discovered dependencies.
  • Makes `Dest` a symlink pointing to the path created in the first step.

This ensures "gcloud app deploy" eventually can upload all *.go files needed to deploy a module.

func (*GoGAEBundleBuildStep) Kind 

func (s *GoGAEBundleBuildStep) Kind() string

func (*GoGAEBundleBuildStep) String 

func (s *GoGAEBundleBuildStep) String() string

type Infra 

type Infra struct {
	// Storage specifies Google Storage location to store *.tar.gz tarballs
	// produced after executing all local build steps.
	//
	// Expected format is "gs://<bucket>/<prefix>". Tarballs will be stored as
	// "gs://<bucket>/<prefix>/<name>/<sha256>.tar.gz", where <name> comes from
	// the manifest and <sha256> is a hex sha256 digest of the tarball.
	//
	// The bucket should exist already. Its contents is trusted, i.e. if there's
	// an object with desired <sha256>.tar.gz there already, it won't be replaced.
	//
	// Required when using Cloud Build.
	Storage string `yaml:"storage"`

	// Registry is a Cloud Registry to push images to e.g. "gcr.io/something".
	//
	// Required when using Cloud Build.
	Registry string `yaml:"registry"`

	// CloudBuild contains configuration presets of Cloud Build infrastructure.
	//
	// Each entry defines a named Cloud Build configuration that can be referenced
	// in individual manifests via `builder` field of `cloudbuild` section.
	CloudBuild map[string]CloudBuildBuilder `yaml:"cloudbuild"`

	// Notify indicates what downstream services to notify once the image is
	// built.
	//
	// It is not interpreted by cloudbuildhelper itself (just validated), and
	// passed to the JSON output of `build` and `upload` commands. Callers
	// (usually the recipes) know the meaning of this field and implement the
	// actual notification logic.
	Notify []NotifyConfig `yaml:"notify"`
}

Infra contains configuration of build infrastructure to use: Google Storage bucket, Cloud Build project, etc.

Note: when adding new fields here, check if they need to have matching restrictions in restrictions.go.

func (*Infra) ResolveCloudBuildConfig 

func (i *Infra) ResolveCloudBuildConfig(cfg CloudBuildConfig) (*CloudBuildBuilder, error)

ResolveCloudBuildConfig combines CloudBuildConfig specified in the target manifest with the corresponding CloudBuildBuilder in this infra section and returns the resolved validated configuration to use when calling Cloud Build.

type Manifest 

type Manifest struct {
	// Name is the name of this target, required.
	//
	// When building Docker images it is an image name (without registry or any
	// tags).
	Name string `yaml:"name"`

	// ManifestDir is a directory that contains this manifest file.
	//
	// Populated when it is loaded.
	ManifestDir string `yaml:"-"`

	// Extends is a unix-style path (relative to this YAML file) to a manifest
	// used as a base.
	//
	// Optional.
	//
	// Such base manifests usually contain definitions shared by many files, such
	// as "imagepins" and "infra".
	//
	// Dicts are merged (recursively), lists are joined (base entries first).
	Extends string `yaml:"extends,omitempty"`

	// Dockerfile is a unix-style path to the image's Dockerfile, relative to this
	// YAML file.
	//
	// If unset, but there exists `${contextdir}/Dockerfile`, it will be used
	// instead (similarly to how Docker discovers Dockerfile).
	//
	// All images referenced in a Dockerfile are resolved into concrete digests
	// via an external file. See ImagePins field for more information.
	Dockerfile string `yaml:"dockerfile,omitempty"`

	// ContextDir is a unix-style path to the directory to use as a basis for
	// the build. The path is relative to this YAML file.
	//
	// All non-gitignored files there end up available to the remote builder
	// (e.g. a docker daemon will see this directory as a context directory when
	// building the image).
	//
	// All symlinks there are resolved to their targets. Only +w and +x file mode
	// bits are preserved (all files have 0444 mode by default, +w adds additional
	// 0200 bit and +x adds additional 0111 bis). All other file metadata (owners,
	// setuid bits, modification times) are ignored.
	//
	// The default value depends on whether Dockerfile is set. If it is, then
	// ContextDir defaults to the directory with Dockerfile. Otherwise the context
	// directory is assumed to be empty.
	ContextDir string `yaml:"contextdir,omitempty"`

	// InputsDir is an optional directory that can be used to reference files
	// consumed by build steps (as "${inputsdir}/path").
	//
	// Unlike ContextDir, its full content does not automatically end up in the
	// output.
	//
	// If unset, defaults to ContextDir.
	InputsDir string `yaml:"inputsdir,omitempty"`

	// ImagePins is a unix-style path to the YAML file with pre-resolved mapping
	// from (docker image, tag) pair to the corresponding docker image digest.
	//
	// The path is relative to the manifest YAML file. It should point to a YAML
	// file with the following structure:
	//
	//    pins:
	//      - image: <img>
	//        tag: <tag>
	//        digest: sha256:<sha256>
	//      - image: <img>
	//        tag: <tag>
	//        digest: sha256:<sha256>
	//      ...
	//
	// See dockerfile.Pins struct for more details.
	//
	// This file will be used to rewrite the input Dockerfile to reference all
	// images (in "FROM ..." lines) only by their digests. This is useful for
	// reproducibility of builds.
	//
	// Only following forms of "FROM ..." statement are allowed:
	//  * FROM <image> [AS <name>] (assumes "latest" tag)
	//  * FROM <image>[:<tag>] [AS <name>] (resolves the given tag)
	//  * FROM <image>[@<digest>] [AS <name>] (passes the definition through)
	//
	// In particular ARGs in FROM line (e.g. "FROM base:${CODE_VERSION}") are
	// not supported.
	//
	// If not set, the Dockerfile must use only digests to begin with, i.e.
	// all FROM statements should have form "FROM <image>@<digest>".
	//
	// Ignored if Dockerfile field is not set.
	ImagePins string `yaml:"imagepins,omitempty"`

	// Deterministic is true if Dockerfile (with all "FROM" lines resolved) can be
	// understood as a pure function of inputs in ContextDir, i.e. it does not
	// depend on the state of the world.
	//
	// Examples of things that make Dockerfile NOT deterministic:
	//   * Using "apt-get" or any other remote calls to non-pinned resources.
	//   * Cloning repositories from "master" ref (or similar).
	//   * Fetching external resources using curl or wget.
	//
	// When building an image marked as deterministic, the builder will calculate
	// a hash of all inputs (including resolve Dockerfile itself) and check
	// whether there's already an image built from them. If there is, the build
	// will be skipped completely and the existing image reused.
	//
	// Images marked as non-deterministic are always rebuilt and reuploaded, even
	// if nothing in ContextDir has changed.
	Deterministic *bool `yaml:"deterministic,omitempty"`

	// Sources is unix-style paths to the directories that contain the source code
	// used to build the artifact, for the express purpose of getting its revision
	// and propagating as artifact's metadata.
	//
	// Values of this field are not involved in the build process itself at all.
	// If omitted, defaults to ${inputsdir} or ${contextdir} whichever is set.
	//
	// The paths are relative to this YAML file.
	Sources []string `yaml:"sources"`

	// Infra is configuration of the build infrastructure to use: Google Storage
	// bucket, Cloud Build project, etc.
	//
	// Keys are names of presets (like "dev", "prod"). What preset is used is
	// controlled via "-infra" command line flag (defaults to "dev").
	Infra map[string]Infra `yaml:"infra"`

	// CloudBuild specifies what Cloud Build configuration to use.
	//
	// All available configurations are defined in `cloudbuild` section of `infra`
	// section. This field allows to pick the active one of this target.
	CloudBuild CloudBuildConfig `yaml:"cloudbuild,omitempty"`

	// Build defines a series of local build steps.
	//
	// Each step may add more files to the context directory. The actual
	// `contextdir` directory on disk won't be modified. Files produced here are
	// stored in a temp directory and the final context directory is constructed
	// from the full recursive copy of `contextdir` and files emitted here.
	Build []*BuildStep `yaml:"build,omitempty"`
}

Manifest is a definition of what to build, how and where.

Comments here describe the structure of the manifest file on disk. In the loaded form all paths use filepath.Separator as a directory separator.

func Load 

func Load(path string) (*Manifest, error)

Load loads the manifest from the given path, traversing all "extends" links.

After the manifest is loaded, its fields (like ContextDir) can be manipulated (e.g. to set defaults), after which all "${dir}/" references in build steps must be resolved by a call to Finalize.

func (*Manifest) Finalize 

func (m *Manifest) Finalize() error

Finalize replaces "${dir}/" in paths in steps with actual values and fills in defaults.

type NotifyConfig 

type NotifyConfig struct {
	// Kind indicates a kind of service to notify.
	//
	// The only supported value now is "git", meaning to checkout a git repo and
	// invoke a script there with results of the build. Note that the actual
	// logic to do that is implemented in recipes that call cloudbuildhelper.
	Kind string `yaml:"kind" json:"kind"`

	// Repo is a git repo to checkout (as "https://..." URL).
	//
	// Effective only for "git" notifiers.
	Repo string `yaml:"repo" json:"repo"`

	// Script is a path to the script inside the repo to invoke.
	//
	// Effective only for "git" notifiers.
	Script string `yaml:"script" json:"script"`
}

NotifyConfig is a single item in `notify` list.

It is read from the YAML manifest and ends up in the -json-output.

func (*NotifyConfig) DestinationID 

func (n *NotifyConfig) DestinationID() string

DestinationID identifies the destination of the notification for the purpose of checking it against -restrict-notifications flags in restrictions.go.

type Restrictions 

type Restrictions struct {
	// contains filtered or unexported fields
}

Restrictions restrict what can be targeted by a manifest.

This is useful when `cloudbuildhelper` operates over partially untrusted manifests to limit what they can reach.

If some particular set is empty, the corresponding property is considered unrestricted.

func (*Restrictions) AddFlags 

func (r *Restrictions) AddFlags(fs *flag.FlagSet)

AddFlags register -restrict-* flags in the flagset.

func (*Restrictions) CheckBuildSteps 

func (r *Restrictions) CheckBuildSteps(steps []*BuildStep) (violations []string)

CheckBuildSteps checks if any of the build step restrictions are violated.

Returns a list of violations as human readable messages with details.

func (*Restrictions) CheckCloudBuild 

func (r *Restrictions) CheckCloudBuild(c *CloudBuildBuilder) (violations []string)

CheckCloudBuild checks if any of Cloud Build restrictions are violated.

Returns a list of violations as human readable messages with details.

func (*Restrictions) CheckInfra 

func (r *Restrictions) CheckInfra(m *Infra) (violations []string)

CheckInfra checks if any of the infra restrictions are violated.

Returns a list of violations as human readable messages with details.

func (*Restrictions) CheckTargetName 

func (r *Restrictions) CheckTargetName(name string) (violations []string)

CheckTargetName checks if the target name restrictions are violated.

Returns a list of violations as human readable messages with details.

type RunBuildStep 

type RunBuildStep struct {
	// Run indicates a command to run along with all its arguments.
	//
	// Strings that start with "${contextdir}/", "${inputsdir}/" or
	// "${manifestdir}/" will be rendered as absolute paths.
	Run []string `yaml:"run,omitempty"`

	// Outputs is a list of files or directories to put into the output.
	//
	// They are something that `run` should be generating.
	//
	// They are expected to be under "${contextdir}". A single output entry
	// "${contextdir}/generated/file" is equivalent to a copy step that "picks up"
	// the generated file:
	//   - copy: ${contextdir}/generated/file
	//     dest: ${contextdir}/generated/file
	//
	// If outputs are generated outside of the context directory, use `copy` steps
	// explicitly.
	Outputs []string
}

RunBuildStep indicates we want to run some arbitrary command.

The command may modify the checkout or populate the context dir.

func (*RunBuildStep) Kind 

func (s *RunBuildStep) Kind() string

func (*RunBuildStep) String 

func (s *RunBuildStep) String() string

type WorkerPool 

type WorkerPool struct {
	// Region is a Cloud Region that hosts this pool.
	Region string `yaml:"region"`
	// ID is the short pool ID.
	ID string `yaml:"id"`
}

WorkerPool is an ID of a worker pool within a project.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.