Documentation
¶
Overview ¶
Package ensure contains methods and types for interacting with the 'ensure file format'.
The format is used by the CIPD client to describe the desired state of a CIPD installation. This states can be asserted with the CIPD client 'ensure' command. The state is essentially a list of packages, their versions and their installation subdirectories ("subdirs").
Format Description ¶
The file is line-oriented. All statements fit on a single line.
A line can be blank, a comment, a setting, a package, or a directive.
Comments ¶
A comment begins with a # and goes to the end of the line. It is ignored. Example:
# This is a comment. It has no effect.
Settings ¶
A setting looks like `$name value`. Settings are global and can only be set once per file. The following settings are allowed:
- `$ServiceURL <url>` is the url for the CIPD service. It can be used in lieu of the -service-url command line parameter.
- `$VerifiedPlatform <platform> [ <platform>]*` allows the manifest to specify a list of ${platform} expansions to use for verification. Multiple $VerifiedPlatform directives can be specified, and will accumulate.
- `$ParanoidMode NotParanoid|CheckPresence|CheckIntegrity` controls how paranoid CIPD should be when checking the state of already installed packages. `NotParanoid` (default) indicates that CIPD should trust that no one is messing with the installation root directory. `CheckPresence` indicates CIPD should verify all files that are supposed to be already installed are indeed present (this incurs additional overhead). And finally `CheckIntegrity` will verify that files weren't modified since they were installed (this incurs even more overhead).
- `$ResolvedVersions <filename>` is a path (either relative to the ensure file or absolute) to a file that contains "frozen" instance IDs (hashes) of all packages for all verified platforms, resolved at the time the "frozen" file was generated by `cipd ensure-file-resolve`. The CIPD client will use these instance IDs (instead of contacting the backend) when installing packages in `cipd ensure`. Using this file allows to cryptographically bind the intended state of the deployment to a git commit, since all hashes of all packages become part of the git commit object. Use this if you want to reduce trust in the CIPD backend and root in the git server instead.
- `$OverrideInstallMode copy` forces all packages in this ensure-file to be installed with copy mode, regardless of what the package manifests list. `symlink` mode only works on mac/linux, and causes more problems than it solves. We recommend that all ensure files have this setting, and in the future this will become automatically set. See crbug.com/1329641 for additional discussion.
Package Definitions ¶
A package line looks like `<package_template> <version>`. Package templates are CIPD package names, with optional expansion parameters `${os}`, `${arch}`, and `${platform}`. These placeholders can appear anywhere in the package template except for the first letter. All other characters in the template are taken verbatim.
${os} will expand to one of the following, based on the value of GOOS used to build this client:
- windows
- mac
- linux
- freebsd
- netbsd
- ...
(basically, just the GOOS value, with `mac` instead of `darwin` as the only historical exception).
${arch} will expand to one of the following, based on combination of values of GOARCH, GOARM, GOAMD64, etc used to build this client:
- 386: GOARCH=386, GO386=sse2 (default)
- amd64: GOARCH=amd64, GOAMD64=v1 (default)
- arm64: GOARCH=arm64
- armv6l: GOARCH=arm, GOARM=6
- armv7l: GOARCH=arm, GOARM=7
- loong64: GOARCH=loong64
- mips: GOARCH=mips, GOMIPS=hardfloat (default)
- mips64: GOARCH=mips64, GOMIPS64=hardfloat (default)
- mips64le: GOARCH=mips64le, GOMIPS64=hardfloat (default)
- mipsle: GOARCH=mipsle, GOMIPS=hardfloat (default)
- ppc64: GOARCH=ppc64, GOPPC64=power8 (default)
- ppc64le: GOARCH=ppc64le, GOPPC64=power8 (default)
- riscv64: GOARCH=riscv64
- s390x: GOARCH=s390x
Since these two often appear together, the convenience placeholder `${platform}` expands to the equivalent of `${os}-${arch}`. Note that not all combinations of `${os}` and `${arch}` are supported (e.g. there's no windows-mips64).
All of these parameters also support the syntax ${var=possible,values}. What this means is that the package line will be expanded if, and only if, var equals one of the possible values. If that var does not match a possible value, the line is ignored. This allows you to do, e.g.:
path/to/package/${os=windows} windows_release
path/to/package/${os=linux} linux_release
# no version for mac
path/to/posix/tool/${os=mac,linux} some_tag:value
# no version for windows
Directives ¶
A directive looks like `@name value`. Directives are 'sticky' and apply until the next same-name directive. The following directive names are allowed:
- `@Subdir` allows you to change the subdirectory that subsequent packages are installed to.
- The subdir value is always relative to the root of the CIPD installation (the directory containing the .cipd folder).
- The value of Subdir before any @Subdir directives is "", or the root of the CIPD installation.
- You can reset the directory back to root by doing `@Subdir` by itself, without a value.
- @Subdir directives also support expansion parameters `${os}`, `${arch}` and `${platform}`. Using a subdir expansion like `${param=val}` will cause that subdirectory, and any packages in it, to only exist if the param matches one of the values.
Example ¶
Here is an example ensure file which demonstrates all the various features.
# This is an ensure file!
$ServiceURL https://chrome-infra-packages.appspot.com/
$ParanoidMode CheckPresence
$ResolvedVersions cipd_lock.versions
# This is the CIPD client itself
infra/tools/cipd/${os}-${arch} latest
@Subdir python
python/wheels/pip version:8.1.2
# use the convenience placeholder
python/wheels/coverage/${platform} version:4.1
@Subdir infra/support
infra/some/other/package deadbeefdeadbeefdeadbeefdeadbeefdeadbeef
# only exists on windows machines
@Subdir support/${os=windows}-${arch}
some/support/package latest
some/other/support/package latest
# Always exists, but the directory changes based on the os
@Subdir platform/${os}
a/platform/package latest
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type File ¶
type File struct {
ServiceURL string
ParanoidMode deployer.ParanoidMode
ResolvedVersions string
OverrideInstallMode pkg.InstallMode
PackagesBySubdir map[string]PackageSlice
VerifyPlatforms []template.Platform
}
File is an in-process representation of the 'ensure file' format.
func LoadEnsureFile ¶
LoadEnsureFile loads the ensure file from the given path or stdin (if 'path' is "-").
If the ensure file has $ResolvedVersions directive, the returned File will have ResolvedVersions field set to an absolute path to the resolved versions file. Its presence or correctness is not checked.
func ParseFile ¶
ParseFile parses an ensure file from the given reader. See the package docs for the format of this file.
This file will contain unresolved template strings for package names as well as unpinned package versions. Use File.Resolve() to obtain resolved+pinned versions of these.
func (*File) Resolve ¶
func (f *File) Resolve(rslv VersionResolver, expander template.Expander) (*ResolvedFile, error)
Resolve takes the current unresolved File and expands all package templates using the provided expander (usually template.DefaultExpander()), and also resolves all versions with the provided VersionResolver, calling it concurrently from multiple goroutines.
Returns either a single error (if something is wrong with the ensure file), or a multi-error with all resolution errors, sorted by definition line numbers.
type PackageDef ¶
type PackageDef struct {
PackageTemplate string
UnresolvedVersion string
// LineNo is set while parsing an ensure file by the ParseFile method. It is
// used by File.Resolve to give additional context if an error occurs.
LineNo int
}
PackageDef defines a package line parsed out of an ensure file.
func (*PackageDef) Expand ¶
func (p *PackageDef) Expand(expander template.Expander) (pkg string, err error)
Expand expands the package name template and checks that resulting package name and version are syntactically correct.
May return template.ErrSkipTemplate is this package definition should be skipped given the current expansion variables values.
func (PackageDef) String ¶
func (p PackageDef) String() string
type PackageSlice ¶
type PackageSlice []PackageDef
PackageSlice is a sortable slice of PackageDef
func (PackageSlice) Len ¶
func (ps PackageSlice) Len() int
func (PackageSlice) Less ¶
func (ps PackageSlice) Less(i, j int) bool
func (PackageSlice) Swap ¶
func (ps PackageSlice) Swap(i, j int)
type ResolvedFile ¶
type ResolvedFile struct {
ServiceURL string
ParanoidMode deployer.ParanoidMode
OverrideInstallMode pkg.InstallMode
PackagesBySubdir common.PinSliceBySubdir
}
ResolvedFile only contains valid, fully-resolved information and is the result of calling File.Resolve.
type VersionResolver ¶
VersionResolver transforms a {PackageName, Version} tuple (corresponding to the given `def`) into a resolved pin.
- `pkg` is guaranteed to pass common.ValidatePackageName
- `vers` is guaranteed to pass common.ValidateInstanceVersion
VersionResolver should expect to be called concurrently from multiple goroutines.
type VersionsFile ¶
type VersionsFile map[unresolvedVer]string
VersionsFile contains a mapping "(package name, version) -> instance ID" used to resolve versions (instead of backend calls) for ensure files that have $ResolvedVersions directive.
In serialized form it is represented as a set of triples separated by one or more new lines (there must be no new lines inside the triple):
""" # Comments are allowed, they are skipped (not even considered as '\n'). <package name> <version> <resolved instance ID>
<package name> <version> <resolved instance ID> """
Leading and trailing whitespace on a line is ignored.
In the canonical serialization triples are ordered by (package, version).
VersionsFile is safe for read-only concurrent use. Concurrent modifications should be protected by a lock. This is just a map in disguise.
func ParseVersionsFile ¶
func ParseVersionsFile(r io.Reader) (VersionsFile, error)
ParseVersionsFile parses previously serialized versions file.
func (VersionsFile) AddVersion ¶
func (v VersionsFile) AddVersion(pkg, ver, iid string) error
AddVersion adds (or overrides) an instance ID mapped to the given version.
Returns an error if any of the arguments is invalid.
If 'ver' is already an instance ID, just checks if it is equal to 'iid' and silently doesn't modify the map.
func (VersionsFile) Equal ¶
func (v VersionsFile) Equal(a VersionsFile) bool
Equal returns true if version files have same entries.
func (VersionsFile) ResolveVersion ¶
func (v VersionsFile) ResolveVersion(pkg, ver string) (common.Pin, error)
ResolveVersion returns a pin matching the given version or an error if such version is not in the map.
If 'ver' is already an instance ID, returns it right away.
Source Files