condaenvupdate

README

Paketo Buildpack for Conda Env Update Cloud Native

The Conda Env Update Buildpack runs commands to update the conda environment. It installs the conda environment into a layer which makes it available for subsequent buildpacks and in the final running container.

The buildpack is published for consumption at gcr.io/paketo-buildpacks/conda-env-update and paketobuildpacks/conda-env-update.

Behavior

This buildpack participates when there is an environment.yml or package-list.txt file in the app directory.

The buildpack will do the following:

• At build time:
  • Requires that conda has already been installed in the build container
  • Updates the conda environment and stores it in a layer
    • If a package-list.txt is in the app dir, a new environment is created from it
    • Otherwise, the environment.yml file is used to update the environment
  • Reuses the cached conda environment layer from a previous build if and only if a package-list.txt is in the app dir and it has not changed since the previous build
• At run time:
  • Does nothing

Integration

The Conda Env Update Buildpack provides conda-environment as a dependency. Downstream buildpacks can require the conda-environment dependency by generating a Build Plan TOML file that looks like the following:

[[requires]]
# The name of the Conda Env Update dependency is "conda-environment". This value is considered
# part of the public API for the buildpack and will not change without a plan
# for deprecation.
name = "conda-environment"

# The Conda Env Update buildpack supports some non-required metadata options.
[requires.metadata]

# Setting the build flag to true will ensure that the conda environment
# layer is available for subsequent buildpacks during their build phase.
# If you are writing a buildpack that needs the conda environment
# during its build process, this flag should be set to true.
build = true

# Setting the launch flag to true will ensure that the conda environment is
# available to the running application. If you are writing an application
# that needs to use the conda environment at runtime, this flag should be set to true.
launch = true

SBOM

This buildpack can generate a Software Bill of Materials (SBOM) for the dependencies of an application.

However, this feature only works if an application vendors its dependencies in the vendor directory. This is due to a limitation in the upstream SBOM generation library (Syft).

Applications that declare their dependencies via a package-list.txt file but do not vendor them will result in an empty SBOM. Check out the Paketo SBOM documentation for more information about how to access the SBOM.

Usage

To package this buildpack for consumption:

$ ./scripts/package.sh --version <version-number>

This will create a buildpackage.cnb file under the build directory which you can use to build your app as follows: pack build <app-name> -p <path-to-app> -b build/buildpackage.cnb -b <other-buildpacks..>

To run the unit and integration tests for this buildpack:

$ ./scripts/unit.sh && ./scripts/integration.sh

Documentation

Index

• Constants
• func Build(runner Runner, sbomGenerator SBOMGenerator, logger scribe.Emitter, ...) packit.BuildFunc
• func Detect() packit.DetectFunc
• type CondaRunner
  • func NewCondaRunner(executable Executable, summer Summer, logger scribe.Emitter) CondaRunner
  • func (c CondaRunner) Execute(condaLayerPath string, condaCachePath string, workingDir string) error
  • func (c CondaRunner) ShouldRun(workingDir string, metadata map[string]interface{}) (run bool, sha string, err error)
• type Executable
• type Runner
• type SBOMGenerator
• type Summer

Constants

const (
	// CondaEnvLayer is the name of the layer into which conda environment is installed.
	CondaEnvLayer = "conda-env"

	// CondaEnvCache is the name of the layer that is used as the conda package directory.
	CondaEnvCache = "conda-env-cache"

	// CondaEnvPlanEntry is the name of the Build Plan requirement that this buildpack provides.
	CondaEnvPlanEntry = "conda-environment"

	// CondaPlanEntry is the name of the Build Plan requirement for the miniconda
	// dependency that this buildpack requires.
	CondaPlanEntry = "conda"

	// LockfileShaName is the key in the Layer Content Metadata used to determine if layer
	// can be reused.
	LockfileShaName = "lockfile-sha"

	// LockfileName is the name of the export file from which the buildpack reinstalls packages
	// See https://docs.conda.io/projects/conda/en/latest/commands/list.html
	LockfileName = "package-list.txt"

	// EnvironmentFileName is the name of the conda environment file.
	EnvironmentFileName = "environment.yml"
)

Functions

func Build

func Build(runner Runner, sbomGenerator SBOMGenerator, logger scribe.Emitter, clock chronos.Clock) packit.BuildFunc

Build will return a packit.BuildFunc that will be invoked during the build phase of the buildpack lifecycle.

Build updates the conda environment and stores the result in a layer. It may reuse the environment layer from a previous build, depending on conditions determined by the runner.

func Detect

func Detect() packit.DetectFunc

Detect returns a packit.DetectFunc that will be invoked during the detect phase of the buildpack lifecycle.

Detection passes when there is an environment.yml or package-list.txt file in the app directory, and will contribute a Build Plan that provides conda-environment and requires conda.

Types

type CondaRunner

type CondaRunner struct {
	// contains filtered or unexported fields
}

CondaRunner implements the Runner interface.

func NewCondaRunner

func NewCondaRunner(executable Executable, summer Summer, logger scribe.Emitter) CondaRunner

NewCondaRunner creates an instance of CondaRunner given an Executable, a Summer, and a Logger.

func (CondaRunner) Execute

func (c CondaRunner) Execute(condaLayerPath string, condaCachePath string, workingDir string) error

Execute runs the conda environment setup command and cleans up unnecessary artifacts. If a vendor directory is present, it uses vendored packages and installs them in offline mode. If a packages-list.txt file is present, it creates a new environment based on the packages list. Otherwise, it updates the existing packages to their latest versions.

For more information about the commands used, see: https://docs.conda.io/projects/conda/en/latest/commands/create.html https://docs.conda.io/projects/conda/en/latest/commands/update.html https://docs.conda.io/projects/conda/en/latest/commands/clean.html

func (CondaRunner) ShouldRun

func (c CondaRunner) ShouldRun(workingDir string, metadata map[string]interface{}) (run bool, sha string, err error)

ShouldRun determines whether the conda environment setup command needs to be run, given the path to the app directory and the metadata from the preexisting conda-env layer. It returns true if the conda environment setup command must be run during this build, the SHA256 of the package-list.txt in the app directory, and an error. If there is no package-list.txt, the sha returned is an empty string.

type Executable

type Executable interface {
	Execute(pexec.Execution) error
}

Executable defines the interface for invoking an executable.

type Runner

type Runner interface {
	Execute(condaEnvPath string, condaCachePath string, workingDir string) error
	ShouldRun(workingDir string, metadata map[string]interface{}) (bool, string, error)
}

Runner defines the interface for setting up the conda environment.

type SBOMGenerator

type SBOMGenerator interface {
	Generate(dir string) (sbom.SBOM, error)
}

type Summer

type Summer interface {
	Sum(arg ...string) (string, error)
}

Summer defines the interface for computing a SHA256 for a set of files and/or directories.