cpython

README

Paketo Buildpack for CPython Cloud Native

The CPython Buildpack provides CPython (reference implementation of Python) 3. The buildpack installs CPython onto the $PATH which makes it available for subsequent buildpacks and in the final running container. It also sets the $PYTHONPATH environment variable for subsequent buildpacks and launch-time processes.

The buildpack is published for consumption at gcr.io/paketo-buildpacks/cpython and paketobuildpacks/cpython.

Behavior

This buildpack always participates.

Build time

This buildpack performs the following actions at build time:

• Contributes cpython to a layer - either via a precompiled dependency or by compiling from source.
  • Python is precompiled for recognized, supported stacks (currently the Ubuntu-based stacks)
  • Python is compiled from source during the build phase for unrecognized stacks.
  • dependency/ contains more information how cpython is built.
• Sets the PYTHONPYCACHEPREFIX environment variable to the /tmp directory for this and any subsequent buildpacks.
  • This effectively disables the __pycache__ directories, in turn enabling reproducible builds.
  • It can be overridden via standard means (e.g. pack build --env "PYTHONPYCACHEPREFIX=/some/other/dir") if there is a need to keep the __pycache__ directories in place of reproducible builds.
• Sets the PYTHONPATH to the cpython layer path for this and any subsequent buildpacks.
• Adds the newly installed cpython bin dir location to the PATH.

Launch time

This buildpack does the following at launch time:

• Sets a default value for the PYTHONPATH environment variable.

Configuration

BP_CPYTHON_VERSION

The BP_CPYTHON_VERSION variable allows you to specify the version of CPython that is installed. (Available versions can be found in the buildpack.toml.)

pack build flag

pack build my-app --env BP_CPYTHON_VERSION=3.10.*

In a project.toml

[build]
  [[build.env]]
    name = 'BP_CPYTHON_VERSION'
    value = '3.10.*' # any valid semver constraints (e.g. 3.10.2, 3.*) are acceptable

BP_CPYTHON_CONFIGURE_FLAGS

The BP_CPYTHON_CONFIGURE_FLAGS variable allows you to specify configure flags when python is installed from source. This is only applicable when using custom stacks. Paketo stacks such as io.buildpacks.stacks.bionic install pre-built binaries.

• The format is space-separated strings, and they are passed directly to the cpython ./configure process , e.g. --foo --bar=baz.
• See python documentation for supported flags.
• Default flags if not specified: --with-ensurepip
• Note that default flags are overridden if you specify this environment variable, which means you almost certainly want to include the defaults along with any custom flags.
  • e.g. --with-ensurepip --foo --bar=baz

BP_LOG_LEVEL

The BP_LOG_LEVEL flag controls the level of verbosity of the buildpack output logs. For more detail, set it to debug.

For example, when compiling from source, setting BP_LOG_LEVEL=debug shows the commands and outputs run to build python.

Integration

The CPython Buildpack provides cpython as a dependency. Downstream buildpacks, can require the cpython dependency by generating Build Plan TOML file that looks like the following:

[[requires]]
  # The name of the CPython dependency is "cpython". This value is considered
  # part of the public API for the buildpack and will not change without a plan
  # for deprecation.
  name = "cpython"

  # The CPython buildpack supports some non-required metadata options.
  [requires.metadata]

    # The version of the CPython dependency is not required. In the case it
    # is not specified, the buildpack will provide the default version, which can
    # be seen in the buildpack.toml file.
    # If you wish to request a specific version, the buildpack supports
    # specifying a semver constraint in the form of "3.*", "3.10.*", or even
    # "3.10.2".
    version = "3.10.2"

    # Setting the build flag to true will ensure that the CPython
    # depdendency is available on the $PATH for subsequent buildpacks during
    # their build phase. If you are writing a buildpack that needs to use CPython
    # during its build process, this flag should be set to true.
    build = true

    # Setting the launch flag to true will ensure that cpython is
    # available to the running application. If you are writing an application
    # that needs to run cpython at runtime, this flag should be set to true.
    launch = true

Usage

To package this buildpack for consumption:

$ ./scripts/package.sh --version <version-number>

This will create a buildpackage.cnb file under the build directory which you can use to build your app as follows: pack build <app-name> -p <path-to-app> -b build/buildpackage.cnb -b <other-buildpacks..>

To run the unit and integration tests for this buildpack:

$ ./scripts/unit.sh && ./scripts/integration.sh

Compatibility

This buildpack is currently only supported on linux distributions.

Pre-compiled distributions of Python are provided for the Paketo stacks (i.e. io.buildpacks.stack.jammy and io.buildpacks.stacks.bionic).

Source distributions of Python are provided for all other linux stacks.

Documentation

Index

• Constants
• Variables
• func Build(dependencies DependencyManager, pythonInstaller PythonInstaller, ...) packit.BuildFunc
• func Detect() packit.DetectFunc
• type BuildPlanMetadata
• type CPythonInstaller
  • func NewCPythonInstaller(configureProcess Executable, makeProcess Executable, logger scribe.Emitter) CPythonInstaller
  • func (i CPythonInstaller) Install(sourcePath string, workingDir string, entry packit.BuildpackPlanEntry, ...) error
• type DependencyManager
• type Executable
• type PipCleanup
  • func NewPipCleanup(pythonProcess Executable, logger scribe.Emitter) PipCleanup
  • func (i PipCleanup) Cleanup(packages []string, targetLayer string) error
• type PythonInstaller
• type PythonPipCleanup
• type SBOMGenerator
• type VersionParser

Constants

const Cpython = "cpython"

Cpython is the name of the layer into which cpython dependency is installed.

const DepKey = "dependency-sha"

DepKey is the key in the Layer Content Metadata used to determine if layer can be reused.

Variables

var Priorities = []interface{}{
	"BP_CPYTHON_VERSION",
	"Pipfile.lock",
	"Pipfile",
	"pyproject.toml",
}

Priorities is a list of version-source values that may appear in the BuildpackPlan entries that the buildpack receives. The list is from highest priority to lowest priority and determines the precedence of version-sources.

Functions

func Build

func Build(
	dependencies DependencyManager,
	pythonInstaller PythonInstaller,
	pipCleanup PythonPipCleanup,
	sbomGenerator SBOMGenerator,
	logger scribe.Emitter,
	clock chronos.Clock,
) packit.BuildFunc

Build will return a packit.BuildFunc that will be invoked during the build phase of the buildpack lifecycle.

Build will find the right cpython dependency to install, install it in a layer, and generate Bill-of-Materials. It also makes use of the checksum of the dependency to reuse the layer when possible.

func Detect

func Detect() packit.DetectFunc

Detect will return a packit.DetectFunc that will be invoked during the detect phase of the buildpack lifecycle.

Detect always passes, and will contribute a Build Plan that provides cpython.

Types

type BuildPlanMetadata

type BuildPlanMetadata struct {
	// Version denotes the version constraint to be requested in the requirements.
	Version string `toml:"version"`

	// VersionSource denotes the source of the version information. This may be
	// used by the consumer of the metadata to determine the priority of this
	// version request.
	VersionSource string `toml:"version-source"`

	// ConfigureFlags denotes the configure flags to be requested in the requirements.
	// This is used to run configure before make and make install.
	ConfigureFlags string `toml:"configure-flags"`
}

BuildPlanMetadata is the buildpack specific data included in build plan requirements.

type CPythonInstaller

type CPythonInstaller struct {
	// contains filtered or unexported fields
}

CPythonInstaller implements the PythonInstaller interface.

func NewCPythonInstaller

func NewCPythonInstaller(
	configureProcess Executable,
	makeProcess Executable,
	logger scribe.Emitter,
) CPythonInstaller

NewCPythonInstaller creates an instance of the CPythonInstaller given a scribe.Emitter.

func (CPythonInstaller) Install

func (i CPythonInstaller) Install(
	sourcePath string,
	workingDir string,
	entry packit.BuildpackPlanEntry,
	dependency postal.Dependency,
	layerPath string,
) error

Installs python from source code located in the given sourcePath into the layer path designated by layerPath.

type DependencyManager

type DependencyManager interface {
	Resolve(path, id, version, stack string) (postal.Dependency, error)
	Deliver(dependency postal.Dependency, cnbPath, destinationPath, platformPath string) error
	GenerateBillOfMaterials(dependencies ...postal.Dependency) []packit.BOMEntry
}

DependencyManager defines the interface for picking the best matching dependency installing it, and generating a BOM.

type Executable

type Executable interface {
	Execute(pexec.Execution) error
}

Executable defines the interface for invoking an executable.

type PipCleanup

type PipCleanup struct {
	// contains filtered or unexported fields
}

PipCleanup implements the PythonPipCleanup interface.

func NewPipCleanup

func NewPipCleanup(pythonProcess Executable, logger scribe.Emitter) PipCleanup

NewPipCleanup creates an instance of PipCleanup given a python Executable and a scribe.Emitter.

func (PipCleanup) Cleanup

func (i PipCleanup) Cleanup(packages []string, targetLayer string) error

type PythonInstaller

type PythonInstaller interface {
	Install(
		sourcePath string,
		workingDir string,
		entry packit.BuildpackPlanEntry,
		dependency postal.Dependency,
		layerPath string,
	) error
}

PythonInstaller defines the interface for installing python from source

type PythonPipCleanup

type PythonPipCleanup interface {
	Cleanup(packages []string, targetLayer string) error
}

PythonPipCleanup defines the interface for cleaning up pip after a python installation

type SBOMGenerator

type SBOMGenerator interface {
	GenerateFromDependency(dependency postal.Dependency, dir string) (sbom.SBOM, error)
}

type VersionParser

type VersionParser interface {
	ParseVersion(path string) (version string, err error)
}

VersionParser defines the interface for determining the version of Cpython.