retrieve

Documentation

Overview

Package retrieve assists buildpack authors to create the retrieve step as outlined in the Dependency RFCs, especially `Dependency Management Phase 2: Workflow and Github Action Generalization RFC`.

Buildpacks should call retrieve.NewMetadata from the `main` func of their retrieval step. See the godoc for that function for additional information.

Index

• Variables
• func GenerateAllMetadata(newVersions versionology.VersionFetcherArray, ...) []versionology.Dependency
• func GeneratePURL(id, version, checksum, source string) string
• func GetNewVersionsForId(id string, config cargo.Config, getAllVersions GetAllVersionsFunc) (versionology.VersionFetcherArray, error)
• func LookupLicenses(sourceURL string, f DecompressArtifactFunc) []interface{}
• func NewMetadata(id string, getAllVersions GetAllVersionsFunc, ...)
• type DecompressArtifactFunc
• type FetchArgsFunc
• type GenerateMetadataFunc
• type GetAllVersionsFunc

Examples

• GenerateAllMetadata
• GetNewVersionsForId

Variables

var FetchArgs = func() (buildpackTomlPath, output string) {
	buildpackTomlPathUsage := "full path to the buildpack.toml file, using only one of camelCase, snake_case, or dash_case"

	flag.StringVar(&buildpackTomlPath, "buildpackTomlPath", "", buildpackTomlPathUsage)
	flag.StringVar(&buildpackTomlPath, "buildpack_toml_path", buildpackTomlPath, buildpackTomlPathUsage)
	flag.StringVar(&buildpackTomlPath, "buildpack-toml-path", buildpackTomlPath, buildpackTomlPathUsage)
	flag.StringVar(&output, "output", "", "filename for the output JSON metadata")
	flag.Parse()
	return
}

FetchArgs is public for testing purposes

Functions

func GenerateAllMetadata

func GenerateAllMetadata(newVersions versionology.VersionFetcherArray, generateMetadata GenerateMetadataFunc) []versionology.Dependency

GenerateAllMetadata is public for testing purposes only

Example

package main

import (
	"github.com/joshuatcasey/libdependency/retrieve"
	"github.com/joshuatcasey/libdependency/versionology"
	"github.com/paketo-buildpacks/packit/v2/cargo"
)

func main() {

	versions, _ := versionology.NewSimpleVersionFetcherArray("1.2.3", "4.5.6", "7.8.9")

	generateMetadata := func(version versionology.VersionFetcher) ([]versionology.Dependency, error) {
		dep := cargo.ConfigMetadataDependency{ID: "dep-id", Version: version.Version().String()}

		switch version.Version().String() {
		case "1.2.3":
			return versionology.NewDependencyArray(dep, "target1")
		case "4.5.6":
			dep1, _ := versionology.NewDependency(dep, "target1")
			dep2, _ := versionology.NewDependency(dep, "target2")
			return []versionology.Dependency{dep1, dep2}, nil
		case "7.8.9":
			dep1, _ := versionology.NewDependency(dep, "target1")
			dep2, _ := versionology.NewDependency(dep, "target2")
			dep3, _ := versionology.NewDependency(dep, "target3")
			return []versionology.Dependency{dep1, dep2, dep3}, nil
		default:
			panic("unknown version")
		}
	}

	retrieve.GenerateAllMetadata(versions, generateMetadata)

}

Output:

Generating metadata for 1.2.3, with targets [target1]
Generating metadata for 4.5.6, with targets [target1, target2]
Generating metadata for 7.8.9, with targets [target1, target2, target3]

func GeneratePURL

func GeneratePURL(id, version, checksum, source string) string

GeneratePURL can be used to populate the `purl` field of dependency metadata PURL stands for package URL. https://github.com/package-url/purl-spec

func GetNewVersionsForId

func GetNewVersionsForId(id string, config cargo.Config, getAllVersions GetAllVersionsFunc) (versionology.VersionFetcherArray, error)

GetNewVersionsForId will return only those versions with the following properties: - returned by getAllVersions - match constraints - newer than all existing dependencies

Example

config, _ := buildpack_config.ParseBuildpackToml(filepath.Join("..", "retrieve", "testdata", "happy_path", "buildpack.toml"))
getAllVersions := func() (versionology.VersionFetcherArray, error) {
	return versionology.VersionFetcherArray{
		versionology.NewSimpleVersionFetcher(semver.MustParse("1.0.0")),
		versionology.NewSimpleVersionFetcher(semver.MustParse("1.1.0")),
		versionology.NewSimpleVersionFetcher(semver.MustParse("1.2.0")),
		versionology.NewSimpleVersionFetcher(semver.MustParse("1.3.0")),
		versionology.NewSimpleVersionFetcher(semver.MustParse("1.4.0")),
		versionology.NewSimpleVersionFetcher(semver.MustParse("1.5.0")),
	}, nil
}
_, _ = retrieve.GetNewVersionsForId("fake-dependency-id", config, getAllVersions)

Output:

Found 6 versions of fake-dependency-id from upstream
[
  "1.5.0", "1.4.0", "1.3.0", "1.2.0", "1.1.0",
  "1.0.0"
]
Found 6 versions of fake-dependency-id for constraint 1.*.*
[
  "1.5.0", "1.4.0", "1.3.0", "1.2.0", "1.1.0",
  "1.0.0"
]
Found 2 versions of fake-dependency-id newer than '1.1.0' for constraint 1.*.*, after limiting for 2 patches
[
  "1.5.0", "1.4.0"
]
Found 2 versions of fake-dependency-id as new versions
[
  "1.5.0", "1.4.0"
]

func LookupLicenses

func LookupLicenses(sourceURL string, f DecompressArtifactFunc) []interface{}

LookupLicenses uses licensedb to detect licenses contained within a compressed directory

func NewMetadata

func NewMetadata(id string, getAllVersions GetAllVersionsFunc, generateMetadata GenerateMetadataFunc)

NewMetadata is the entrypoint for a buildpack to retrieve new versions and the metadata thereof. Given a way to retrieve all versions (getNewVersions) and a way to generate metadata for a version (generateMetadata), this function will take in the dependency workflow inputs and the dependency workflow outputs

Types

type DecompressArtifactFunc

type DecompressArtifactFunc func(artifact io.Reader, destination string) error

type FetchArgsFunc

type FetchArgsFunc func() (string, string)

type GenerateMetadataFunc

type GenerateMetadataFunc func(version versionology.VersionFetcher) ([]versionology.Dependency, error)

GenerateMetadataFunc is a function type that buildpack authors will implement and pass in to NewMetadata. Given a versionology.VersionFetcher, the implementation must return the associated metadata for that version. If there are multiple targets for the same version, return multiple versionology.Dependency.

type GetAllVersionsFunc

type GetAllVersionsFunc func() (versionology.VersionFetcherArray, error)

GetAllVersionsFunc is a function type that buildpack authors will implement and pass in to NewMetadata. The implementation should return all known upstream versions of a dependency. Buildpack authors can choose the source of these versions. Some examples include:

- `nginx` versions from https://github.com/nginx/nginx/tags - `bundler` versions from https://rubygems.org/api/v1/versions/bundler.json