tofudl

package module
v0.0.0-...-d4254f2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 29, 2025 License: MPL-2.0 Imports: 24 Imported by: 1

README

TofuDL: OpenTofu downloader library for Go with minimal dependencies

This library provides an easy way to download, verify, and unpack OpenTofu binaries for local use in Go. It has a minimal set of dependencies and is easy to integrate.

Note: This is not a standalone tool to download OpenTofu, it is purely meant to be used as Go library in support of other tools that need to run tofu. Please check the installation instructions for supported ways to perform an OpenTofu installation.

Basic usage

The downloader will work without any extra configuration out of the box:

package main

import (
    "context"
    "os"
    "os/exec"
    "runtime"

    "github.com/opentofu/tofudl"
)

func main() {
    // Initialize the downloader:
    dl, err := tofudl.New()
    if err != nil {
        panic(err)
    }

    // Download the latest stable version
    // for the current architecture and platform:
    binary, err := dl.Download(context.TODO())
    if err != nil {
        panic(err)
    }

    // Write out the tofu binary to the disk:
    file := "tofu"
    if runtime.GOOS == "windows" {
        file += ".exe"
    }
    if err := os.WriteFile(file, binary, 0755); err != nil {
        panic(err)
    }

    // Run tofu:
    cmd := exec.Command("./"+file, "init")
    if err := cmd.Run(); err != nil {
        panic(err)
    }
}

Caching

This library also supports caching using the mirror tool:

package main

import (
    "context"
    "os"
    "os/exec"
    "runtime"
    "time"

    "github.com/opentofu/tofudl"
)

func main() {
    // Initialize the downloader:
    dl, err := tofudl.New()
    if err != nil {
        panic(err)
    }

    // Set up the caching layer:
    storage, err := tofudl.NewFilesystemStorage("/tmp")
    if err != nil {
        panic(err)
    }
    mirror, err := tofudl.NewMirror(
        tofudl.MirrorConfig{
            AllowStale: false,
            APICacheTimeout: time.Minute * 10,
            ArtifactCacheTimeout: time.Hour * 24,
        },
        storage,
        dl,
    )
    if err != nil {
        panic(err)
    }

    // Download the latest stable version
    // for the current architecture and platform:
    binary, err := mirror.Download(context.TODO())
    if err != nil {
        panic(err)
    }

    // Write out the tofu binary to the disk:
    file := "tofu"
    if runtime.GOOS == "windows" {
        file += ".exe"
    }
    if err := os.WriteFile(file, binary, 0755); err != nil {
        panic(err)
    }

    // Run tofu:
    cmd := exec.Command("./"+file, "init")
    if err := cmd.Run(); err != nil {
        panic(err)
    }
}

You can also use the mirror variable as an http.Handler. Additionally, you can also call PreWarm on the caching layer in order to pre-warm your local caches. (Be careful, this may take a long time!)

Standalone mirror

The example above showed a cache/mirror that acts as a pull-through cache to upstream. You can alternatively also use the mirror as a stand-alone mirror and publish your own binaries. The mirror has functions to facilitate uploading basic artifacts, but you can also use the ReleaseBuilder to make building releases easier. (Note: the ReleaseBuilder only builds artifacts needed for TofuDL, not all artifacts OpenTofu typically publishes.)

Advanced usage

Both New() and Download() accept a number of options. You can find the detailed documentation here.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type APIResponse 

type APIResponse struct {
	// Versions is the list of versions from the API.
	Versions []VersionWithArtifacts `json:"versions"`
}

APIResponse is the JSON response from the API URL.

type Architecture 

type Architecture string

Architecture describes the architecture to download OpenTofu for. It defaults to the current system architecture. 

const (
	// ArchitectureAuto is the default value and defaults to downloading OpenTofu for the current architecture.
	ArchitectureAuto Architecture = ""
	// Architecture386 describes the 32-bit Intel CPU architecture.
	Architecture386 Architecture = "386"
	// ArchitectureAMD64 describes the 64-bit Intel/AMD CPU architecture.
	ArchitectureAMD64 Architecture = "amd64"
	// ArchitectureARM describes the 32-bit ARM (v7) architecture.
	ArchitectureARM Architecture = "arm"
	// ArchitectureARM64 describes the 64-bit ARM (v8) architecture.
	ArchitectureARM64 Architecture = "arm64"
)

func ArchitectureValues 

func ArchitectureValues() []Architecture

ArchitectureValues returns all supported values for Architecture excluding ArchitectureAuto.

func (Architecture) ResolveAuto 

func (a Architecture) ResolveAuto() (Architecture, error)

ResolveAuto resolves the value of ArchitectureAuto if needed based on the current runtime.GOARCH.

func (Architecture) Validate 

func (a Architecture) Validate() error

Validate returns an error if the platform is not a valid platform descriptor.

type ArtifactCorruptedError 

type ArtifactCorruptedError struct {
	Artifact string
	Cause    error
}

ArtifactCorruptedError indicates that the downloaded artifact is corrupt.

func (ArtifactCorruptedError) Error 

func (e ArtifactCorruptedError) Error() string

Error returns the error message.

func (ArtifactCorruptedError) Unwrap 

func (e ArtifactCorruptedError) Unwrap() error

Unwrap returns the original error.

type CacheMissError 

type CacheMissError struct {
	File  string
	Cause error
}

CacheMissError indicates that the artifact or file is not cached.

func (CacheMissError) Error 

func (e CacheMissError) Error() string

Error returns the error message.

func (CacheMissError) Unwrap 

func (e CacheMissError) Unwrap() error

Unwrap returns the original error.

type CachedAPIResponseStaleError 

type CachedAPIResponseStaleError struct {
}

CachedAPIResponseStaleError indicates that the API response is cached, but stale.

func (CachedAPIResponseStaleError) Error 

func (e CachedAPIResponseStaleError) Error() string

Error returns the error message.

type CachedArtifactStaleError 

type CachedArtifactStaleError struct {
	Version  Version
	Artifact string
}

CachedArtifactStaleError indicates that the file is cached, but stale.

func (CachedArtifactStaleError) Error 

func (e CachedArtifactStaleError) Error() string

Error returns the error message.

type Config 

type Config struct {
	// GPGKey holds the ASCII-armored GPG key to verify the binaries against. Defaults to the bundled
	// signing key.
	GPGKey string
	// APIURL describes the URL to the JSON API listing the versions and artifacts. Defaults to branding.DownloadAPIURL.
	APIURL string
	// APIURLAuthorization is an optional Authorization header to add to all request to the API URL. For requests
	// to the default API URL leave this empty.
	APIURLAuthorization string
	// DownloadMirrorAuthorization is an optional Authorization header to add to all requests to the download mirror.
	// Typically, you'll want to set this to "Bearer YOUR-GITHUB-TOKEN".
	DownloadMirrorAuthorization string
	// DownloadMirrorURLTemplate is a Go text template containing a URL with MirrorURLTemplateParameters embedded to
	// generate the download URL. Defaults to branding.DefaultMirrorURLTemplate.
	DownloadMirrorURLTemplate string
	// HTTPClient holds an HTTP client to use for requests. Defaults to the standard HTTP client with hardened TLS
	// settings.
	HTTPClient *http.Client
}

Config describes the base configuration for the downloader.

func (*Config) ApplyDefaults 

func (c *Config) ApplyDefaults()

ApplyDefaults applies defaults for all fields that are not set.

type ConfigOpt 

type ConfigOpt func(config *Config) error

ConfigOpt is a function that modifies the config.

func ConfigAPIAuthorization 

func ConfigAPIAuthorization(authorization string) ConfigOpt

ConfigAPIAuthorization adds an authorization header to any request sent to the API server. This is not needed for the default API, but may be needed for private mirrors.

func ConfigAPIURL 

func ConfigAPIURL(url string) ConfigOpt

ConfigAPIURL adds an API URL for the version listing. Defaults to branding.DownloadAPIURL.

func ConfigDownloadMirrorAuthorization 

func ConfigDownloadMirrorAuthorization(authorization string) ConfigOpt

ConfigDownloadMirrorAuthorization adds the specified value to any request when connecting the download mirror. For example, you can add your GitHub token by specifying "Bearer YOUR-TOKEN-HERE".

func ConfigDownloadMirrorURLTemplate 

func ConfigDownloadMirrorURLTemplate(urlTemplate string) ConfigOpt

ConfigDownloadMirrorURLTemplate adds a Go text template containing a URL with MirrorURLTemplateParameters embedded to generate the download URL. Defaults to branding.DefaultMirrorURLTemplate.

func ConfigGPGKey 

func ConfigGPGKey(gpgKey string) ConfigOpt

ConfigGPGKey is a config option to set an ASCII-armored GPG key.

func ConfigHTTPClient 

func ConfigHTTPClient(client *http.Client) ConfigOpt

ConfigHTTPClient adds a customized HTTP client to the downloader.

type DownloadOpt 

type DownloadOpt func(spec *DownloadOptions) error

DownloadOpt is a function that modifies the download options.

func DownloadOptArchitecture 

func DownloadOptArchitecture(architecture Architecture) DownloadOpt

DownloadOptArchitecture specifies the architecture to download for. This defaults to the current architecture.

func DownloadOptMinimumStability 

func DownloadOptMinimumStability(stability Stability) DownloadOpt

DownloadOptMinimumStability specifies the minimum stability of the version to download. This is mutually exclusive with setting the Version.

func DownloadOptPlatform 

func DownloadOptPlatform(platform Platform) DownloadOpt

DownloadOptPlatform specifies the platform to download for. This defaults to the current platform.

func DownloadOptVersion 

func DownloadOptVersion(version Version) DownloadOpt

DownloadOptVersion specifies the version to download. Defaults to the latest version with the specified minimum stability.

type DownloadOptions 

type DownloadOptions struct {
	Platform         Platform
	Architecture     Architecture
	Version          Version
	MinimumStability *Stability
}

DownloadOptions describes the settings for downloading. They default to the current architecture and platform.

type Downloader 

type Downloader interface {
	// ListVersions lists all versions matching the filter options in descending order.
	ListVersions(ctx context.Context, opts ...ListVersionOpt) ([]VersionWithArtifacts, error)

	// DownloadArtifact downloads an artifact for a version.
	DownloadArtifact(ctx context.Context, version VersionWithArtifacts, artifactName string) ([]byte, error)

	// VerifyArtifact verifies a named artifact against a checksum file with SHA256 hashes and the checksum file against a GPG signature file.
	VerifyArtifact(artifactName string, artifactContents []byte, sumsFileContents []byte, signatureFileContent []byte) error

	// DownloadVersion downloads the OpenTofu binary from a specific artifact obtained from ListVersions.
	DownloadVersion(ctx context.Context, version VersionWithArtifacts, platform Platform, architecture Architecture) ([]byte, error)

	// Download downloads the OpenTofu binary and provides it as a byte slice.
	Download(ctx context.Context, opts ...DownloadOpt) ([]byte, error)
}

Downloader describes the functions the downloader provides.

func New 

func New(opts ...ConfigOpt) (Downloader, error)

type InvalidArchitectureError 

type InvalidArchitectureError struct {
	Architecture Architecture
}

InvalidArchitectureError describes an error where an architecture name was found to be invalid.

func (InvalidArchitectureError) Error 

func (e InvalidArchitectureError) Error() string

Error returns the error message.

type InvalidConfigurationError 

type InvalidConfigurationError struct {
	Message string
	Cause   error
}

InvalidConfigurationError indicates that the base configuration for the downloader is invalid.

func (InvalidConfigurationError) Error 

func (e InvalidConfigurationError) Error() string

Error returns the error message.

func (InvalidConfigurationError) Unwrap 

func (e InvalidConfigurationError) Unwrap() error

type InvalidOptionsError 

type InvalidOptionsError struct {
	Cause error
}

InvalidOptionsError indicates that the request options are invalid.

func (InvalidOptionsError) Error 

func (e InvalidOptionsError) Error() string

Error returns the error message.

func (InvalidOptionsError) Unwrap 

func (e InvalidOptionsError) Unwrap() error

Unwrap returns the original error.

type InvalidPlatformError 

type InvalidPlatformError struct {
	Platform Platform
}

InvalidPlatformError describes an error where a platform name was found to be invalid.

func (InvalidPlatformError) Error 

func (e InvalidPlatformError) Error() string

Error returns the error message.

type InvalidVersionError 

type InvalidVersionError struct {
	Version Version
}

InvalidVersionError describes an error where the version string is invalid.

func (InvalidVersionError) Error 

func (e InvalidVersionError) Error() string

Error returns the error message.

type ListVersionOpt 

type ListVersionOpt func(options *ListVersionsOptions) error

ListVersionOpt is an option for the ListVersions call.

func ListVersionOptMinimumStability 

func ListVersionOptMinimumStability(stability Stability) ListVersionOpt

ListVersionOptMinimumStability sets the minimum stability for listing versions.

type ListVersionsOptions 

type ListVersionsOptions struct {
	Stability *Stability
}

ListVersionsOptions are the options for listing versions.

type Mirror 

type Mirror interface {
	Downloader
	http.Handler

	// PreWarm downloads the last specified number of versions into the storage directory from the pull-through
	// downloader if present. If versions is negative, all versions are downloaded. Note: the versions include alpha,
	// beta and release candidate versions. Make sure you pre-warm with enough versions for your use case.
	//
	// If no pull-through downloader is configured, this function does not do anything.
	PreWarm(ctx context.Context, versionCount int, progress func(pct int8)) error

	// CreateVersion creates a new version in the cache, adding it to the version index. Note that this is not supported
	// when working in pull-through cache mode.
	CreateVersion(ctx context.Context, version Version) error

	// CreateVersionAsset creates a new asset for a version, storing it in the storage and adding it to the version
	// list. Note that this is not supported when working in pull-through cache mode.
	CreateVersionAsset(ctx context.Context, version Version, assetName string, assetData []byte) error
}

Mirror is a downloader that caches artifacts. It also supports pre-warming caches by calling the PreWarm function. You can use this as a handler for an HTTP server in order to act as a mirror to a regular Downloader.

func NewMirror 

func NewMirror(config MirrorConfig, storage MirrorStorage, pullThroughDownloader Downloader) (Mirror, error)

NewMirror creates a new mirror, optionally acting as a pull-through cache when passing a pullThroughDownloader.

type MirrorConfig 

type MirrorConfig struct {
	// AllowStale enables using stale cached resources if the download fails.
	AllowStale bool `json:"allow_stale"`
	// APICacheTimeout is the time the cached API JSON should be considered valid. A duration of 0 means the API
	// responses should not be cached. A duration of -1 means the API responses should be cached indefinitely.
	APICacheTimeout time.Duration `json:"api_cache_timeout"`
	// ArtifactCacheTimeout is the time the cached artifacts should be considered valid. A duration of 0 means that
	// artifacts should not be cached. A duration of -1 means that artifacts should be cached indefinitely.
	ArtifactCacheTimeout time.Duration `json:"artifact_cache_timeout"`

	// GPGKey is the ASCII-armored key to verify downloaded artifacts against. This is only needed in standalone mode.
	GPGKey string `json:"gpg_key"`
}

MirrorConfig is the configuration structure for the caching downloader.

type MirrorStorage 

type MirrorStorage interface {
	// ReadAPIFile reads the API file cache and returns a reader for it. It also returns the time when the cached
	// response was written. It will return a CacheMissError if the API response is not cached.
	ReadAPIFile() (io.ReadCloser, time.Time, error)
	// StoreAPIFile stores the API file in the cache.
	StoreAPIFile(apiFile []byte) error

	// ReadArtifact reads a binary artifact from the cache for a specific version and returns a reader to it.
	// It also returns the time the artifact was stored as the second parameter. It will return a CacheMissError if
	// there is no such artifact in the cache.
	ReadArtifact(version Version, artifactName string) (io.ReadCloser, time.Time, error)
	// StoreArtifact stores a binary artifact in the cache for a specific version.
	StoreArtifact(version Version, artifactName string, contents []byte) error
}

MirrorStorage is responsible for handling the low-level storage of caches.

func NewFilesystemStorage 

func NewFilesystemStorage(directory string) (MirrorStorage, error)

NewFilesystemStorage returns a mirror storage that relies on files and modification timestamps. This storage can also be used to mirror the artifacts for air-gapped usage. The filesystem layout is as follows:

- api.json - v1.2.3/artifact.name

Note: when used as a pull-through cache, the underlying filesystem must support modification timestamps or the cache timeout must be set to -1 to prevent the mirror from re-fetching every time.

type MirrorURLTemplateParameters 

type MirrorURLTemplateParameters struct {
	Version  Version
	Artifact string
}

MirrorURLTemplateParameters describes the parameters to a URL template for mirrors.

type NoSuchArtifactError 

type NoSuchArtifactError struct {
	ArtifactName string
}

NoSuchArtifactError indicates that there is no artifact for the given version with the given name.

func (NoSuchArtifactError) Error 

func (e NoSuchArtifactError) Error() string

Error returns the error message.

type NoSuchVersionError 

type NoSuchVersionError struct {
	Version Version
}

NoSuchVersionError indicates that the given version does not exist on the API endpoint.

func (NoSuchVersionError) Error 

func (e NoSuchVersionError) Error() string

Error returns the error message.

type Platform 

type Platform string

Platform describes the operating system to download OpenTofu for. Defaults to the current operating system. 

const (
	// PlatformAuto is the default value and describes the current operating system.
	PlatformAuto Platform = ""
	// PlatformWindows describes the Windows platform.
	PlatformWindows Platform = "windows"
	// PlatformLinux describes the Linux platform.
	PlatformLinux Platform = "linux"
	// PlatformMacOS describes the macOS (Darwin) platform.
	PlatformMacOS Platform = "darwin"
	// PlatformSolaris describes the Solaris platform. (Note: this is currently only supported on AMD64.)
	PlatformSolaris Platform = "solaris"
	// PlatformOpenBSD describes the OpenBSD platform. (Note: this is currently only supported on 386 and AMD64.)
	PlatformOpenBSD Platform = "openbsd"
	// PlatformFreeBSD describes the FreeBSD platform. (Note: this is currently not supported on ARM64)
	PlatformFreeBSD Platform = "freebsd"
)

func PlatformValues 

func PlatformValues() []Platform

PlatformValues returns all supported values for Platform excluding PlatformAuto.

func (Platform) ResolveAuto 

func (p Platform) ResolveAuto() (Platform, error)

ResolveAuto resolves the value of PlatformAuto if needed based on the current runtime.GOOS.

func (Platform) Validate 

func (p Platform) Validate() error

Validate returns an error if the platform is not a valid platform descriptor.

type ReleaseBuilder 

type ReleaseBuilder interface {
	// PackageBinary creates a .tar.gz file for the specific platform and architecture based on the binary contents.
	// You may pass extra files to package, such as LICENSE, etc. in extraFiles.
	PackageBinary(platform Platform, architecture Architecture, contents []byte, extraFiles map[string][]byte) error

	// AddArtifact adds an artifact to the release, adds it to the checksum file and signs the checksum file.
	AddArtifact(artifactName string, data []byte) error

	// Build builds the release and adds it to the specified mirror. Note that the ReleaseBuilder should not be
	// reused after calling Build.
	Build(ctx context.Context, version Version, mirror Mirror) error
}

ReleaseBuilder is a tool to build a release and add it to a mirror. Note that this does not (yet) produce a full release suitable for end users as this does not support signing with cosign and does not produce other artifacts.

func NewReleaseBuilder 

func NewReleaseBuilder(gpgKey *crypto.Key) (ReleaseBuilder, error)

NewReleaseBuilder creates a new ReleaseBuilder with the given gpgKey to sign the release.

type RequestFailedError 

type RequestFailedError struct {
	Cause error
}

RequestFailedError indicates that a request to an API or the download mirror failed.

func (RequestFailedError) Error 

func (e RequestFailedError) Error() string

Error returns the error message.

func (RequestFailedError) Unwrap 

func (e RequestFailedError) Unwrap() error

Unwrap returns the original error.

type SignatureError 

type SignatureError struct {
	Message string
	Cause   error
}

SignatureError indicates that the signature verification failed.

func (SignatureError) Error 

func (e SignatureError) Error() string

Error returns the error message.

func (SignatureError) Unwrap 

func (e SignatureError) Unwrap() error

type Stability 

type Stability string

Stability describes the minimum stability to download. 

const (
	// StabilityAlpha accepts any stability.
	StabilityAlpha Stability = "alpha"
	// StabilityBeta accepts beta, release candidate and stable versions.
	StabilityBeta Stability = "beta"
	// StabilityRC accepts release candidate and stable versions.
	StabilityRC Stability = "rc"
	// StabilityStable accepts only stable versions.
	StabilityStable Stability = ""
)

func StabilityValues 

func StabilityValues() []Stability

StabilityValues returns all supported values for Stability excluding StabilityStable.

func (Stability) AsInt 

func (s Stability) AsInt() int

AsInt returns a numeric representation of the stability for easier comparison.

func (Stability) Matches 

func (s Stability) Matches(version Version) bool

Matches returns true if the provided version matches the current stability or higher.

func (Stability) Validate 

func (s Stability) Validate() error

Validate returns an error if the stability is not one of the listed stabilities.

type UnsupportedArchitectureError 

type UnsupportedArchitectureError struct {
	Architecture Architecture
}

UnsupportedArchitectureError indicates that the given runtime.GOARCH architecture is not supported and cannot automatically resolve to a build artifact.

func (UnsupportedArchitectureError) Error 

func (e UnsupportedArchitectureError) Error() string

Error returns the error message.

type UnsupportedPlatformError 

type UnsupportedPlatformError struct {
	Platform Platform
}

UnsupportedPlatformError indicates that the given runtime.GOOS platform is not supported and cannot automatically resolve to a build artifact.

func (UnsupportedPlatformError) Error 

func (e UnsupportedPlatformError) Error() string

Error returns the error message.

type UnsupportedPlatformOrArchitectureError 

type UnsupportedPlatformOrArchitectureError struct {
	Platform     Platform
	Architecture Architecture
	Version      Version
}

UnsupportedPlatformOrArchitectureError describes an error where the platform name and architecture are syntactically valid, but no release artifact was found matching that name.

func (UnsupportedPlatformOrArchitectureError) Error 

func (e UnsupportedPlatformOrArchitectureError) Error() string

type Version 

type Version string

Version describes a version number with this project's version and stability understanding.

func (Version) Compare 

func (v Version) Compare(other Version) int

Compare returns 1 if the current version is larger than the other, -1 if it is smaller, 0 otherwise.

func (Version) Major 

func (v Version) Major() int

Major returns the major version. The version must be valid or this function will panic.

func (Version) Minor 

func (v Version) Minor() int

Minor returns the minor version. The version must be valid or this function will panic.

func (Version) Patch 

func (v Version) Patch() int

Patch returns the patch version. The version must be valid or this function will panic.

func (Version) Stability 

func (v Version) Stability() Stability

Stability returns the stability string for the version. The version must be valid or this function will panic.

func (Version) StabilityVer 

func (v Version) StabilityVer() int

StabilityVer returns the stability version number for the version. The version must be valid or this function will panic.

func (Version) Validate 

func (v Version) Validate() error

Validate checks if the version is valid

type VersionWithArtifacts 

type VersionWithArtifacts struct {
	ID    Version  `json:"id"`
	Files []string `json:"files"`
}

VersionWithArtifacts is a version and the list of artifacts belonging to that version.

Source Files

View all Source files

Directories

Path Synopsis
Package cli is a demonstration how a CLI downloader can be implemented with this library.
 Package cli is a demonstration how a CLI downloader can be implemented with this library.
cmd
tofudl
internal
helloworld
tools/bundle-gpg-key
tools/license-headers
tools/lint

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.