forge

README

forge

Forge is a library and CLI for running reusable steps from various proprietary CI systems using a pluggable container runtime. This, for example, makes the functionality provided to GitHub Actions easily consumable (or testable) by users of other CI systems.

Forge currently exposes running GitHub Actions (e.g. actions/setup-go) and Concourse Resources (e.g. concourse/git-resource).

install

macOS

brew install frantjc/tap/forge

usage

GitHub Actions

For GitHub Actions, Forge will try to source the GitHub Actions variables from the working directory's Git configuration as well as GitHub's default environment variables.

forge use actions/setup-go@v3 -w go-version=1.19

Forge mounts the current working directory to the Action's GITHUB_WORKSPACE as well as cache directories respecting the XDG Base Directory Specification to the Action's RUNNER_TOOLCACHE and RUNNER_TEMP.

That is to say, after running the above command, go should be installed to XDG_CACHE_HOME/.forge/runner/toolcache.

You can also use local GitHub Actions by starting the reference with "/" or "./" to signify that it is an absolute or relative local filepath, respectively.

forge use ./testdata/actions/mock/docker

Concourse Resources

For Concourse Resources, Forge will source resource_types and resources from the working directory's forge.yml (overridable with -c). This schema is conveniently compatible with Concourse's pipeline schema.

forge get mock -i version=v0.0.0

why?

Automation begins with a shell script that executes a bunch of CLI commands often to test, build and publish some code. The next step is to set up some CI system that executes that script, for example, on every commit to a repository's main branch. Such CI systems tend to identify that all of the scripts that they are executing do a lot of the same things--checkout a Git repository, setup a tool and so on.

In an effort to make their platform easier to use and to refactor the shared functionality out of all of the aforementioned scripts, CI systems in the past have introduced reusable "plugins"/"Actions"/"Resources" which take minimal configuration to do a complex task. GitHub Actions' actions/checkout, for example, takes one short line of code to invoke and accepts much optional configuration to modify its functionality to fulfill many related use cases.

Unfortunately, using such powerful plugins outside of the the system they were built for can be wildly difficult. This makes debugging the use of these plugins painful due to the long feedback loops. It also makes migrating from one CI system to another treacherous, having to replace uses of one system's plugins with another's.

Forge aims to remedy this.

developing

• make is recommended - version 3.81 is tested
• golang is required - version 1.18.x or above is required for generics
• docker is required - version 20.10.x is tested
• upx is required for compressing shim

Documentation

Index

• Variables
• func GetSemver() string
• func WithLogger(ctx context.Context, logger Logger) context.Context
• type Container
• type ContainerConfig
• type ContainerRuntime
• type Drains
  • func StdDrains() *Drains
  • func (d *Drains) GoString() string
  • func (d *Drains) ToStreams(in io.Reader) *Streams
• type FileDescriptor
• type Foundry
  • func NewFoundry(containerRuntime ContainerRuntime) *Foundry
  • func (f *Foundry) GoString() string
  • func (f *Foundry) Process(ctx context.Context, ore Ore, drains *Drains) (*Metal, error)
• type Image
• type Logger
  • func LoggerFrom(ctx context.Context) Logger
  • func NewLogger() Logger
• type Metal
• type Mount
• type Ore
• type Streams
  • func StdStreams() *Streams
  • func StdTerminalStreams() (*Streams, func() error)
  • func TerminalStreams(stdin io.Reader, stdout, stderr io.Writer) (*Streams, func() error, error)
• type Volume

Variables

var DefaultDetachKeys = "ctrl-d"

DefaultDetachKeys are the default key combinations to use when detaching from a Container that has been attached to.

var Semver = "0.0.0"

Semver is the semantic version of forge. Meant to be be overridden at build time.

Functions

func GetSemver

func GetSemver() string

GetSemver returns the semantic version of Forge as built from Semver and debug build info.

func WithLogger

func WithLogger(ctx context.Context, logger Logger) context.Context

WithLogger returns a Context from the parent Context with the given Logger inside of it.

Types

type Container

type Container interface {
	GetID() string

	CopyTo(context.Context, string, io.Reader) error
	CopyFrom(context.Context, string) (io.ReadCloser, error)

	Run(context.Context, *Streams) (int, error)
	Start(context.Context) error
	Restart(context.Context) error
	Exec(context.Context, *ContainerConfig, *Streams) (int, error)

	Stop(context.Context) error
	Remove(context.Context) error
	Kill(context.Context) error
}

Container represents a container created by a ContainerRuntime.

type ContainerConfig

type ContainerConfig struct {
	Entrypoint []string `json:"entrypoint,omitempty"`
	Cmd        []string `json:"cmd,omitempty"`
	WorkingDir string   `json:"working_dir,omitempty"`
	Env        []string `json:"env,omitempty"`
	User       string   `json:"user,omitempty"`
	Privileged bool     `json:"privileged,omitempty"`
	Mounts     []*Mount `json:"mounts,omitempty"`
}

type ContainerRuntime

type ContainerRuntime interface {
	GetContainer(context.Context, string) (Container, error)
	CreateContainer(context.Context, Image, *ContainerConfig) (Container, error)
	PullImage(context.Context, string) (Image, error)
	BuildDockerfile(context.Context, string, string) (Image, error)
	CreateVolume(context.Context, string) (Volume, error)
	Close() error
}

type Drains

type Drains struct {
	Out, Err io.Writer
	Tty      bool
}

Drains represents only outward streams from an Ore, namely stdout and stderr.

func StdDrains

func StdDrains() *Drains

StdDrains returns a Drains draining to os.Stdout and os.Stderr.

func (*Drains) GoString

func (d *Drains) GoString() string

GoString implements fmt.GoStringer.

func (*Drains) ToStreams

func (d *Drains) ToStreams(in io.Reader) *Streams

ToStreams turns a Drains to a Streams for use by Ores to pass to a Container.

type FileDescriptor

type FileDescriptor interface {
	Fd() uintptr
}

FileDescriptor is an interface to check io.Readers and io.Writers against to inspect if they are terminals.

type Foundry

type Foundry struct {
	ContainerRuntime
}

Foundry is a wrapper around a ContainerRuntime for processing Ores.

func NewFoundry

func NewFoundry(containerRuntime ContainerRuntime) *Foundry

NewFoundry returns a Foundry.

func (*Foundry) GoString

func (f *Foundry) GoString() string

GoString implements fmt.GoStringer.

func (*Foundry) Process

func (f *Foundry) Process(ctx context.Context, ore Ore, drains *Drains) (*Metal, error)

Process Liquifies the Ore and returns the resulting Metal.

type Image

type Image interface {
	Manifest() (*imagespecsv1.Manifest, error)
	Config() (*imagespecsv1.ImageConfig, error)
	Digest() (digest.Digest, error)
	Blob() io.Reader
	Name() string
}

Image represents a image pulled by a ContainerRuntime. Used to create Containers from.

type Logger

type Logger = logr.Logger

Logger is an alias to logr.Logger in case the logging library is desired to be swapped out.

func LoggerFrom

func LoggerFrom(ctx context.Context) Logger

LoggerFrom returns a Logger embedded within the given Context or a no-op Logger if no such Logger exists.

func NewLogger

func NewLogger() Logger

NewLogger creates a new Logger.

type Metal

type Metal struct {
	ExitCode int64 `json:"exit_code,omitempty"`
}

type Mount

type Mount struct {
	Source      string `json:"source,omitempty"`
	Destination string `json:"destination,omitempty"`
}

type Ore

type Ore interface {
	Liquify(context.Context, ContainerRuntime, *Drains) (*Metal, error)
}

Ore represents one or more sequential containerized commands. Ores are meant to represent the entire input to said commands, so that if two Ore's digests match, their resulting Metals should be the same. Because of this, Ores can be cached, using said Digest as the key.

type Streams

type Streams struct {
	*Drains
	In         io.Reader
	DetachKeys string
}

Streams represents streams to and from a process inside of a Container.

func StdStreams

func StdStreams() *Streams

StdStreams returns a Streams consisting of os.Stdin, os.Stdout and os.Stderr.

func StdTerminalStreams

func StdTerminalStreams() (*Streams, func() error)

StdTerminalStreams creates a Streams with os.Stdin, os.Stdout and os.Stderr made raw and a restore function to return them to their previous state. For use with attaching to a shell inside of a Container.

func TerminalStreams

func TerminalStreams(stdin io.Reader, stdout, stderr io.Writer) (*Streams, func() error, error)

TerminalStreams creates a Streams with each of the given streams that is a terminal made raw and a restore function to return them to their previous states. For use with attaching to a shell inside of a Container.

type Volume

type Volume interface {
	GetID() string
	Remove(context.Context) error
}

Volume represents a volume created by a ContainerRuntime which can be attached to a Container via its ContainerConfig.Mounts.