fab

package module
v0.50.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 18, 2024 License: MIT Imports: 36 Imported by: 0

README

Fab - software fabricator

Go Reference Go Report Card Tests Coverage Status

This is Fab, a system for orchestrating software builds.

Fab is like Make, but with named, parameterized target types. You can describe your build with YAML files and/or Go code, and you can also define new target types in Go.

(But that doesn’t mean Fab is for building Go programs only, any more than writing shell commands in a Makefile means Make builds only shell programs.)

Running fab on one or more targets ensures that the targets’ prerequisites, and the targets themselves, are up to date according to your build rules, while avoiding unnecessarily rebuilding any target that is already up to date.

Usage

You will need an installation of Go version 1.20 or later. Download Go here: go.dev/dl.

Once Go is installed you can install Fab like this:

go install github.com/bobg/fab/cmd/fab@latest

To define build targets in your software project, write Go code in a _fab subdirectory and/or write a fab.yaml file. See Targets below for how to do this.

To build targets in your software project, run

fab TARGET1 TARGET2 ...

To see the progress of your build you can add the -v flag (for “verbose”):

fab -v TARGET1 TARGET2 ...

If you have a target that takes command-line parameters, you can invoke it like this:

fab TARGET ARG1 ARG2 ...

In this form, ARG1 must start with a -, and no other targets may be specified.

To see the available build targets in your project, run

fab -list

Targets

Each fab target has a type that dictates the parameters required by the target (if any), and the actions that the target should take when it runs.

Fab predefines several target types. Here is a partial list:

  • Command invokes a shell command when it runs.
  • F invokes an arbitrary Go function.
  • Files specifies a set of input files and a set of output files, and a nested subtarget that runs only when the outputs are out-of-date with respect to the inputs.
  • All invokes a set of subtargets in parallel.
  • Seq invokes subtargets in sequence.
  • Deps invokes a subtarget only after its dependencies (other subtargets) have run.
  • Clean deletes a set of files.

You define targets by instantiating one of these types, supplying it with any necessary arguments, and giving it a name. There are three ways to do this: statically in Go code; dynamically in Go code; and declaratively in one or more YAML files. These options are discussed below.

You can also define new target types by implementing the fab.Target interface.

Static target definition in Go

You can write Go code to define targets that Fab can run. To do this, create a subdirectory named _fab at the root of your project, and create .go files in that directory. This name prevents the code in that directory from being considered as part of the public API for your project. (Citation.)

You can use any package name for the code in directory _fab; the official suggestion is _fab, to match the directory name.

Any exported identifiers at the top level of this package whose type implements the fab.Target interface are usable Fab targets. For example:

package _fab

import (
    "os"

    "github.com/bobg/fab"
)

// Test runs tests.
var Test = &fab.Command{Shell: "go test -cover ./...", Stdout: os.Stdout}

This creates a Command-typed target named Test, which you can invoke with fab Test. When it runs, it executes the given shell command and copies its output to fab’s standard output.

The comment above the variable declaration gets attached to the created target. Running fab -list will show that comment as a docstring:

$ fab -list
Test
    Test runs tests.

Dynamic target definition in Go

Not all targets are suitable for creation via top-level variable declarations. Those that require more complex processing can be defined dynamically using the fab.RegisterTarget function.

for m := time.January; m <= time.December; m++ {
  fab.RegisterTarget(
    m.String(),
    "Say that it’s "+m.String(),
    &fab.Command{Shell: "echo It is "+m.String(), Stdout: os.Stdout},
  )
}

This creates targets named January, February, March, etc.

Internally, static target definition works by calling fab.RegisterTarget.

Declarative target definition in YAML

In addition to Go code in the _fab subdirectory, or instead of it, you can define targets in one or more fab.yaml files.

The top-level structure of the YAML file is a mapping from names to targets. Targets are specified using YAML type tags. Most Fab target types define a tag and a syntax for extracting necessary arguments from YAML. Targets may also be referred to by name.

Here is an example fab.yaml file:

# Prog rebuilds prog if any of the files it depends on changes.
Prog: !Files
  In: !go.Deps
    Dir: cmd/prog
  Out:
    - prog
  Target: Build

# Unconditionally rebuild prog.
Build: !Command
  Shell: go build -o prog ./cmd/prog

# Test runs all tests.
Test: !Command
  Shell: go test -race -cover ./...
  Stdout: $stdout

This defines Prog as a Fab target of type Files. The In argument is the list of input files that may change and the Out argument is the list of expected output files that result from running the nested subtarget. That subtarget is a reference to the Build rule, which is a Command that runs go build. In is defined as the result of the go.Deps rule, which produces the list of files on which the Go package in a given directory depends.

This also defines a Test target as a Command that runs go test.

You may write fab.yaml files in multiple subdirectories of your project. When you write a fab.yaml file in a subdirectory foo/bar of your project’s top directory, it must include this declaration:

_dir: foo/bar

A fab.yaml file at the top level of your project does not need this declaration.

When a target T is defined in a YAML file in subdirectory D, it is registered (using RegisterTarget) with the name D/T. A target in one fab.yaml file may refer to a target in another by specifying the relative path to it. for example, if the top-level fab.yaml contains:

A: foo/B

this means that foo/fab.yaml defines a target B.

_dir: foo

B: ../bar/C

Here, B is defined in terms of a target in yet another file, bar/fab.yaml:

_dir: bar

C: !Command
  Shell: echo Hello
  Stdout: $stdout

All of the target types in the github.com/bobg/fab package are available to your YAML file by default. To make other target types available, it is necessary to import their packages in Go code in the _fab directory. For example, to make the !go.Binary tag work, you’ll need a .go file under _fab that contains:

import _ "github.com/bobg/fab/golang"

(The _ means your Go code doesn’t use anything in the golang package directly, but imports it for its side effects — namely, registering YAML tags like !go.Binary.)

If you rely entirely on YAML files, it’s possible that your .go code will contain only import statements like this and not define any targets or types, which is fine.

Note than any imports in Go code in the _fab subdirectory must be reflected in the dependencies in the top-level go.mod and go.sum files (even if this isn’t a Go project!). TODO: add documentation about this use case.

Defining new target types

You can define new target types in Go code in the _fab subdirectory (or anywhere else, that is then imported into the _fab package).

Your type must implement fab.Target, which requires two methods: Desc and Run.

Desc produces a short string describing the target. It is used by Describe to describe targets that don’t have a name (i.e., ones that were never registered with RegisterTarget, possibly because they are nested inside some other target).

Run should unconditionally execute your target type’s logic. The Fab runtime will take care of making sure your target runs only when it needs to. More about this appears below.

If part of your Run method involves running other targets, do not invoke their Run methods directly. Instead, invoke the Run method on the Controller that your Run method receives as an argument, passing it the target you want to run. This will skip that target if it has already run. This means that a “diamond dependency” — A depends on B and C, and B and C each separately depend on X — won’t cause X to run twice when the user runs fab A.

Your implementation should be a pointer type, which is required for targets passed to Describe and RegisterTarget.

If you would like your type to be usable as the subtarget in a Files rule, it must be JSON-encodable (unlike F, for example). Among other things, this means that struct fields should be exported, or it should implement json.Marshaler. See json.Marshal for more detail on what’s encodable.

If you would like your new target type to be usable in fab.yaml, you must define a YAML parser for it. This is done with RegisterYAMLTarget, which associates a name with a YAMLTargetFunc. When the YAML tag !name is encountered in fab.yaml (in a context where a target may be specified), your function will be invoked to parse the YAML node. The function YAMLTarget parses a YAML node into a Target using the functions in this registry.

There is also a registry for functions that parse a YAML node into a list of strings. For example, this YAML snippet:

!go.Deps
  Dir: foo/bar
  Recursive: true

produces the list of files on which the Go code in foo/bar depends. You can add functions to this registry with RegisterYAMLStringList, and parse a YAML node into a string list using functions from this registry with YAMLStringList.

Files

The Files target type specifies a set of input files, a set of expected output files, and a nested subtarget for producing one from the other. It uses this information in two special ways: for file chaining and for content-based dependency checking.

File chaining

When a Files target runs, it looks for filenames in its input list that appear in the output lists of other Files targets. Other targets found in this way are Run first as prerequisites.

Here is a simple example in YAML:

AB: !Files
  In: [a]
  Out: [b]
  Target: !Command
    Shell: cp a b

BC: !Files
  In: [b]
  Out: [c]
  Target: !Command
    Shell: cp b c

(File a produces b by copying; file b produces c by copying.)

If you run fab BC to update c from b, Fab will discover that the input file b appears in the output list of target AB, and run that target first.

(If b is already up to date with respect to a, running AB will have no effect. See the next section for more about this.)

Content-based dependency checking

After running any prerequisites found via file chaining, a Files target computes a hash combining the content of all the input files, all the output files (those that exist), and the rules for the nested subtarget. It then checks for the presence of this hash in a persistent hash database that records the state of things after any successful past run of the target.

If the hash is there, then the run succeeds trivially; the output files are already up to date with respect to the inputs, and running the subtarget is skipped.

Otherwise the nested subtarget runs, and then the hash is computed again and placed into the hash database. The next time this target runs, if none of the files has changed, then the hash will be the same and running the subtarget will be skipped. On the other hand, if any file has changed, the hash will be different and won’t be found in the database, so the subtarget will run.

(It is possible for input and output files to change in such a way that the hash is found in the database, because they match a previous “up to date” state. Consider a simple Files rule for example that copies a single input file in to a single output file out. Let’s say the first time it runs, in contains Hello and that gets copied to out, and the resulting post-run hash is 17. [Actual hashes are much, much, much bigger numbers.] Now you change in to contain Goodbye and rerun the target. The hash with in=Goodbye and out=Hello isn’t in the database, so the copy rule runs again and the new hash is 42. If you now change both in and out back to Hello and rerun the target, the hash will again be 17, representing an earlier state where out is up to date with respect to in, so there is no copying needed.)

This is a key difference between Fab and Make. Make uses file modification times to decide when a set of output files needs to be recomputed from their inputs. Considering the limited resolution of filesystem timestamps, the possibility of clock skew, etc., the content-based test that Fab uses is preferable. (But it would be easy to define a file-modtime-based target type in Fab if that’s what you wanted.)

The hash database is stored in $HOME/.cache/fab by default, and hash values normally expire after thirty days.

Using the Files target type to translate Makefiles

It is possible to translate Makefile rules to Fab rules using the Files target type.

The following Makefile snippet means, “produce files a and b from input files c and d by running command1 followed by command2.”

a b: c d
  command1
  command2

The same thing in Fab’s YAML format looks like this.

Name: !Files
  - In: [c, d]
  - Out: [a, b]
  - Target: !Seq
    - !Command
      Shell: command1
    - !Command
      Shell: command2

Note that the Fab version has a Name whereas the Make version does not.

The Fab runtime

A Fab Controller is responsible for invoking targets’ Run methods, keeping track of which ones have already run so that they don’t get invoked a second time.

The controller uses the address of each target as a unique key. This means that pointer types should be used to implement Target. After a target runs, the controller records its outcome (error or no error). The second and subsequent attempts to run a given target will use the previously computed outcome.

Program startup

If you have Go code in a _fab subdirectory, Fab combines it with its own main function to produce a driver, which is an executable Go binary that is stored in $HOME/.cache/fab by default.

The driver calls fab.RegisterTarget on each of the eligible top-level identifiers in your package; then it looks for a fab.yaml file and registers the target definitions it finds there. After that, the driver runs the targets you specified on the fab command line (or lists targets if you specified -list, etc).

When you run fab and the driver is already built and up to date (as determined by a hash of the code in the _fab dir), then fab simply executes the driver without rebuilding it. You can force a rebuild of the driver by specifying -f to fab.

If you do not have a _fab subdirectory, then Fab operates in “driverless” mode, in which the fab.yaml file is loaded and the targets on the command line executed.

Note that when you have both a _fab subdirectory and a fab.yaml file, you may use target types in the YAML file that are defined in your _fab package. When you have only a fab.yaml file you are limited to the target types that are predefined in Fab.

Why not Mage?

Fab was strongly inspired by the excellent Mage tool, which works similarly and has a similar feature set. But Fab has some features the author needed and did not find in Mage:

  • Errors from Target rules propagate out instead of causing an exit.
  • Targets are values, not functions, and are composable (e.g. with Seq, All, and Deps).
  • Rebuilding of up-to-date targets can be skipped based on file contents, not modtimes.

Documentation

Index

Constants

View Source 
const LoadMode = packages.NeedName | packages.NeedFiles | packages.NeedTypes | packages.NeedDeps

LoadMode is the minimal set of flags to enable for Config.Mode in a call to Packages.Load in order to produce a suitable package object for CompilePackage.

Variables

This section is empty.

Functions

func Compile added in v0.8.0 

func Compile(ctx context.Context, pkgdir, binfile string) error

Compile compiles a "driver" from a directory of user code (combined with a main function supplied by fab) and places the executable result in a given file. The driver converts command-line target names into the necessary Fab rule invocations.

The package of user code should contain one or more exported identifiers whose types satisfy the Target interface. These become the build rules that the driver can invoke.

When Compile runs the "go" program must exist in the user's PATH. It must be Go version 1.19 or later.

How it works:

  • The user's code is loaded with packages.Load.
  • The set of exported top-level identifiers is filtered to find those implementing the fab.Target interface.
  • The user's code is then copied to a temp directory together with a main package (and main() function) that registers (with Register) that set of targets.
  • The go compiler is invoked to produce an executable, which is renamed into place as binfile.

For the synthesized calls to Register on Target-valued variables, the driver uses the variable's name as the "name" argument and the variable's doc comment as the "doc" argument.

The user's code is able to make its own calls to Register during program initialization in order to augment the set of available targets.

func CompilePackage added in v0.28.0 

func CompilePackage(ctx context.Context, pkg *packages.Package, binfile string) error

CompilePackage compiles a driver from a package object already loaded with packages.Load. The call to packages.Load must use a value for Config.Mode that contains at least the bits in LoadMode. See Compile for further details.

func GetArgs added in v0.29.0 

func GetArgs(ctx context.Context) []string

GetArgs returns the list of arguments added to `ctx` with WithArgs. The default, if WithArgs was not used, is nil.

func GetDryRun added in v0.46.0 

func GetDryRun(ctx context.Context) bool

GetDryRun returns the value of the dryrun boolean added to `ctx` with WithDryRun. The default, if WithDryRun was not used, is false.

func GetForce added in v0.16.0 

func GetForce(ctx context.Context) bool

GetForce returns the value of the force boolean added to `ctx` with WithForce. The default, if WithForce was not used, is false.

func GetVerbose 

func GetVerbose(ctx context.Context) bool

GetVerbose returns the value of the verbose boolean added to `ctx` with WithVerbose. The default, if WithVerbose was not used, is false.

func OpenHashDB added in v0.30.0 

func OpenHashDB(dir string) (*sqlite.DB, error)

OpenHashDB ensures the given directory exists and opens (or creates) the hash DB there. Callers must make sure to call Close on the returned DB when finished with it.

func RegisterYAMLStringList added in v0.30.0 

func RegisterYAMLStringList(name string, fn YAMLStringListFunc)

RegisterYAMLStringList places a function in the YAML string-list registry with the given name. Use a YAML `!name` tag to introduce a node that should be parsed using this function.

func RegisterYAMLTarget added in v0.30.0 

func RegisterYAMLTarget(name string, fn YAMLTargetFunc)

RegisterYAMLTarget places a function in the YAML target registry with the given name. Use a YAML `!name` tag to introduce a node that should be parsed using this function.

func TopDir added in v0.44.0 

func TopDir(dir string) (string, error)

TopDir finds the top directory of a project, given a directory inside it.

The top directory is the one containing a _fab subdirectory or (since that might not exist) the one that fab.yaml files' _dir declarations are relative to.

If TopDir can't find the answer in dir, it will look in dir's parent, and so on up the tree.

func WithArgs added in v0.29.0 

func WithArgs(ctx context.Context, args ...string) context.Context

WithArgs decorates a context with a list of arguments as a slice of strings. Retrieve it with GetArgs.

func WithDryRun added in v0.46.0 

func WithDryRun(ctx context.Context, dryrun bool) context.Context

WithDryRun decorates a context with the value of a "dryrun" boolean. Retrieve it with GetDryRun.

func WithForce added in v0.16.0 

func WithForce(ctx context.Context, force bool) context.Context

WithForce decorates a context with the value of a "force" boolean. Retrieve it with GetForce.

func WithHashDB 

func WithHashDB(ctx context.Context, db HashDB) context.Context

WithHashDB decorates a context with a HashDB. Retrieve it with GetHashDB.

func WithVerbose 

func WithVerbose(ctx context.Context, verbose bool) context.Context

WithVerbose decorates a context with the value of a "verbose" boolean. Retrieve it with GetVerbose.

Types

type BadYAMLNodeKindError added in v0.47.0 

type BadYAMLNodeKindError struct {
	Got, Want yaml.Kind
}

BadYAMLNodeKindError is the type of error returned by various functions when the kind of a YAML node does not match expectations.

func (BadYAMLNodeKindError) Error added in v0.47.0 

func (e BadYAMLNodeKindError) Error() string

type Clean added in v0.18.0 

type Clean struct {
	Files     []string
	Autoclean bool
}

Clean is a Target that deletes the files named in Files when it runs. Files that already don't exist are silently ignored.

If Autoclean is true, files listed in the "autoclean registry" are also removed. See Autoclean for more about this feature.

A Clean target may be specified in YAML using the tag !Clean. It may introduce a sequence, in which case the elements are files to delete, or a mapping with fields `Files`, the files to delete, and `Autoclean`, a boolean for enabling the autoclean feature.

When GetDryRun is true, Clean will not remove any files.

func (*Clean) Desc added in v0.48.0 

func (*Clean) Desc() string

Desc implements Target.Desc.

func (*Clean) Run added in v0.18.0 

func (c *Clean) Run(ctx context.Context, con *Controller) error

Run implements Target.Run.

type Command 

type Command struct {
	// Shell is the command to run,
	// as a single string with command name and arguments together.
	// It is invoked with $SHELL -c,
	// with $SHELL defaulting to /bin/sh.
	//
	// If you prefer to specify a command that is not executed by a shell,
	// leave Shell blank and fill in Cmd and Args instead.
	//
	// To bypass this parsing behavior,
	// you may specify Cmd and Args directly.
	Shell string `json:"shell,omitempty"`

	// Cmd is the command to invoke,
	// either the path to a file,
	// or an executable file found in some directory
	// named in the PATH environment variable.
	//
	// If you need your command string to be parsed by a shell,
	// leave Cmd and Args blank and specify Shell instead.
	Cmd string `json:"cmd,omitempty"`

	// Args is the list of command-line arguments
	// to pass to the command named in Cmd.
	Args []string `json:"args,omitempty"`

	// Stdout tells where to send the command's output.
	// When no output destination is specified,
	// the default depends on whether Fab is running in verbose mode
	// (i.e., if [GetVerbose] returns true).
	// In verbose mode,
	// the command's output is indented and copied to Fab's standard output
	// (using [IndentingCopier]).
	// Otherwise,
	// the command's output is captured
	// and bundled together with any error into a [CommandErr].
	//
	// Stdout, StdoutFile, and StdoutFn are all mutually exclusive.
	Stdout io.Writer `json:"-"`

	// Stderr tells where to send the command's error output.
	// When no error-output destination is specified,
	// the default depends on whether Fab is running in verbose mode
	// (i.e., if [GetVerbose] returns true).
	// In verbose mode,
	// the command's error output is indented and copied to Fab's standard error
	// (using [IndentingCopier]).
	// Otherwise,
	// the command's error output is captured
	// and bundled together with any error into a [CommandErr].
	//
	// Stderr, StderrFile, and StderrFn are all mutually exclusive.
	Stderr io.Writer `json:"-"`

	// StdoutFn lets you defer assigning a value to Stdout
	// until Run is invoked,
	// at which time this function is called with the context and the [Controller]
	// to produce the [io.Writer] to use.
	// If the writer produced by this function is also an [io.Closer],
	// its Close method will be called before Run exits.
	//
	// Stdout, StdoutFile, and StdoutFn are all mutually exclusive.
	StdoutFn func(context.Context, *Controller) io.Writer `json:"-"`

	// StderrFn lets you defer assigning a value to Stderr
	// until Run is invoked,
	// at which time this function is called with the context and the [Controller]
	// to produce the [io.Writer] to use.
	// If the writer produced by this function is also an [io.Closer],
	// its Close method will be called before Run exits.
	//
	// Stderr, StderrFile, and StderrFn are all mutually exclusive.
	StderrFn func(context.Context, *Controller) io.Writer `json:"-"`

	// StdoutFile is the name of a file to which the command's standard output should go.
	// When the command runs,
	// the file is created or overwritten,
	// unless this string has a >> prefix,
	// which means "append."
	// If StdoutFile and StderrFile name the same file,
	// output from both streams is combined there.
	//
	// Stdout, StdoutFile, and StdoutFn are all mutually exclusive.
	StdoutFile string `json:"stdout_file,omitempty"`

	// StderrFile is the name of a file to which the command's standard error should go.
	// When the command runs,
	// the file is created or overwritten,
	// unless this string has a >> prefix,
	// which means "append."
	// If StdoutFile and StderrFile name the same file,
	// output from both streams is combined there.
	//
	// Stderr, StderrFile, and StderrFn are all mutually exclusive.
	StderrFile string `json:"stderr_file,omitempty"`

	// Stdin tells where to read the command's standard input.
	Stdin io.Reader `json:"-"`

	// StdinFile is the name of a file from which the command should read its standard input.
	// It is mutually exclusive with Stdin.
	// It is an error for the file not to exist when the command runs.
	StdinFile string `json:"stdin_file,omitempty"`

	// Dir is the directory in which to run the command.
	Dir string `json:"dir,omitempty"`

	// Env is a list of VAR=VALUE strings to add to the environment when the command runs.
	Env []string `json:"env,omitempty"`
}

Command is a Target whose Run function executes a command in a subprocess.

It is JSON-encodable (and therefore usable as the subtarget in Files).

A Command target may be specified in YAML using the !Command tag, which introduces a mapping with the following fields:

  • Shell, the command string to execute with $SHELL, mutually exclusive with Cmd.
  • Cmd, an executable command invoked with Args as its arguments, mutually exclusive with Shell.
  • Args, list of arguments for Cmd.
  • Stdin, the name of a file from which the command's standard input should be read, or the special string $stdin to mean read Fab's standard input.
  • Stdout, the name of a file to which the command's standard output should be written, either absolute or relative to the directory in which the YAML file is found. The file is overwritten unless this is prefixed with >> which means append. This may also be one of these special strings: $stdout (copy the command's output to Fab's standard output); $stderr (copy the command's output to Fab's standard error); $indent (indent the command's output with [IndentingCopier] and copy it to Fab's standard output); $verbose (like $indent, but produce output only when fab is running in verbose mode [with the -v flag]); $discard (discard the command's output).
  • Stderr, the name of a file to which the command's standard error should be written, either absolute or relative to the directory in which the YAML file is found. The file is overwritten unless this is prefixed with >> which means append. This may also be one of these special strings: $stdout (copy the command's error output to Fab's standard error); $stderr (copy the command's error output to Fab's standard error); $indent (indent the command's error output with [IndentingCopier] and copy it to Fab's standard error); $verbose (like $indent, but produce output only when fab is running in verbose mode [with the -v flag]); $discard (discard the command's error output).
  • Dir, the directory in which the command should run, either absolute or relative to the directory in which the YAML file is found.
  • Env, a list of VAR=VALUE strings to add to the command's environment.

As a special case, a !Command whose shell is a list instead of a single string will produce a Seq of Commands, one for each of the Shell strings. The Commands in the Seq are otherwise identical, with one further special case: if Stdout and/or Stderr refers to a file, then the second and subsequent Commands in the Seq will always append to the file rather than overwrite it, even without the >> prefix. (If you really do want some command in the sequence to overwrite a file, you can always add >FILE to the Shell string.)

func Shellf added in v0.34.0 

func Shellf(format string, args ...any) *Command

Shellf is a convenience routine that produces a *Command whose Shell field is initialized by processing `format` and `args` with fmt.Sprintf.

func (*Command) Desc added in v0.34.0 

func (*Command) Desc() string

Desc implements Target.Desc.

func (*Command) Run 

func (c *Command) Run(ctx context.Context, con *Controller) (err error)

Run implements Target.Run.

type CommandErr 

type CommandErr struct {
	Err    error
	Output []byte
}

CommandErr is a type of error that may be returned from command.Run. If the command's Stdout or Stderr field was nil, then that output from the subprocess is in CommandErr.Output and the underlying error is in CommandErr.Err.

func (CommandErr) Error 

func (e CommandErr) Error() string

Error implements error.Error.

func (CommandErr) Unwrap 

func (e CommandErr) Unwrap() error

Unwrap produces the underlying error.

type Controller added in v0.44.0 

type Controller struct {
	// contains filtered or unexported fields
}

Controller is in charge of registering and running targets. It keeps track of targets that are presently running or have previously run. It will not run the same target more than once. The second and subsequent request to run a given target will used the cached outcome (error or no error) of the first run.

func NewController added in v0.44.0 

func NewController(topdir string) *Controller

NewController creates a new Controller for the project with the given top-level directory.

The top directory is where a _fab subdirectory and/or a top-level fab.yaml file is expected.

func (*Controller) Describe added in v0.44.0 

func (con *Controller) Describe(target Target) string

Describe describes a target. The description is the target's name in the registry, if it has one (i.e., if the target was registered with [RegisterTarget]), otherwise it's "unnamed X" where X is the result of calling the target's Desc method.

func (*Controller) Indentf added in v0.44.0 

func (con *Controller) Indentf(format string, args ...any)

Indentf formats and prints its arguments with leading indentation based on the nesting depth of the controller. The nesting depth increases with each call to Controller.Run and decreases at the end of the call.

A newline is added to the end of the string if one is not already there.

func (*Controller) IndentingCopier added in v0.44.0 

func (con *Controller) IndentingCopier(w io.Writer, prefix string) io.Writer

IndentingCopier creates an io.Writer that copies its data to an underlying writer, indenting each line according to the indentation depth of the controller. After indentation, each line additionally gets any prefix specified in `prefix`.

The wrapper converts \r\n to \n, and bare \r to \n. A \r at the very end of the input is silently dropped.

func (*Controller) JoinPath added in v0.44.0 

func (con *Controller) JoinPath(elts ...string) string

JoinPath is like filepath.Join with some additional behavior. Any absolute path segment discards everything to the left of it. If all path segments are relative, then con's top directory is implicitly joined at the beginning.

Examples:

  • JoinPath("a/b", "c/d") -> TOP/a/b/c/d
  • JoinPath("a/b", "/c/d") -> /c/d

func (*Controller) ListTargets added in v0.44.0 

func (con *Controller) ListTargets(w io.Writer)

ListTargets outputs a formatted list of the targets in the registry and their docstrings.

func (*Controller) ParseArgs added in v0.44.0 

func (con *Controller) ParseArgs(args []string) ([]Target, error)

ParseArgs parses the remaining arguments on a fab command line, after option flags. They are either a list of target names in the registry, in which case those targets are returned; or a single registry target followed by option flags for that, in which case the target is wrapped up in an ArgTarget with its options. The two cases are distinguished by whether there is a second argument and whether it begins with a hyphen. (That's the ArgTarget case.)

func (*Controller) ReadYAML added in v0.44.0 

func (con *Controller) ReadYAML(r io.Reader, dir string) error

ReadYAML reads a YAML document from the given source, registering Targets that it finds. The `dir` argument is relative to the top directory of `con` and serves as the prefix for any targets registered.

The top level of the YAML document should be a mapping from names to targets. Each target is either a target-typed node, selected by a !tag, or the name of some other target.

For example, the following creates a target named `Check`, which is an `All`-typed target referring to two other targets: `Vet` and `Test`. Each of those is a `Command`-typed target executing specific shell commands. 

Check: !All
  - Vet
  - Test

Vet: !Command
  - go vet ./...

Test: !Command
  - go test ./...

func (*Controller) ReadYAMLFile added in v0.44.0 

func (con *Controller) ReadYAMLFile(dir string) error

ReadYAMLFile calls ReadYAML on the file `fab.yaml` in the given directory or, if that doesn't exist, `fab.yml`.

func (*Controller) RegisterTarget added in v0.44.0 

func (con *Controller) RegisterTarget(name, doc string, target Target) (Target, error)

RegisterTarget places a target in the registry with a given name and doc string.

func (*Controller) RegistryNames added in v0.44.0 

func (con *Controller) RegistryNames() []string

RegistryNames returns the names in the target registry.

func (*Controller) RegistryTarget added in v0.44.0 

func (con *Controller) RegistryTarget(name string) (Target, string)

RegistryTarget returns the target in the registry with the given name, and its doc string.

func (*Controller) RelPath added in v0.44.0 

func (con *Controller) RelPath(path string) (string, error)

RelPath returns the relative path to `path` from con's top directory.

func (*Controller) Run added in v0.44.0 

func (con *Controller) Run(ctx context.Context, targets ...Target) error

Run runs the given targets, skipping any that have already run.

A controller remembers which targets it has already run (whether in this call or any previous call to Run).

The targets are executed concurrently. A separate goroutine is created for each one passed to Run. If the controller has never yet run the target, it does so, and caches the result (error or no error). If the target did already run, the cached error value is used. If another goroutine concurrently requests the same target, it blocks until the first one completes, then uses the first one's result.

This function waits for all goroutines to complete. The return value may be an accumulation of multiple errors produced with errors.Join.

func (*Controller) YAMLFileList added in v0.44.0 

func (con *Controller) YAMLFileList(node *yaml.Node, dir string) ([]string, error)

YAMLFileList constructs a slice of filenames from a YAML node. It does this by calling [YAMLStringList] and passing the result through Controller.JoinPath, joining each string with the given directory. In this way, the files are interpreted as either absolute or relative to `dir`.

func (*Controller) YAMLFileListFromNodes added in v0.44.0 

func (con *Controller) YAMLFileListFromNodes(nodes []*yaml.Node, dir string) ([]string, error)

YAMLFileListFromNodes constructs a slice of filenames from a slice of YAML nodes. It does this by calling [YAMLStringListFromNodes] and passing the result through Controller.JoinPath, joining each string with the given directory. In this way, the files are interpreted as either absolute or relative to `dir`.

func (*Controller) YAMLStringList added in v0.49.0 

func (con *Controller) YAMLStringList(node *yaml.Node, dir string) ([]string, error)

YAMLStringList parses a []string from a YAML node. If the node has a tag `!foo`, then the YAMLStringListFunc in the YAML string-list registry named `foo` is used to parse the node. Otherwise, the node is expected to be a sequence, and [YAMLStringListFromNodes] is called on its children.

func (*Controller) YAMLStringListFromNodes added in v0.49.0 

func (con *Controller) YAMLStringListFromNodes(nodes []*yaml.Node, dir string) ([]string, error)

YAMLStringListFromNodes constructs a slice of strings from a slice of YAML nodes. Each node may be a plain scalar, in which case it is added to the result slice; or a tagged node, in which case it is parsed with the corresponding YAML string-list registry function and the output appended to the result slice.

func (*Controller) YAMLTarget added in v0.44.0 

func (con *Controller) YAMLTarget(node *yaml.Node, dir string) (Target, error)

YAMLTarget parses a Target from a YAML node. If the node has a tag `!foo`, then the YAMLTargetFunc in the YAML target registry named `foo` is used to parse the node. Otherwise, if the node is a bare string `foo`, then it is presumed to refer to a target in the (non-YAML) target registry named `foo`. This string may refer to a target in another directory's YAML file, in which case it should have a path prefix relative to `dir` (e.g. x/foo or ../a/b/foo).

type FilesOpt added in v0.48.0 

type FilesOpt func(*files)

func Autoclean added in v0.48.0 

func Autoclean(autoclean bool) FilesOpt

Autoclean is an option for passing to Files. It causes the output files of the Files target to be added to the "autoclean registry." A Clean target may then choose to remove the files listed in that registry (instead of, or in addition to, any explicitly listed files) by setting its Autoclean field to true.

type HashDB 

type HashDB interface {
	// Has tells whether the database contains the given entry.
	Has(context.Context, []byte) (bool, error)

	// Add adds an entry to the database.
	Add(context.Context, []byte) error
}

HashDB is the type of a database for storing hashes. It must permit concurrent operations safely. It may expire entries to save space.

func GetHashDB 

func GetHashDB(ctx context.Context) HashDB

GetHashDB returns the value of the HashDB added to `ctx` with WithHashDB. The default, if WithHashDB was not used, is nil.

type Main added in v0.11.0 

type Main struct {
	// Fabdir is where to find the user's hash DB and compiled binaries, e.g. $HOME/.cache/fab.
	Fabdir string

	// Topdir is the directory containing a _fab subdir or top-level fab.yaml file.
	// If this is not specified, it will be computed by traversing upward from the current directory.
	Topdir string

	// Verbose tells whether to run the driver in verbose mode
	// (by supplying the -v command-line flag).
	Verbose bool

	// List tells whether to run the driver in list-targets mode
	// (by supplying the -list command-line flag).
	List bool

	// Force tells whether to force recompilation of the driver before running it.
	Force bool

	// DryRun tells whether to run targets in "dry run" mode - i.e., with state-changing operations (like file creation and updating) suppressed.
	DryRun bool

	// Args contains the additional command-line arguments to pass to the driver, e.g. target names.
	Args []string
}

Main is the structure whose Run methods implements the main logic of the fab command.

func (*Main) Run added in v0.11.0 

func (m *Main) Run(ctx context.Context) error

Run executes the main logic of the fab command. A driver binary with a name matching the Go package path of the _fab subdir is sought in m.Fabdir. If it does not exist, or if its corresponding dirhash is wrong (i.e., out of date with respect to the user's code), or if m.Force is true, it is created with Compile. It is then invoked with the command-line arguments indicated by the fields of m. Typically this will include one or more target names, in which case the driver will execute the associated rules as defined by the code in _fab and by any fab.yaml files.

If there is no _fab directory, Run operates in "driverless" mode, in which target definitions are found in fab.yaml files only.

type Target 

type Target interface {
	// Run invokes the target's logic.
	// It receives a context object and the [Controller] running this target as arguments.
	//
	// Callers should not invoke a target's Run method directly.
	// Instead, pass the target to a Controller's Run method.
	// That will handle concurrency properly
	// and make sure that the target is not rerun
	// when it doesn't need to be.
	Run(context.Context, *Controller) error

	// Desc produces a short descriptive string for this target.
	// It is used by [Describe] when the target is not found in the target registry.
	Desc() string
}

Target is the interface that Fab targets must implement.

func All added in v0.3.0 

func All(targets ...Target) Target

All produces a target that runs a collection of targets in parallel.

It is JSON-encodable (and therefore usable as the subtarget in Files) if all of the targets in its collection are.

An All target may be specified in YAML using the tag !All, which introduces a sequence. The elements in the sequence are targets themselves, or target names.

func ArgTarget added in v0.29.0 

func ArgTarget(target Target, args ...string) Target

ArgTarget produces a target with associated arguments as a list of strings, suitable for parsing with the flag package. When the target runs, its arguments are available from the context using GetArgs.

It is JSON-encodable (and therefore usable as the subtarget in Files) if its subtarget is.

An ArgTarget target may be specified in YAML using the tag !ArgTarget, which introduces a sequence. The first element of the sequence is a target or target name. The remaining elements of the sequence are interpreted byu [YAMLStringListFromNodes] to produce the arguments for the target.

func Deps added in v0.3.0 

func Deps(target Target, depTargets ...Target) Target

Deps wraps a target with a set of dependencies, making sure those run first.

It is equivalent to Seq(All(depTargets...), target).

A Deps target may be specified in YAML using the !Deps tag. This may introduce a sequence or a mapping.

If a sequence, then the first element is the main subtarget (or target name), and the remaining elements are dependency targets (or names). Example: 

Foo: !Deps
  - Main
  - Pre1
  - Pre2

This creates target Foo, which runs target Main after running Pre1 and Pre2.

If a mapping, then the `Pre` field specifies a sequence of dependency targets, and the `Post` field specifies the main subtarget. Example: 

Foo: !Deps
  Pre:
    - Pre1
    - Pre2
  Post: Main

This is equivalent to the first example above.

func F 

func F(f func(context.Context, *Controller) error) Target

F produces a target whose Run function invokes the given function. It is not JSON-encodable, so it should not be used as the subtarget in a Files rule.

The behavior of F does not change according to GetDryRun. It's up to the function you pass to F to detect dry-run mode and avoid adding, removing, or updating files, or making other state-altering changes.

func Files added in v0.25.0 

func Files(target Target, in, out []string, opts ...FilesOpt) Target

Files creates a target that contains a list of input files and a list of expected output files. It also contains a nested subtarget whose Run method should produce or update the expected output files.

When the Files target runs, it does the following:

  • It checks to see whether any of its input files are listed as output files in other Files targets. Other targets found in this way are run first, as prerequisites.
  • It then computes a hash from the nested subtarget and all the input and output files. If this hash is found in the “hash database” (obtained with GetHashDB), that means none of the files has changed since the last time the output files were built, so running of the subtarget can be skipped.
  • Otherwise the subtarget is run. The hash is then recomputed and added to the hash database, telling the next run of this target that this collection of input and output files can be considered up-to-date.

The nested subtarget must be of a type that can be JSON-marshaled. Notably this excludes F.

The list of input files should mention every file where a change should cause a rebuild. Ideally this includes any files required by the nested subtarget plus any transitive dependencies. See the Deps function in the golang subpackage for an example of a function that can compute such a list for a Go package.

Passing Autoclean(true) as one of the options causes the output files to be added to the "autoclean registry." A Clean target may then choose to remove the files listed in that registry (instead of, or in addition to, any explicitly listed files) by setting _its_ Autoclean field to true.

The list of input and output files may include directories too. These are walked recursively for computing the hash described above. Be careful when using directories in the output-file list together with the Autoclean feature: the entire directory tree will be deleted.

When GetDryRun is true, checking and updating of the hash DB is skipped.

A Files target may be specified in YAML using the !Files tag, which introduces a mapping whose fields are:

  • Target: the nested subtarget, or target name
  • In: the list of input files, interpreted with [YAMLFilesList]
  • Out: the list of output files, interpreted with [YAMLFilesList]
  • Autoclean: a boolean

Example: 

Foo: !Files
  Target: !Command
    - go build -o thingify ./cmd/thingify
  In: !golang.Deps
    Dir: cmd
  Out:
    - thingify

This creates target Foo, which runs the given `go build` command to update the output file `thingify` when any files depended on by the Go package in `cmd` change.

func Seq added in v0.4.0 

func Seq(targets ...Target) Target

Seq produces a target that runs a collection of targets in sequence. Its Run method exits early when a target in the sequence fails.

It is JSON-encodable (and therefore usable as the subtarget in Files) if all of the targets in its collection are.

A Seq target may be specified in YAML using the tag !Seq, which introduces a sequence. The elements in the sequence are targets themselves, or target names.

type UnknownStringListTagError added in v0.47.0 

type UnknownStringListTagError struct {
	Tag string
}

UnknownStringListTagError is the type of error returned by YAMLStringList when it encounters an unknown node tag.

func (UnknownStringListTagError) Error added in v0.47.0 

func (e UnknownStringListTagError) Error() string

type YAMLStringListFunc added in v0.30.0 

type YAMLStringListFunc = func(*Controller, *yaml.Node, string) ([]string, error)

YAMLStringListFunc is the type of a function in the YAML string-list registry.

type YAMLTargetFunc added in v0.30.0 

type YAMLTargetFunc = func(*Controller, *yaml.Node, string) (Target, error)

YAMLTargetFunc is the type of a function in the YAML target registry.

Source Files

View all Source files

Directories

Path Synopsis
_testdata
topdir/case1/_fab
cmd
fab
ts
_testdata/input/_fab

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.