loader

Documentation

Overview

Package loader loads a complete Go program from source code, parsing and type-checking the initial packages plus their transitive closure of dependencies. The ASTs and the derived facts are retained for later use.

THIS INTERFACE IS EXPERIMENTAL AND IS LIKELY TO CHANGE.

The package defines two primary types: Config, which specifies a set of initial packages to load and various other options; and Program, which is the result of successfully loading the packages specified by a configuration.

The configuration can be set directly, but *Config provides various convenience methods to simplify the common cases, each of which can be called any number of times. Finally, these are followed by a call to Load() to actually load and type-check the program.

var conf loader.Config

// Use the command-line arguments to specify
// a set of initial packages to load from source.
// See FromArgsUsage for help.
rest, err := conf.FromArgs(os.Args[1:], wantTests)

// Parse the specified files and create an ad hoc package with path "foo".
// All files must have the same 'package' declaration.
conf.CreateFromFilenames("foo", "foo.go", "bar.go")

// Create an ad hoc package with path "foo" from
// the specified already-parsed files.
// All ASTs must have the same 'package' declaration.
conf.CreateFromFiles("foo", parsedFiles)

// Add "runtime" to the set of packages to be loaded.
conf.Import("runtime")

// Adds "fmt" and "fmt_test" to the set of packages
// to be loaded.  "fmt" will include *_test.go files.
conf.ImportWithTests("fmt")

// Finally, load all the packages specified by the configuration.
prog, err := conf.Load()

See examples_test.go for examples of API usage.

CONCEPTS AND TERMINOLOGY

The WORKSPACE is the set of packages accessible to the loader. The workspace is defined by Config.Build, a *build.Context. The default context treats subdirectories of $GOROOT and $GOPATH as packages, but this behavior may be overridden.

An AD HOC package is one specified as a set of source files on the command line. In the simplest case, it may consist of a single file such as $GOROOT/src/net/http/triv.go.

EXTERNAL TEST packages are those comprised of a set of *_test.go files all with the same 'package foo_test' declaration, all in the same directory. (go/build.Package calls these files XTestFiles.)

An IMPORTABLE package is one that can be referred to by some import spec. Every importable package is uniquely identified by its PACKAGE PATH or just PATH, a string such as "fmt", "encoding/json", or "cmd/vendor/golang.org/x/arch/x86/x86asm". A package path typically denotes a subdirectory of the workspace.

An import declaration uses an IMPORT PATH to refer to a package. Most import declarations use the package path as the import path.

Due to VENDORING (https://golang.org/s/go15vendor), the interpretation of an import path may depend on the directory in which it appears. To resolve an import path to a package path, go/build must search the enclosing directories for a subdirectory named "vendor".

ad hoc packages and external test packages are NON-IMPORTABLE. The path of an ad hoc package is inferred from the package declarations of its files and is therefore not a unique package key. For example, Config.CreatePkgs may specify two initial ad hoc packages, both with path "main".

An AUGMENTED package is an importable package P plus all the *_test.go files with same 'package foo' declaration as P. (go/build.Package calls these files TestFiles.)

The INITIAL packages are those specified in the configuration. A DEPENDENCY is a package loaded to satisfy an import in an initial package or another dependency.

Index

• Constants
• type Config
  • func (conf *Config) CreateFromFilenames(path string, filenames ...string)
  • func (conf *Config) CreateFromFiles(path string, files ...*ast.File)
  • func (conf *Config) FromArgs(args []string, xtest bool) ([]string, error)
  • func (conf *Config) Import(path string)
  • func (conf *Config) ImportWithTests(path string)
  • func (conf *Config) Load() (*Program, error)
  • func (conf *Config) ParseFile(filename string, src interface{}) (*ast.File, error)
• type PackageInfo
  • func (info *PackageInfo) String() string
• type PkgSpec
• type Program
  • func (prog *Program) InitialPackages() []*PackageInfo
  • func (prog *Program) Package(path string) *PackageInfo
  • func (prog *Program) PathEnclosingInterval(start, end token.Pos) (pkg *PackageInfo, path []ast.Node, exact bool)

Constants

const FromArgsUsage = `` /* 1067-byte string literal not displayed */

FromArgsUsage is a partial usage message that applications calling FromArgs may wish to include in their -help output.

Types

type Config

type Config struct {
	// Fset is the file set for the parser to use when loading the
	// program.  If nil, it may be lazily initialized by any
	// method of Config.
	Fset *token.FileSet

	// ParserMode specifies the mode to be used by the parser when
	// loading source packages.
	ParserMode parser.Mode

	// TypeChecker contains options relating to the type checker.
	//
	// The supplied IgnoreFuncBodies is not used; the effective
	// value comes from the TypeCheckFuncBodies func below.
	// The supplied Import function is not used either.
	TypeChecker types.Config

	// TypeCheckFuncBodies is a predicate over package paths.
	// A package for which the predicate is false will
	// have its package-level declarations type checked, but not
	// its function bodies; this can be used to quickly load
	// dependencies from source.  If nil, all func bodies are type
	// checked.
	TypeCheckFuncBodies func(path string) bool

	// If Build is non-nil, it is used to locate source packages.
	// Otherwise &build.Default is used.
	//
	// By default, cgo is invoked to preprocess Go files that
	// import the fake package "C".  This behaviour can be
	// disabled by setting CGO_ENABLED=0 in the environment prior
	// to startup, or by setting Build.CgoEnabled=false.
	Build *build.Context

	// The current directory, used for resolving relative package
	// references such as "./go/loader".  If empty, os.Getwd will be
	// used instead.
	Cwd string

	// If DisplayPath is non-nil, it is used to transform each
	// file name obtained from Build.Import().  This can be used
	// to prevent a virtualized build.Config's file names from
	// leaking into the user interface.
	DisplayPath func(path string) string

	// If AllowErrors is true, Load will return a Program even
	// if some of the its packages contained I/O, parser or type
	// errors; such errors are accessible via PackageInfo.Errors.  If
	// false, Load will fail if any package had an error.
	AllowErrors bool

	// CreatePkgs specifies a list of non-importable initial
	// packages to create.  The resulting packages will appear in
	// the corresponding elements of the Program.Created slice.
	CreatePkgs []PkgSpec

	// ImportPkgs specifies a set of initial packages to load.
	// The map keys are package paths.
	//
	// The map value indicates whether to load tests.  If true, Load
	// will add and type-check two lists of files to the package:
	// non-test files followed by in-package *_test.go files.  In
	// addition, it will append the external test package (if any)
	// to Program.Created.
	ImportPkgs map[string]bool

	// FindPackage is called during Load to create the build.Package
	// for a given import path from a given directory.
	// If FindPackage is nil, (*build.Context).Import is used.
	// A client may use this hook to adapt to a proprietary build
	// system that does not follow the "go build" layout
	// conventions, for example.
	//
	// It must be safe to call concurrently from multiple goroutines.
	FindPackage func(ctxt *build.Context, importPath, fromDir string, mode build.ImportMode) (*build.Package, error)

	// AfterTypeCheck is called immediately after a list of files
	// has been type-checked and appended to info.Files.
	//
	// This optional hook function is the earliest opportunity for
	// the client to observe the output of the type checker,
	// which may be useful to reduce analysis latency when loading
	// a large program.
	//
	// The function is permitted to modify info.Info, for instance
	// to clear data structures that are no longer needed, which can
	// dramatically reduce peak memory consumption.
	//
	// The function may be called twice for the same PackageInfo:
	// once for the files of the package and again for the
	// in-package test files.
	//
	// It must be safe to call concurrently from multiple goroutines.
	AfterTypeCheck func(info *PackageInfo, files []*ast.File)
}

Config specifies the configuration for loading a whole program from Go source code. The zero value for Config is a ready-to-use default configuration.

func (*Config) CreateFromFilenames

func (conf *Config) CreateFromFilenames(path string, filenames ...string)

CreateFromFilenames is a convenience function that adds a conf.CreatePkgs entry to create a package of the specified *.go files.

func (*Config) CreateFromFiles

func (conf *Config) CreateFromFiles(path string, files ...*ast.File)

CreateFromFiles is a convenience function that adds a conf.CreatePkgs entry to create package of the specified path and parsed files.

func (*Config) FromArgs

func (conf *Config) FromArgs(args []string, xtest bool) ([]string, error)

FromArgs interprets args as a set of initial packages to load from source and updates the configuration. It returns the list of unconsumed arguments.

It is intended for use in command-line interfaces that require a set of initial packages to be specified; see FromArgsUsage message for details.

Only superficial errors are reported at this stage; errors dependent on I/O are detected during Load.

func (*Config) Import

func (conf *Config) Import(path string)

Import is a convenience function that adds path to ImportPkgs, the set of initial packages that will be imported from source.

func (*Config) ImportWithTests

func (conf *Config) ImportWithTests(path string)

ImportWithTests is a convenience function that adds path to ImportPkgs, the set of initial source packages located relative to $GOPATH. The package will be augmented by any *_test.go files in its directory that contain a "package x" (not "package x_test") declaration.

In addition, if any *_test.go files contain a "package x_test" declaration, an additional package comprising just those files will be added to CreatePkgs.

func (*Config) Load

func (conf *Config) Load() (*Program, error)

Load creates the initial packages specified by conf.{Create,Import}Pkgs, loading their dependencies packages as needed.

On success, Load returns a Program containing a PackageInfo for each package. On failure, it returns an error.

If AllowErrors is true, Load will return a Program even if some packages contained I/O, parser or type errors, or if dependencies were missing. (Such errors are accessible via PackageInfo.Errors. If false, Load will fail if any package had an error.

It is an error if no packages were loaded.

func (*Config) ParseFile

func (conf *Config) ParseFile(filename string, src interface{}) (*ast.File, error)

ParseFile is a convenience function (intended for testing) that invokes the parser using the Config's FileSet, which is initialized if nil.

src specifies the parser input as a string, []byte, or io.Reader, and filename is its apparent name. If src is nil, the contents of filename are read from the file system.

type PackageInfo

type PackageInfo struct {
	Pkg                   *types.Package
	Importable            bool        // true if 'import "Pkg.Path()"' would resolve to this
	TransitivelyErrorFree bool        // true if Pkg and all its dependencies are free of errors
	Files                 []*ast.File // syntax trees for the package's files
	Errors                []error     // non-nil if the package had errors
	types.Info                        // type-checker deductions.
	// contains filtered or unexported fields
}

PackageInfo holds the ASTs and facts derived by the type-checker for a single package.

Not mutated once exposed via the API.

func (*PackageInfo) String

func (info *PackageInfo) String() string

type PkgSpec

type PkgSpec struct {
	Path      string      // package path ("" => use package declaration)
	Files     []*ast.File // ASTs of already-parsed files
	Filenames []string    // names of files to be parsed
}

A PkgSpec specifies a non-importable package to be created by Load. Files are processed first, but typically only one of Files and Filenames is provided. The path needn't be globally unique.

For vendoring purposes, the package's directory is the one that contains the first file.

type Program

type Program struct {
	Fset *token.FileSet // the file set for this program

	// Created[i] contains the initial package whose ASTs or
	// filenames were supplied by Config.CreatePkgs[i], followed by
	// the external test package, if any, of each package in
	// Config.ImportPkgs ordered by ImportPath.
	//
	// NOTE: these files must not import "C".  Cgo preprocessing is
	// only performed on imported packages, not ad hoc packages.
	//
	// TODO(adonovan): we need to copy and adapt the logic of
	// goFilesPackage (from $GOROOT/src/cmd/go/build.go) and make
	// Config.Import and Config.Create methods return the same kind
	// of entity, essentially a build.Package.
	// Perhaps we can even reuse that type directly.
	Created []*PackageInfo

	// Imported contains the initially imported packages,
	// as specified by Config.ImportPkgs.
	Imported map[string]*PackageInfo

	// AllPackages contains the PackageInfo of every package
	// encountered by Load: all initial packages and all
	// dependencies, including incomplete ones.
	AllPackages map[*types.Package]*PackageInfo
	// contains filtered or unexported fields
}

A Program is a Go program loaded from source as specified by a Config.

func (*Program) InitialPackages

func (prog *Program) InitialPackages() []*PackageInfo

InitialPackages returns a new slice containing the set of initial packages (Created + Imported) in unspecified order.

func (*Program) Package

func (prog *Program) Package(path string) *PackageInfo

Package returns the ASTs and results of type checking for the specified package.

func (*Program) PathEnclosingInterval

func (prog *Program) PathEnclosingInterval(start, end token.Pos) (pkg *PackageInfo, path []ast.Node, exact bool)

PathEnclosingInterval returns the PackageInfo and ast.Node that contain source interval [start, end), and all the node's ancestors up to the AST root. It searches all ast.Files of all packages in prog. exact is defined as for astutil.PathEnclosingInterval.

The zero value is returned if not found.