analysis

Documentation

Overview

Package analysis defines the interface between a modular static analysis and an analysis driver program.

Background

A static analysis is a function that inspects a package of Go code and reports a set of diagnostics (typically mistakes in the code), and perhaps produces other results as well, such as suggested refactorings or other facts. An analysis that reports mistakes is informally called a "checker". For example, the printf checker reports mistakes in fmt.Printf format strings.

A "modular" analysis is one that inspects one package at a time but can save information from a lower-level package and use it when inspecting a higher-level package, analogous to separate compilation in a toolchain. The printf checker is modular: when it discovers that a function such as log.Fatalf delegates to fmt.Printf, it records this fact, and checks calls to that function too, including calls made from another package.

By implementing a common interface, checkers from a variety of sources can be easily selected, incorporated, and reused in a wide range of driver programs including command-line tools (such as vet), text editors and IDEs, build and test systems (such as go build, Bazel, or Buck), test frameworks, code review tools, code-base indexers (such as SourceGraph), documentation viewers (such as godoc), batch pipelines for large code bases, and so on.

Analyzer

The primary type in the API is Analyzer. An Analyzer statically describes an analysis function: its name, documentation, flags, relationship to other analyzers, and of course, its logic.

To define an analysis, a user declares a (logically constant) variable of type Analyzer. Here is a typical example from one of the analyzers in the go/analysis/passes/ subdirectory:

package unusedresult

var Analyzer = &analysis.Analyzer{
	Name: "unusedresult",
	Doc:  "check for unused results of calls to some functions",
	Run:  run,
	...
}

func run(pass *analysis.Pass) (interface{}, error) {
	...
}

An analysis driver is a program such as vet that runs a set of analyses and prints the diagnostics that they report. The driver program must import the list of Analyzers it needs. Typically each Analyzer resides in a separate package. To add a new Analyzer to an existing driver, add another item to the list:

import ( "unusedresult"; "nilness"; "printf" )

var analyses = []*analysis.Analyzer{
	unusedresult.Analyzer,
	nilness.Analyzer,
	printf.Analyzer,
}

A driver may use the name, flags, and documentation to provide on-line help that describes the analyses it performs. The doc comment contains a brief one-line summary, optionally followed by paragraphs of explanation.

The Analyzer type has more fields besides those shown above:

type Analyzer struct {
	Name             string
	Doc              string
	Flags            flag.FlagSet
	Run              func(*Pass) (interface{}, error)
	RunDespiteErrors bool
	ResultType       reflect.Type
	Requires         []*Analyzer
	FactTypes        []Fact
}

The Flags field declares a set of named (global) flag variables that control analysis behavior. Unlike vet, analysis flags are not declared directly in the command line FlagSet; it is up to the driver to set the flag variables. A driver for a single analysis, a, might expose its flag f directly on the command line as -f, whereas a driver for multiple analyses might prefix the flag name by the analysis name (-a.f) to avoid ambiguity. An IDE might expose the flags through a graphical interface, and a batch pipeline might configure them from a config file. See the "findcall" analyzer for an example of flags in action.

The RunDespiteErrors flag indicates whether the analysis is equipped to handle ill-typed code. If not, the driver will skip the analysis if there were parse or type errors. The optional ResultType field specifies the type of the result value computed by this analysis and made available to other analyses. The Requires field specifies a list of analyses upon which this one depends and whose results it may access, and it constrains the order in which a driver may run analyses. The FactTypes field is discussed in the section on Modularity. The analysis package provides a Validate function to perform basic sanity checks on an Analyzer, such as that its Requires graph is acyclic, its fact and result types are unique, and so on.

Finally, the Run field contains a function to be called by the driver to execute the analysis on a single package. The driver passes it an instance of the Pass type.

Pass

A Pass describes a single unit of work: the application of a particular Analyzer to a particular package of Go code. The Pass provides information to the Analyzer's Run function about the package being analyzed, and provides operations to the Run function for reporting diagnostics and other information back to the driver.

type Pass struct {
	Fset         *token.FileSet
	Files        []*ast.File
	OtherFiles   []string
	IgnoredFiles []string
	Pkg          *types.Package
	TypesInfo    *types.Info
	ResultOf     map[*Analyzer]interface{}
	Report       func(Diagnostic)
	...
}

The Fset, Files, Pkg, and TypesInfo fields provide the syntax trees, type information, and source positions for a single package of Go code.

The OtherFiles field provides the names of non-Go files such as assembly that are part of this package. Similarly, the IgnoredFiles field provides the names of Go and non-Go source files that are not part of this package with the current build configuration but may be part of other build configurations. The contents of these files may be read using Pass.ReadFile; see the "asmdecl" or "buildtags" analyzers for examples of loading non-Go files and reporting diagnostics against them.

The ResultOf field provides the results computed by the analyzers required by this one, as expressed in its Analyzer.Requires field. The driver runs the required analyzers first and makes their results available in this map. Each Analyzer must return a value of the type described in its Analyzer.ResultType field. For example, the "ctrlflow" analyzer returns a *ctrlflow.CFGs, which provides a control-flow graph for each function in the package (see github.com/block/ftl-golang-tools/go/cfg); the "inspect" analyzer returns a value that enables other Analyzers to traverse the syntax trees of the package more efficiently; and the "buildssa" analyzer constructs an SSA-form intermediate representation. Each of these Analyzers extends the capabilities of later Analyzers without adding a dependency to the core API, so an analysis tool pays only for the extensions it needs.

The Report function emits a diagnostic, a message associated with a source position. For most analyses, diagnostics are their primary result. For convenience, Pass provides a helper method, Reportf, to report a new diagnostic by formatting a string. Diagnostic is defined as:

type Diagnostic struct {
	Pos      token.Pos
	Category string // optional
	Message  string
}

The optional Category field is a short identifier that classifies the kind of message when an analysis produces several kinds of diagnostic.

The Diagnostic struct does not have a field to indicate its severity because opinions about the relative importance of Analyzers and their diagnostics vary widely among users. The design of this framework does not hold each Analyzer responsible for identifying the severity of its diagnostics. Instead, we expect that drivers will allow the user to customize the filtering and prioritization of diagnostics based on the producing Analyzer and optional Category, according to the user's preferences.

Most Analyzers inspect typed Go syntax trees, but a few, such as asmdecl and buildtag, inspect the raw text of Go source files or even non-Go files such as assembly. To report a diagnostic against a line of a raw text file, use the following sequence:

content, err := pass.ReadFile(filename)
if err != nil { ... }
tf := fset.AddFile(filename, -1, len(content))
tf.SetLinesForContent(content)
...
pass.Reportf(tf.LineStart(line), "oops")

Modular analysis with Facts

To improve efficiency and scalability, large programs are routinely built using separate compilation: units of the program are compiled separately, and recompiled only when one of their dependencies changes; independent modules may be compiled in parallel. The same technique may be applied to static analyses, for the same benefits. Such analyses are described as "modular".

A compiler’s type checker is an example of a modular static analysis. Many other checkers we would like to apply to Go programs can be understood as alternative or non-standard type systems. For example, vet's printf checker infers whether a function has the "printf wrapper" type, and it applies stricter checks to calls of such functions. In addition, it records which functions are printf wrappers for use by later analysis passes to identify other printf wrappers by induction. A result such as “f is a printf wrapper” that is not interesting by itself but serves as a stepping stone to an interesting result (such as a diagnostic) is called a Fact.

The analysis API allows an analysis to define new types of facts, to associate facts of these types with objects (named entities) declared within the current package, or with the package as a whole, and to query for an existing fact of a given type associated with an object or package.

An Analyzer that uses facts must declare their types:

var Analyzer = &analysis.Analyzer{
	Name:      "printf",
	FactTypes: []analysis.Fact{new(isWrapper)},
	...
}

type isWrapper struct{} // => *types.Func f “is a printf wrapper”

The driver program ensures that facts for a pass’s dependencies are generated before analyzing the package and is responsible for propagating facts from one package to another, possibly across address spaces. Consequently, Facts must be serializable. The API requires that drivers use the gob encoding, an efficient, robust, self-describing binary protocol. A fact type may implement the GobEncoder/GobDecoder interfaces if the default encoding is unsuitable. Facts should be stateless. Because serialized facts may appear within build outputs, the gob encoding of a fact must be deterministic, to avoid spurious cache misses in build systems that use content-addressable caches. The driver makes a single call to the gob encoder for all facts exported by a given analysis pass, so that the topology of shared data structures referenced by multiple facts is preserved.

The Pass type has functions to import and export facts, associated either with an object or with a package:

type Pass struct {
	...
	ExportObjectFact func(types.Object, Fact)
	ImportObjectFact func(types.Object, Fact) bool

	ExportPackageFact func(fact Fact)
	ImportPackageFact func(*types.Package, Fact) bool
}

An Analyzer may only export facts associated with the current package or its objects, though it may import facts from any package or object that is an import dependency of the current package.

Conceptually, ExportObjectFact(obj, fact) inserts fact into a hidden map keyed by the pair (obj, TypeOf(fact)), and the ImportObjectFact function retrieves the entry from this map and copies its value into the variable pointed to by fact. This scheme assumes that the concrete type of fact is a pointer; this assumption is checked by the Validate function. See the "printf" analyzer for an example of object facts in action.

Some driver implementations (such as those based on Bazel and Blaze) do not currently apply analyzers to packages of the standard library. Therefore, for best results, analyzer authors should not rely on analysis facts being available for standard packages. For example, although the printf checker is capable of deducing during analysis of the log package that log.Printf is a printf wrapper, this fact is built in to the analyzer so that it correctly checks calls to log.Printf even when run in a driver that does not apply it to standard packages. We would like to remove this limitation in future.

Testing an Analyzer

The analysistest subpackage provides utilities for testing an Analyzer. In a few lines of code, it is possible to run an analyzer on a package of testdata files and check that it reported all the expected diagnostics and facts (and no more). Expectations are expressed using "// want ..." comments in the input code.

Standalone commands

Analyzers are provided in the form of packages that a driver program is expected to import. The vet command imports a set of several analyzers, but users may wish to define their own analysis commands that perform additional checks. To simplify the task of creating an analysis command, either for a single analyzer or for a whole suite, we provide the singlechecker and multichecker subpackages.

The singlechecker package provides the main function for a command that runs one analyzer. By convention, each analyzer such as go/analysis/passes/findcall should be accompanied by a singlechecker-based command such as go/analysis/passes/findcall/cmd/findcall, defined in its entirety as:

package main

import (
	"github.com/block/ftl-golang-tools/go/analysis/passes/findcall"
	"github.com/block/ftl-golang-tools/go/analysis/singlechecker"
)

func main() { singlechecker.Main(findcall.Analyzer) }

A tool that provides multiple analyzers can use multichecker in a similar way, giving it the list of Analyzers.

Index

• func Validate(analyzers []*Analyzer) error
• type Analyzer
  • func (a *Analyzer) String() string
• type CycleInRequiresGraphError
  • func (e *CycleInRequiresGraphError) Error() string
• type Diagnostic
  • func (d Diagnostic) ToSimple(fset *token.FileSet) SimpleDiagnostic
• type Fact
• type ObjectFact
• type PackageFact
• type Pass
  • func (pass *Pass) ReportRangef(rng Range, format string, args ...interface{})
  • func (pass *Pass) Reportf(pos token.Pos, format string, args ...interface{})
  • func (pass *Pass) String() string
• type Range
• type RelatedInformation
• type SimpleDiagnostic
• type SimplePosition
• type SuggestedFix
• type TextEdit

Functions

func Validate

func Validate(analyzers []*Analyzer) error

Validate reports an error if any of the analyzers are misconfigured. Checks include: that the name is a valid identifier; that the Doc is not empty; that the Run is non-nil; that the Requires graph is acyclic; that analyzer fact types are unique; that each fact type is a pointer.

Analyzer names need not be unique, though this may be confusing.

Types

type Analyzer

type Analyzer struct {
	// The Name of the analyzer must be a valid Go identifier
	// as it may appear in command-line flags, URLs, and so on.
	Name string

	// Doc is the documentation for the analyzer.
	// The part before the first "\n\n" is the title
	// (no capital or period, max ~60 letters).
	Doc string

	// URL holds an optional link to a web page with additional
	// documentation for this analyzer.
	URL string

	// Flags defines any flags accepted by the analyzer.
	// The manner in which these flags are exposed to the user
	// depends on the driver which runs the analyzer.
	Flags flag.FlagSet

	// Run applies the analyzer to a package.
	// It returns an error if the analyzer failed.
	//
	// On success, the Run function may return a result
	// computed by the Analyzer; its type must match ResultType.
	// The driver makes this result available as an input to
	// another Analyzer that depends directly on this one (see
	// Requires) when it analyzes the same package.
	//
	// To pass analysis results between packages (and thus
	// potentially between address spaces), use Facts, which are
	// serializable.
	Run func(*Pass) (interface{}, error)

	// RunDespiteErrors allows the driver to invoke
	// the Run method of this analyzer even on a
	// package that contains parse or type errors.
	// The Pass.TypeErrors field may consequently be non-empty.
	RunDespiteErrors bool

	// Requires is a set of analyzers that must run successfully
	// before this one on a given package. This analyzer may inspect
	// the outputs produced by each analyzer in Requires.
	// The graph over analyzers implied by Requires edges must be acyclic.
	//
	// Requires establishes a "horizontal" dependency between
	// analysis passes (different analyzers, same package).
	Requires []*Analyzer

	// ResultType is the type of the optional result of the Run function.
	ResultType reflect.Type

	// FactTypes indicates that this analyzer imports and exports
	// Facts of the specified concrete types.
	// An analyzer that uses facts may assume that its import
	// dependencies have been similarly analyzed before it runs.
	// Facts must be pointers.
	//
	// FactTypes establishes a "vertical" dependency between
	// analysis passes (same analyzer, different packages).
	FactTypes []Fact
}

An Analyzer describes an analysis function and its options.

func (*Analyzer) String

func (a *Analyzer) String() string

type CycleInRequiresGraphError

type CycleInRequiresGraphError struct {
	AnalyzerNames map[string]bool
}

func (*CycleInRequiresGraphError) Error

func (e *CycleInRequiresGraphError) Error() string

type Diagnostic

type Diagnostic struct {
	Pos      token.Pos
	End      token.Pos // optional
	Category string    // optional
	Message  string

	// URL is the optional location of a web page that provides
	// additional documentation for this diagnostic.
	//
	// If URL is empty but a Category is specified, then the
	// Analysis driver should treat the URL as "#"+Category.
	//
	// The URL may be relative. If so, the base URL is that of the
	// Analyzer that produced the diagnostic;
	// see https://pkg.go.dev/net/url#URL.ResolveReference.
	URL string

	// SuggestedFixes is an optional list of fixes to address the
	// problem described by the diagnostic. Each one represents
	// an alternative strategy; at most one may be applied.
	//
	// Fixes for different diagnostics should be treated as
	// independent changes to the same baseline file state,
	// analogous to a set of git commits all with the same parent.
	// Combining fixes requires resolving any conflicts that
	// arise, analogous to a git merge.
	// Any conflicts that remain may be dealt with, depending on
	// the tool, by discarding fixes, consulting the user, or
	// aborting the operation.
	SuggestedFixes []SuggestedFix

	// Related contains optional secondary positions and messages
	// related to the primary diagnostic.
	Related []RelatedInformation
}

A Diagnostic is a message associated with a source location or range.

An Analyzer may return a variety of diagnostics; the optional Category, which should be a constant, may be used to classify them. It is primarily intended to make it easy to look up documentation.

All Pos values are interpreted relative to Pass.Fset. If End is provided, the diagnostic is specified to apply to the range between Pos and End.

func (Diagnostic) ToSimple

func (d Diagnostic) ToSimple(fset *token.FileSet) SimpleDiagnostic

type Fact

type Fact interface {
	AFact() // dummy method to avoid type errors
}

A Fact is an intermediate fact produced during analysis.

Each fact is associated with a named declaration (a types.Object) or with a package as a whole. A single object or package may have multiple associated facts, but only one of any particular fact type.

A Fact represents a predicate such as "never returns", but does not represent the subject of the predicate such as "function F" or "package P".

Facts may be produced in one analysis pass and consumed by another analysis pass even if these are in different address spaces. If package P imports Q, all facts about Q produced during analysis of that package will be available during later analysis of P. Facts are analogous to type export data in a build system: just as export data enables separate compilation of several passes, facts enable "separate analysis".

Each pass (a, p) starts with the set of facts produced by the same analyzer a applied to the packages directly imported by p. The analysis may add facts to the set, and they may be exported in turn. An analysis's Run function may retrieve facts by calling Pass.Import{Object,Package}Fact and update them using Pass.Export{Object,Package}Fact.

A fact is logically private to its Analysis. To pass values between different analyzers, use the results mechanism; see Analyzer.Requires, Analyzer.ResultType, and Pass.ResultOf.

A Fact type must be a pointer. Facts are encoded and decoded using encoding/gob. A Fact may implement the GobEncoder/GobDecoder interfaces to customize its encoding. Fact encoding should not fail.

A Fact should not be modified once exported.

type ObjectFact

type ObjectFact struct {
	Object types.Object
	Fact   Fact
}

ObjectFact is an object together with an associated fact.

type PackageFact

type PackageFact struct {
	Package *types.Package
	Fact    Fact
}

PackageFact is a package together with an associated fact.

type Pass

type Pass struct {
	Analyzer *Analyzer // the identity of the current analyzer

	// syntax and type information
	Fset         *token.FileSet // file position information; Run may add new files
	Files        []*ast.File    // the abstract syntax tree of each file
	OtherFiles   []string       // names of non-Go files of this package
	IgnoredFiles []string       // names of ignored source files in this package
	Pkg          *types.Package // type information about the package
	TypesInfo    *types.Info    // type information about the syntax trees
	TypesSizes   types.Sizes    // function for computing sizes of types
	TypeErrors   []types.Error  // type errors (only if Analyzer.RunDespiteErrors)

	// Report reports a Diagnostic, a finding about a specific location
	// in the analyzed source code such as a potential mistake.
	// It may be called by the Run function.
	Report func(Diagnostic)

	// ResultOf provides the inputs to this analysis pass, which are
	// the corresponding results of its prerequisite analyzers.
	// The map keys are the elements of Analysis.Required,
	// and the type of each corresponding value is the required
	// analysis's ResultType.
	ResultOf map[*Analyzer]interface{}

	// ReadFile returns the contents of the named file.
	//
	// The only valid file names are the elements of OtherFiles
	// and IgnoredFiles, and names returned by
	// Fset.File(f.FileStart).Name() for each f in Files.
	//
	// Analyzers must use this function (if provided) instead of
	// accessing the file system directly. This allows a driver to
	// provide a virtualized file tree (including, for example,
	// unsaved editor buffers) and to track dependencies precisely
	// to avoid unnecessary recomputation.
	ReadFile func(filename string) ([]byte, error)

	// ImportObjectFact retrieves a fact associated with obj.
	// Given a value ptr of type *T, where *T satisfies Fact,
	// ImportObjectFact copies the value to *ptr.
	//
	// ImportObjectFact panics if called after the pass is complete.
	// ImportObjectFact is not concurrency-safe.
	ImportObjectFact func(obj types.Object, fact Fact) bool

	// ImportPackageFact retrieves a fact associated with package pkg,
	// which must be this package or one of its dependencies.
	// See comments for ImportObjectFact.
	ImportPackageFact func(pkg *types.Package, fact Fact) bool

	// ExportObjectFact associates a fact of type *T with the obj,
	// replacing any previous fact of that type.
	//
	// ExportObjectFact panics if it is called after the pass is
	// complete, or if obj does not belong to the package being analyzed.
	// ExportObjectFact is not concurrency-safe.
	ExportObjectFact func(obj types.Object, fact Fact)

	// ExportPackageFact associates a fact with the current package.
	// See comments for ExportObjectFact.
	ExportPackageFact func(fact Fact)

	// AllPackageFacts returns a new slice containing all package
	// facts of the analysis's FactTypes in unspecified order.
	AllPackageFacts func() []PackageFact

	// AllObjectFacts returns a new slice containing all object
	// facts of the analysis's FactTypes in unspecified order.
	AllObjectFacts func() []ObjectFact
}

A Pass provides information to the Run function that applies a specific analyzer to a single Go package.

It forms the interface between the analysis logic and the driver program, and has both input and an output components.

As in a compiler, one pass may depend on the result computed by another.

The Run function should not call any of the Pass functions concurrently.

func (*Pass) ReportRangef

func (pass *Pass) ReportRangef(rng Range, format string, args ...interface{})

ReportRangef is a helper function that reports a Diagnostic using the range provided. ast.Node values can be passed in as the range because they satisfy the Range interface.

func (*Pass) Reportf

func (pass *Pass) Reportf(pos token.Pos, format string, args ...interface{})

Reportf is a helper function that reports a Diagnostic using the specified position and formatted error message.

func (*Pass) String

func (pass *Pass) String() string

type Range

type Range interface {
	Pos() token.Pos // position of first character belonging to the node
	End() token.Pos // position of first character immediately after the node
}

The Range interface provides a range. It's equivalent to and satisfied by ast.Node.

type RelatedInformation

type RelatedInformation struct {
	Pos     token.Pos
	End     token.Pos // optional
	Message string
}

RelatedInformation contains information related to a diagnostic. For example, a diagnostic that flags duplicated declarations of a variable may include one RelatedInformation per existing declaration.

type SimpleDiagnostic

type SimpleDiagnostic struct {
	Pos      SimplePosition
	End      SimplePosition // optional
	Category string         // optional
	Message  string
}

A SimpleDiagnostic is a simplified representation of Diagnostic for use in APIs that don't have access to package loader types.

type SimplePosition

type SimplePosition struct {
	Filename string
	Line     int
	Column   int
	Offset   int
}

A SimplePosition is a simplified representation of token.Position for use in APIs that don't have access to package loader types.

type SuggestedFix

type SuggestedFix struct {
	// A verb phrase describing the fix, to be shown to
	// a user trying to decide whether to accept it.
	//
	// Example: "Remove the surplus argument"
	Message   string
	TextEdits []TextEdit
}

A SuggestedFix is a code change associated with a Diagnostic that a user can choose to apply to their code. Usually the SuggestedFix is meant to fix the issue flagged by the diagnostic.

The TextEdits must not overlap, nor contain edits for other packages.

type TextEdit

type TextEdit struct {
	// For a pure insertion, End can either be set to Pos or token.NoPos.
	Pos     token.Pos
	End     token.Pos
	NewText []byte
}

A TextEdit represents the replacement of the code between Pos and End with the new text. Each TextEdit should apply to a single file. End should not be earlier in the file than Pos.