runner

Documentation

Overview

Package runner implements a go/analysis runner. It makes heavy use of on-disk caching to reduce overall memory usage and to speed up repeat runs.

Public API

A Runner maps a list of analyzers and package patterns to a list of results. Results provide access to diagnostics, directives, errors encountered, and information about packages. Results explicitly do not contain ASTs or type information. All position information is returned in the form of token.Position, not token.Pos. All work that requires access to the loaded representation of a package has to occur inside analyzers.

Planning and execution

Analyzing packages is split into two phases: planning and execution.

During planning, a directed acyclic graph of package dependencies is computed. We materialize the full graph so that we can execute the graph from the bottom up, without keeping unnecessary data in memory during a DFS and with simplified parallel execution.

During execution, leaf nodes (nodes with no outstanding dependencies) get executed in parallel, bounded by a semaphore sized according to the number of CPUs. Conceptually, this happens in a loop, processing new leaf nodes as they appear, until no more nodes are left. In the actual implementation, nodes know their dependents, and the last dependency of a node to be processed is responsible for scheduling its dependent.

The graph is rooted at a synthetic root node. Upon execution of the root node, the algorithm terminates.

Analyzing a package repeats the same planning + execution steps, but this time on a graph of analyzers for the package. Parallel execution of individual analyzers is bounded by the same semaphore as executing packages.

Parallelism

Actions are executed in parallel where the dependency graph allows. Overall parallelism is bounded by a semaphore, sized according to GOMAXPROCS. Each concurrently processed package takes up a token, as does each analyzer – but a package can always execute at least one analyzer, using the package's token.

Depending on the overall shape of the graph, there may be GOMAXPROCS packages running a single analyzer each, a single package running GOMAXPROCS analyzers, or anything in between.

Total memory consumption grows roughly linearly with the number of CPUs, while total execution time is inversely proportional to the number of CPUs. Overall, parallelism is affected by the shape of the dependency graph. A lot of inter-connected packages will see less parallelism than a lot of independent packages.

Caching

The runner caches facts, directives and diagnostics in a content-addressable cache that is designed after Go's own cache. Additionally, it makes use of Go's export data.

This cache not only speeds up repeat runs, it also reduces peak memory usage. When we've analyzed a package, we cache the results and drop them from memory. When a dependent needs any of this information, or when analysis is complete and we wish to render the results, the data gets loaded from disk again.

Data only exists in memory when it is immediately needed, not retained for possible future uses. This trades increased CPU usage for reduced memory usage. A single dependency may be loaded many times over, but it greatly reduces peak memory usage, as an arbitrary amount of time may pass between analyzing a dependency and its dependent, during which other packages will be processed.

Index

• Constants
• type Diagnostic
• type RelatedInformation
• type Result
  • func (r Result) Load() (ResultData, error)
  • func (r Result) LoadTest() (TestData, error)
• type ResultData
• type Runner
  • func New(cfg config.Config, c *cache.Cache) (*Runner, error)
  • func (r *Runner) ActiveWorkers() int
  • func (r *Runner) Run(cfg *packages.Config, analyzers []*analysis.Analyzer, patterns []string) ([]Result, error)
  • func (r *Runner) TotalWorkers() int
• type SerializedDirective
• type Stats
  • func (s *Stats) InitialPackages() int
  • func (s *Stats) ProcessedInitialPackages() int
  • func (s *Stats) ProcessedPackages() int
  • func (s *Stats) State() int
  • func (s *Stats) TotalPackages() int
• type SuggestedFix
• type TestData
• type TestFact
• type TextEdit

Constants

const (
	StateInitializing = iota
	StateLoadPackageGraph
	StateBuildActionGraph
	StateProcessing
	StateFinalizing
)

Types

type Diagnostic

type Diagnostic struct {
	Position token.Position
	End      token.Position
	Category string
	Message  string

	SuggestedFixes []SuggestedFix
	Related        []RelatedInformation
}

Diagnostic is like go/analysis.Diagnostic, but with all token.Pos resolved to token.Position.

type RelatedInformation

type RelatedInformation struct {
	Position token.Position
	End      token.Position
	Message  string
}

RelatedInformation provides additional context for a diagnostic.

type Result

type Result struct {
	Package *loader.PackageSpec
	Config  config.Config
	Initial bool
	Skipped bool

	Failed bool
	Errors []error
	// contains filtered or unexported fields
}

A Result describes the result of analyzing a single package.

It holds references to cached diagnostics and directives. They can be loaded on demand with the Load method.

func (Result) Load

func (r Result) Load() (ResultData, error)

func (Result) LoadTest

func (r Result) LoadTest() (TestData, error)

LoadTest returns data relevant to testing. It should only be called if Runner.TestMode was set to true.

type ResultData

type ResultData struct {
	Directives  []SerializedDirective
	Diagnostics []Diagnostic
	Unused      unused.SerializedResult
}

type Runner

type Runner struct {
	Stats     Stats
	GoVersion string
	// if GoVersion == "module", and we couldn't determine the
	// module's Go version, use this as the fallback
	FallbackGoVersion string
	// If set to true, Runner will populate results with data relevant to testing analyzers
	TestMode bool
	// contains filtered or unexported fields
}

A Runner executes analyzers on packages.

func New

func New(cfg config.Config, c *cache.Cache) (*Runner, error)

New returns a new Runner.

func (*Runner) ActiveWorkers

func (r *Runner) ActiveWorkers() int

ActiveWorkers returns the number of currently running workers.

func (*Runner) Run

func (r *Runner) Run(cfg *packages.Config, analyzers []*analysis.Analyzer, patterns []string) ([]Result, error)

Run loads the packages specified by patterns, runs analyzers on them and returns the results. Each result corresponds to a single package. Results will be returned for all packages, including dependencies. Errors specific to packages will be reported in the respective results.

If cfg is nil, a default config will be used. Otherwise, cfg will be used, with the exception of the Mode field.

func (*Runner) TotalWorkers

func (r *Runner) TotalWorkers() int

TotalWorkers returns the maximum number of possible workers.

type SerializedDirective

type SerializedDirective struct {
	Command   string
	Arguments []string
	// The position of the comment
	DirectivePosition token.Position
	// The position of the node that the comment is attached to
	NodePosition token.Position
}

type Stats

type Stats struct {

	// optional function to call every time an analyzer has finished analyzing a package.
	PrintAnalyzerMeasurement func(*analysis.Analyzer, *loader.PackageSpec, time.Duration)
	// contains filtered or unexported fields
}

func (*Stats) InitialPackages

func (s *Stats) InitialPackages() int

func (*Stats) ProcessedInitialPackages

func (s *Stats) ProcessedInitialPackages() int

func (*Stats) ProcessedPackages

func (s *Stats) ProcessedPackages() int

func (*Stats) State

func (s *Stats) State() int

func (*Stats) TotalPackages

func (s *Stats) TotalPackages() int

type SuggestedFix

type SuggestedFix struct {
	Message   string
	TextEdits []TextEdit
}

type TestData

type TestData struct {
	// Facts contains facts produced by analyzers for a package.
	// Unlike vetx, this list only contains facts specific to this package,
	// not all facts for the transitive closure of dependencies.
	Facts []TestFact
	// List of files that were part of the package.
	Files []string
}

TestData contains extra information about analysis runs that is only available in test mode.

type TestFact

type TestFact struct {
	ObjectName string
	Position   token.Position
	FactString string
	Analyzer   string
}

TestFact is a serialization of facts that is specific to the test mode.

type TextEdit

type TextEdit struct {
	Position token.Position
	End      token.Position
	NewText  []byte
}