runner

Documentation

Overview

Package runner implements a go/analysis runner. It makes heavy use of on-disk caching to reduce overall memory usage and to speed up repeat runs.

Public API

A Runner maps a list of analyzers and package patterns to a list of results. Results provide access to diagnostics, directives, errors encountered, and information about packages. Results explicitly do not contain ASTs or type information. All position information is returned in the form of token.Position, not token.Pos. All work that requires access to the loaded representation of a package has to occur inside analyzers.

Planning and execution

Analyzing packages is split into two phases: planning and execution.

During planning, a directed acyclic graph of package dependencies is computed. We materialize the full graph so that we can execute the graph from the bottom up, without keeping unnecessary data in memory during a DFS and with simplified parallel execution.

During execution, leaf nodes (nodes with no outstanding dependencies) get executed in parallel, bounded by a semaphore sized according to the number of CPUs. Conceptually, this happens in a loop, processing new leaf nodes as they appear, until no more nodes are left. In the actual implementation, nodes know their dependents, and the last dependency of a node to be processed is responsible for scheduling its dependent.

The graph is rooted at a synthetic root node. Upon execution of the root node, the algorithm terminates.

Analyzing a package repeats the same planning + execution steps, but this time on a graph of analyzers for the package. Parallel execution of individual analyzers is bounded by the same semaphore as executing packages.

Parallelism

Actions are executed in parallel where the dependency graph allows. Overall parallelism is bounded by a semaphore, sized according to GOMAXPROCS. Each concurrently processed package takes up a token, as does each analyzer – but a package can always execute at least one analyzer, using the package's token.

Depending on the overall shape of the graph, there may be GOMAXPROCS packages running a single analyzer each, a single package running GOMAXPROCS analyzers, or anything in between.

Total memory consumption grows roughly linearly with the number of CPUs, while total execution time is inversely proportional to the number of CPUs. Overall, parallelism is affected by the shape of the dependency graph. A lot of inter-connected packages will see less parallelism than a lot of independent packages.

Caching

The runner caches facts, directives and diagnostics in a content-addressable cache that is designed after Go's own cache. Additionally, it makes use of Go's export data.

This cache not only speeds up repeat runs, it also reduces peak memory usage. When we've analyzed a package, we cache the results and drop them from memory. When a dependent needs any of this information, or when analysis is complete and we wish to render the results, the data gets loaded from disk again.

Data only exists in memory when it is immediately needed, not retained for possible future uses. This trades increased CPU usage for reduced memory usage. A single dependency may be loaded many times over, but it greatly reduces peak memory usage, as an arbitrary amount of time may pass between analyzing a dependency and its dependent, during which other packages will be processed.

Index

• Constants
• type Diagnostic
• type RelatedInformation
• type Result
  • func (r Result) Load() (ResultData, error)
• type ResultData
• type Runner
  • func New(cfg config.Config) (*Runner, error)
  • func (r *Runner) ActiveWorkers() int
  • func (r *Runner) Run(cfg *packages.Config, analyzers []*analysis.Analyzer, patterns []string) ([]Result, error)
  • func (r *Runner) TotalWorkers() int
• type SerializedDirective
• type Stats
  • func (s *Stats) InitialPackages() int
  • func (s *Stats) ProcessedInitialPackages() int
  • func (s *Stats) ProcessedPackages() int
  • func (s *Stats) State() int
  • func (s *Stats) TotalPackages() int
• type SuggestedFix
• type TextEdit

Constants

const (
	StateInitializing = iota
	StateLoadPackageGraph
	StateBuildActionGraph
	StateProcessing
	StateFinalizing
)

Types

type Diagnostic

type Diagnostic struct {
	Position token.Position
	End      token.Position
	Category string
	Message  string

	SuggestedFixed []SuggestedFix
	Related        []RelatedInformation
}

type RelatedInformation

type RelatedInformation struct {
	Position token.Position
	End      token.Position
	Message  string
}

RelatedInformation provides additional context for a diagnostic.

type Result

type Result struct {
	Package *loader.PackageSpec
	Config  config.Config
	Initial bool
	Skipped bool

	Failed bool
	Errors []error
	// contains filtered or unexported fields
}

A Result describes the result of analyzing a single package.

It holds references to cached diagnostics and directives. They can be loaded on demand with Diagnostics and Directives respectively.

func (Result) Load

func (r Result) Load() (ResultData, error)

type ResultData

type ResultData struct {
	Directives  []SerializedDirective
	Diagnostics []Diagnostic
	Unused      unused.SerializedResult
}

type Runner

type Runner struct {
	Stats     Stats
	GoVersion int
	// contains filtered or unexported fields
}

A Runner executes analyzers on packages.

func New

func New(cfg config.Config) (*Runner, error)

New returns a new Runner.

func (*Runner) ActiveWorkers

func (r *Runner) ActiveWorkers() int

ActiveWorkers returns the number of currently running workers.

func (*Runner) Run

func (r *Runner) Run(cfg *packages.Config, analyzers []*analysis.Analyzer, patterns []string) ([]Result, error)

Run loads the packages specified by patterns, runs analyzers on them and returns the results. Each result corresponds to a single package. Results will be returned for all packages, including dependencies. Errors specific to packages will be reported in the respective results.

If cfg is nil, a default config will be used. Otherwise, cfg will be used, with the exception of the Mode field.

Run can be called multiple times on the same Runner and it is safe for concurrent use. All runs will share the same semaphore.

func (*Runner) TotalWorkers

func (r *Runner) TotalWorkers() int

TotalWorkers returns the maximum number of possible workers.

type SerializedDirective

type SerializedDirective struct {
	Command   string
	Arguments []string
	// The position of the comment
	DirectivePosition token.Position
	// The position of the node that the comment is attached to
	NodePosition token.Position
}

type Stats

type Stats struct {

	// optional function to call every time an analyzer has finished analyzing a package.
	PrintAnalyzerMeasurement func(*analysis.Analyzer, *loader.PackageSpec, time.Duration)
	// contains filtered or unexported fields
}

func (*Stats) InitialPackages

func (s *Stats) InitialPackages() int

func (*Stats) ProcessedInitialPackages

func (s *Stats) ProcessedInitialPackages() int

func (*Stats) ProcessedPackages

func (s *Stats) ProcessedPackages() int

func (*Stats) State

func (s *Stats) State() int

func (*Stats) TotalPackages

func (s *Stats) TotalPackages() int

type SuggestedFix

type SuggestedFix struct {
	Message   string
	TextEdits []TextEdit
}

type TextEdit

type TextEdit struct {
	Position token.Position
	End      token.Position
	NewText  []byte
}