pkgtree

Documentation

Index

• func DigestFromDirectory(osDirname string) ([]byte, error)
• func DirWalk(osDirname string, walkFn DirWalkFunc) error
• func VerifyDepTree(osDirname string, wantSums map[string][]byte) (map[string]VendorStatus, error)
• type ConflictingImportComments
  • func (e *ConflictingImportComments) Error() string
• type DirWalkFunc
• type IgnoredRuleset
  • func NewIgnoredRuleset(ig []string) *IgnoredRuleset
  • func (ir *IgnoredRuleset) IsIgnored(path string) bool
  • func (ir *IgnoredRuleset) Len() int
  • func (ir *IgnoredRuleset) ToSlice() []string
• type LocalImportsError
  • func (e *LocalImportsError) Error() string
• type NonCanonicalImportRoot
  • func (e *NonCanonicalImportRoot) Error() string
• type Package
• type PackageOrErr
• type PackageTree
  • func ListPackages(fileRoot, importRoot string) (PackageTree, error)
  • func (t PackageTree) Copy() PackageTree
  • func (t PackageTree) ToReachMap(main, tests, backprop bool, ignore *IgnoredRuleset) (ReachMap, map[string]*ProblemImportError)
  • func (t PackageTree) TrimHiddenPackages(main, tests bool, ignore *IgnoredRuleset) PackageTree
• type ProblemImportError
  • func (e *ProblemImportError) Error() string
• type ReachMap
  • func (rm ReachMap) FlattenFn(exclude func(string) bool) []string
• type VendorStatus
  • func (ls VendorStatus) String() string

Functions

func DigestFromDirectory

func DigestFromDirectory(osDirname string) ([]byte, error)

DigestFromDirectory returns a hash of the specified directory contents, which will match the hash computed for any directory on any supported Go platform whose contents exactly match the specified directory.

This function ignores any file system node named `vendor`, `.bzr`, `.git`, `.hg`, and `.svn`, as these are typically used as Version Control System (VCS) directories.

Other than the `vendor` and VCS directories mentioned above, the calculated hash includes the pathname to every discovered file system node, whether it is an empty directory, a non-empty directory, empty file, non-empty file, or symbolic link. If a symbolic link, the referent name is included. If a non-empty file, the file's contents are included. If a non-empty directory, the contents of the directory are included.

While filepath.Walk could have been used, that standard library function skips symbolic links, and for now, we want the hash to include the symbolic link referents.

func DirWalk

func DirWalk(osDirname string, walkFn DirWalkFunc) error

DirWalk walks the file tree rooted at osDirname, calling for each file system node in the tree, including root. All errors that arise visiting nodes are filtered by walkFn. The nodes are walked in lexical order, which makes the output deterministic but means that for very large directories DirWalk can be inefficient. Unlike filepath.Walk, DirWalk does follow symbolic links.

func VerifyDepTree

func VerifyDepTree(osDirname string, wantSums map[string][]byte) (map[string]VendorStatus, error)

VerifyDepTree verifies a dependency tree according to expected digest sums, and returns an associative array of file system nodes and their respective vendor status conditions.

The keys to the expected digest sums associative array represent the project's dependencies, and each is required to be expressed using the solidus character, `/`, as its path separator. For example, even on a GOOS platform where the file system path separator is a character other than solidus, one particular dependency would be represented as "github.com/alice/alice1".

Types

type ConflictingImportComments

type ConflictingImportComments struct {
	ImportPath                string   // An import path referring to this package
	ConflictingImportComments []string // All distinct "canonical" paths encountered in the package files
}

ConflictingImportComments indicates that the package declares more than one different canonical path.

func (*ConflictingImportComments) Error

func (e *ConflictingImportComments) Error() string

type DirWalkFunc

type DirWalkFunc func(osPathname string, info os.FileInfo, err error) error

DirWalkFunc is the type of the function called for each file system node visited by DirWalk. The path argument contains the argument to DirWalk as a prefix; that is, if DirWalk is called with "dir", which is a directory containing the file "a", the walk function will be called with the argument "dir/a", using the correct os.PathSeparator for the Go Operating System architecture, GOOS. The info argument is the os.FileInfo for the named path.

If there was a problem walking to the file or directory named by path, the incoming error will describe the problem and the function can decide how to handle that error (and DirWalk will not descend into that directory). If an error is returned, processing stops. The sole exception is when the function returns the special value filepath.SkipDir. If the function returns filepath.SkipDir when invoked on a directory, DirWalk skips the directory's contents entirely. If the function returns filepath.SkipDir when invoked on a non-directory file system node, DirWalk skips the remaining files in the containing directory.

type IgnoredRuleset

type IgnoredRuleset struct {
	// contains filtered or unexported fields
}

IgnoredRuleset comprises a set of rules for ignoring import paths. It can manage both literal and prefix-wildcard matches.

func NewIgnoredRuleset

func NewIgnoredRuleset(ig []string) *IgnoredRuleset

NewIgnoredRuleset processes a set of strings into an IgnoredRuleset. Strings that end in "*" are treated as wildcards, where any import path with a matching prefix will be ignored. IgnoredRulesets are immutable once created.

Duplicate and redundant (i.e. a literal path that has a prefix of a wildcard path) declarations are discarded. Consequently, it is possible that the returned IgnoredRuleset may have a smaller Len() than the input slice.

func (*IgnoredRuleset) IsIgnored

func (ir *IgnoredRuleset) IsIgnored(path string) bool

IsIgnored indicates whether the provided path should be ignored, according to the ruleset.

func (*IgnoredRuleset) Len

func (ir *IgnoredRuleset) Len() int

Len indicates the number of rules in the ruleset.

func (*IgnoredRuleset) ToSlice

func (ir *IgnoredRuleset) ToSlice() []string

ToSlice converts the contents of the IgnoredRuleset to a string slice.

This operation is symmetrically dual to NewIgnoredRuleset.

type LocalImportsError

type LocalImportsError struct {
	ImportPath   string
	Dir          string
	LocalImports []string
}

LocalImportsError indicates that a package contains at least one relative import that will prevent it from compiling.

TODO(sdboyer) add a Files property once we're doing our own per-file parsing

func (*LocalImportsError) Error

func (e *LocalImportsError) Error() string

type NonCanonicalImportRoot

type NonCanonicalImportRoot struct {
	ImportRoot string // A root path that is being used to import a package
	Canonical  string // A canonical path declared by the package being imported
}

NonCanonicalImportRoot reports the situation when the dependee imports a package via something other than the package's declared canonical path.

func (*NonCanonicalImportRoot) Error

func (e *NonCanonicalImportRoot) Error() string

type Package

type Package struct {
	Name        string   // Package name, as declared in the package statement
	ImportPath  string   // Full import path, including the prefix provided to ListPackages()
	CommentPath string   // Import path given in the comment on the package statement
	Imports     []string // Imports from all go and cgo files
	TestImports []string // Imports from all go test files (in go/build parlance: both TestImports and XTestImports)
}

Package represents a Go package. It contains a subset of the information go/build.Package does.

type PackageOrErr

type PackageOrErr struct {
	P   Package
	Err error
}

PackageOrErr stores the results of attempting to parse a single directory for Go source code.

type PackageTree

type PackageTree struct {
	ImportRoot string
	Packages   map[string]PackageOrErr
}

A PackageTree represents the results of recursively parsing a tree of packages, starting at the ImportRoot. The results of parsing the files in the directory identified by each import path - a Package or an error - are stored in the Packages map, keyed by that import path.

func ListPackages

func ListPackages(fileRoot, importRoot string) (PackageTree, error)

ListPackages reports Go package information about all directories in the tree at or below the provided fileRoot.

The importRoot parameter is prepended to the relative path when determining the import path for each package. The obvious case is for something typical, like:

fileRoot = "/home/user/go/src/github.com/foo/bar"
importRoot = "github.com/foo/bar"

where the fileRoot and importRoot align. However, if you provide:

fileRoot = "/home/user/workspace/path/to/repo"
importRoot = "github.com/foo/bar"

then the root package at path/to/repo will be ascribed import path "github.com/foo/bar", and the package at "/home/user/workspace/path/to/repo/baz" will be "github.com/foo/bar/baz".

A PackageTree is returned, which contains the ImportRoot and map of import path to PackageOrErr - each path under the root that exists will have either a Package, or an error describing why the directory is not a valid package.

func (PackageTree) Copy

func (t PackageTree) Copy() PackageTree

Copy copies the PackageTree.

This is really only useful as a defensive measure to prevent external state mutations.

func (PackageTree) ToReachMap

func (t PackageTree) ToReachMap(main, tests, backprop bool, ignore *IgnoredRuleset) (ReachMap, map[string]*ProblemImportError)

ToReachMap looks through a PackageTree and computes the list of external import statements (that is, import statements pointing to packages that are not logical children of PackageTree.ImportRoot) that are transitively imported by the internal packages in the tree.

main indicates whether (true) or not (false) to include main packages in the analysis. When utilized by gps' solver, main packages are generally excluded from analyzing anything other than the root project, as they necessarily can't be imported.

tests indicates whether (true) or not (false) to include imports from test files in packages when computing the reach map.

backprop indicates whether errors (an actual PackageOrErr.Err, or an import to a nonexistent internal package) should be backpropagated, transitively "poisoning" all corresponding importers to all importers.

ignore is a map of import paths that, if encountered, should be excluded from analysis. This exclusion applies to both internal and external packages. If an external import path is ignored, it is simply omitted from the results.

If an internal path is ignored, then it not only does not appear in the final map, but it is also excluded from the transitive calculations of other internal packages. That is, if you ignore A/foo, then the external package list for all internal packages that import A/foo will not include external packages that are only reachable through A/foo.

Visually, this means that, given a PackageTree with root A and packages at A, A/foo, and A/bar, and the following import chain:

A -> A/foo -> A/bar -> B/baz

In this configuration, all of A's packages transitively import B/baz, so the returned map would be:

 map[string][]string{
	"A": []string{"B/baz"},
	"A/foo": []string{"B/baz"}
	"A/bar": []string{"B/baz"},
 }

However, if you ignore A/foo, then A's path to B/baz is broken, and A/foo is omitted entirely. Thus, the returned map would be:

 map[string][]string{
	"A": []string{},
	"A/bar": []string{"B/baz"},
 }

If there are no packages to ignore, it is safe to pass a nil map.

Finally, if an internal PackageOrErr contains an error, it is always omitted from the result set. If backprop is true, then the error from that internal package will be transitively propagated back to any other internal PackageOrErrs that import it, causing them to also be omitted. So, with the same import chain:

A -> A/foo -> A/bar -> B/baz

If A/foo has an error, then it would backpropagate to A, causing both to be omitted, and the returned map to contain only A/bar:

 map[string][]string{
	"A/bar": []string{"B/baz"},
 }

If backprop is false, then errors will not backpropagate to internal importers. So, with an error in A/foo, this would be the result map:

 map[string][]string{
	"A": []string{},
	"A/bar": []string{"B/baz"},
 }

func (PackageTree) TrimHiddenPackages

func (t PackageTree) TrimHiddenPackages(main, tests bool, ignore *IgnoredRuleset) PackageTree

TrimHiddenPackages returns a new PackageTree where packages that are ignored, or both hidden and unreachable, have been removed.

The package list is partitioned into two sets: visible, and hidden, where packages are considered hidden if they are within or beneath directories with:

• leading dots
• leading underscores
• the exact name "testdata"

Packages in the hidden set are dropped from the returned PackageTree, unless they are transitively reachable from imports in the visible set.

The "main", "tests" and "ignored" parameters have the same behavior as with PackageTree.ToReachMap(): the first two determine, respectively, whether imports from main packages, and imports from tests, should be considered for reachability checks. Setting 'main' to true will additionally result in main packages being trimmed.

"ignored" designates import paths, or patterns of import paths, where the corresponding packages should be excluded from reachability checks, if encountered. Ignored packages are also removed from the final set.

Note that it is not recommended to call this method if the goal is to obtain a set of tree-external imports; calling ToReachMap and FlattenFn will achieve the same effect.

type ProblemImportError

type ProblemImportError struct {
	// The import path of the package with some problem rendering it
	// unimportable.
	ImportPath string
	// The path to the internal package the problem package imports that is the
	// original cause of this issue. If empty, the package itself is the
	// problem.
	Cause []string
	// The actual error from ListPackages that is undermining importability for
	// this package.
	Err error
}

ProblemImportError describes the reason that a particular import path is not safely importable.

func (*ProblemImportError) Error

func (e *ProblemImportError) Error() string

Error formats the ProblemImportError as a string, reflecting whether the error represents a direct or transitive problem.

type ReachMap

type ReachMap map[string]struct {
	Internal, External []string
}

ReachMap maps a set of import paths (keys) to the sets of transitively reachable tree-internal packages, and all the tree-external packages reachable through those internal packages.

See PackageTree.ToReachMap() for more information.

func (ReachMap) FlattenFn

func (rm ReachMap) FlattenFn(exclude func(string) bool) []string

FlattenFn flattens a reachmap into a sorted, deduplicated list of all the external imports named by its contained packages, but excludes imports coming from packages with disallowed patterns in their names: any path element with a leading dot, a leading underscore, with the name "testdata".

Imports for which exclude returns true will be left out.

type VendorStatus

type VendorStatus uint8

VendorStatus represents one of a handful of possible status conditions for a particular file system node in the vendor directory tree.

const (
	// NotInLock is used when a file system node exists for which there is no
	// corresponding dependency in the lock file.
	NotInLock VendorStatus = iota

	// NotInTree is used when a lock file dependency exists for which there is
	// no corresponding file system node.
	NotInTree

	// NoMismatch is used when the digest for a dependency listed in the
	// lockfile matches what is calculated from the file system.
	NoMismatch

	// EmptyDigestInLock is used when the digest for a dependency listed in the
	// lock file is the empty string. While this is a special case of
	// DigestMismatchInLock, keeping both cases discrete is a desired feature.
	EmptyDigestInLock

	// DigestMismatchInLock is used when the digest for a dependency listed in
	// the lock file does not match what is calculated from the file system.
	DigestMismatchInLock
)

func (VendorStatus) String

func (ls VendorStatus) String() string