dep

README

Dep

dep is a prototype dependency management tool for Go. It requires Go 1.8 or newer to compile.

dep is the official experiment, but not yet the official tool. Check out the Roadmap for more on what this means!

Current status

dep is safe for production use. That means two things:

• Any valid metadata file (Gopkg.toml and Gopkg.lock) will be readable and considered valid by any future version of dep.
• The CLI UI is mostly stable. dep init and dep ensure are mostly set; dep status is likely to change a fair bit, and dep prune is going to be absorbed into dep ensure.

That said, keep in mind the following:

• dep init on an existing project can be a rocky experience - we try to automatically convert from other tools' metadata files, and that process is often complex and murky. Once your project is converted and you're using dep ensure, its behavior is quite stable.
• dep still has nasty bugs, but in general these are comparable or fewer to other tools out there.
• dep is pretty slow right now, especially on the first couple times you run it. Just know that there is a lot of headroom for improvement, and work is actively underway.
• dep is still changing rapidly. If you need stability (e.g. for CI), it's best to rely on a released version, not tip.
• dep's exported API interface will continue to change in unpredictable, backwards-incompatible ways until we tag a v1.0.0 release.

Context

• The Saga of Go Dependency Management
• Official Google Docs
  • Go Packaging Proposal Process
  • User Stories
  • Features
  • Design Space
• Frequently Asked Questions

Setup

Grab the latest binary from the releases page.

On macOS you can install or upgrade to the latest released version with Homebrew:

$ brew install dep
$ brew upgrade dep

If you're interested in hacking on dep, you can install via go get:

go get -u github.com/golang/dep/cmd/dep

To start managing dependencies using dep, run the following from your project's root directory:

$ dep init

This does the following:

1. Look for existing dependency management files to convert
2. Check if your dependencies use dep
3. Identify your dependencies
4. Back up your existing vendor/ directory (if you have one) to _vendor-TIMESTAMP/
5. Pick the highest compatible version for each dependency
6. Generate Gopkg.toml ("manifest") and Gopkg.lock files
7. Install the dependencies in vendor/

Usage

There is one main subcommand you will use: dep ensure. ensure first checks that Gopkg.lock is consistent with Gopkg.toml and the imports in your code. If any changes are detected, dep's solver works out a new Gopkg.lock. Then, dep checks if the contents of vendor/ are what Gopkg.lock (the new one if applicable, else the existing one) says it should be, and rewrites vendor/ as needed to bring it into line.

In essence, dep ensure works in two phases to keep four buckets of state in sync:

Note: until we ship vendor verification, we can't efficiently perform the Gopkg.lock <-> vendor/ comparison, so dep ensure unconditionally regenerates all of vendor/ to be safe.

dep ensure is safe to run early and often. See the help text for more detailed usage instructions.

$ dep help ensure

Installing dependencies

(if your vendor/ directory isn't checked in with your code)

$ dep ensure

If a dependency already exists in your vendor/ folder, dep will ensure it matches the constraints from the manifest. If the dependency is missing from vendor/, the latest version allowed by your manifest will be installed.

Adding a dependency

$ dep ensure -add github.com/foo/bar

This adds a version constraint to your Gopkg.toml, and updates Gopkg.lock and vendor/. Now, import and use the package in your code! ✨

dep ensure -add has some subtle behavior variations depending on the project or package named, and the state of your tree. See dep ensure -examples for more information.

Changing dependencies

If you want to:

• Change the allowed version/branch/revision
• Switch to using a fork

for one or more dependencies, do the following:

1. Manually edit your Gopkg.toml.

2. Run

   $ dep ensure
   

Checking the status of dependencies

Run dep status to see the current status of all your dependencies.

$ dep status
PROJECT                             CONSTRAINT     VERSION        REVISION  LATEST
github.com/Masterminds/semver       branch 2.x     branch 2.x     139cc09   c2e7f6c
github.com/Masterminds/vcs          ^1.11.0        v1.11.1        3084677   3084677
github.com/armon/go-radix           *              branch master  4239b77   4239b77

On top of that, if you have added new imports to your project or modified Gopkg.toml without running dep ensure again, dep status will tell you there is a mismatch between Gopkg.lock and the current status of the project.

$ dep status
Lock inputs-digest mismatch due to the following packages missing from the lock:

PROJECT                         MISSING PACKAGES
github.com/Masterminds/goutils  [github.com/Masterminds/goutils]

This happens when a new import is added. Run `dep ensure` to install the missing packages.

As dep status suggests, run dep ensure to update your lockfile. Then run dep status again, and the lock mismatch should go away.

Visualizing dependencies

Generate a visual representation of the dependency tree by piping the output of dep status -dot to graphviz.

Linux

$ sudo apt-get install graphviz
$ dep status -dot | dot -T png | display

MacOS

$ brew install graphviz
$ dep status -dot | dot -T png | open -f -a /Applications/Preview.app

Windows

> choco install graphviz.portable
> dep status -dot | dot -T png -o status.png; start status.png

Updating dependencies

Updating brings the version of a dependency in Gopkg.lock and vendor/ to the latest version allowed by the constraints in Gopkg.toml.

You can update just a targeted subset of dependencies (recommended):

$ dep ensure -update github.com/some/project github.com/other/project
$ dep ensure -update github.com/another/project

Or you can update all your dependencies at once:

$ dep ensure -update

"Latest" means different things depending on the type of constraint in use. If you're depending on a branch, dep will update to the latest tip of that branch. If you're depending on a version using a semver range, it will update to the latest version in that range.

Removing dependencies

1. Remove the imports and all usage from your code.

2. Remove [[constraint]] rules from Gopkg.toml (if any).

3. Run

   $ dep ensure
   

Testing changes to a dependency

Making changes in your vendor/ directory directly is not recommended, as dep will overwrite any changes. Instead:

1. Delete the dependency from the vendor/ directory.

   rm -rf vendor/<dependency>
   

2. Add that dependency to your GOPATH, if it isn't already.

   $ go get <dependency>
   

3. Modify the dependency in $GOPATH/src/<dependency>.

4. Test, build, etc.

Don't run dep ensure until you're done. dep ensure will reinstall the dependency into vendor/ based on your manifest, as if you were installing from scratch.

This solution works for short-term use, but for something long-term, take a look at virtualgo.

To test out code that has been pushed as a new version, or to a branch or fork, see changing dependencies.

Semantic Versioning

dep ensure a uses an external semver library to interpret the version constraints you specify in the manifest. The comparison operators are:

• =: equal
• !=: not equal
• >: greater than
• <: less than
• >=: greater than or equal to
• <=: less than or equal to
• -: literal range. Eg: 1.2 - 1.4.5 is equivalent to >= 1.2, <= 1.4.5
• ~: minor range. Eg: ~1.2.3 is equivalent to >= 1.2.3, < 1.3.0
• ^: major range. Eg: ^1.2.3 is equivalent to >= 1.2.3, < 2.0.0
• [xX*]: wildcard. Eg: 1.2.x is equivalent to >= 1.2.0, < 1.3.0

You might, for example, include a constraint in your manifest that specifies version = "=2.0.0" to pin a dependency to version 2.0.0, or constrain to minor releases with: version = "2.*". Refer to the semver library documentation for more info.

Note: When you specify a version without an operator, dep automatically uses the ^ operator by default. dep ensure will interpret the given version as the min-boundry of a range, for example:

• 1.2.3 becomes the range >=1.2.3, <2.0.0
• 0.2.3 becomes the range >=0.2.3, <0.3.0
• 0.0.3 becomes the range >=0.0.3, <0.1.0

Feedback

Feedback is greatly appreciated. At this stage, the maintainers are most interested in feedback centered on the user experience (UX) of the tool. Do you have workflows that the tool supports well, or doesn't support at all? Do any of the commands have surprising effects, output, or results? Please check the existing issues and FAQ to see if your feedback has already been reported. If not, please file an issue, describing what you did or wanted to do, what you expected to happen, and what actually happened.

Contributing

Contributions are greatly appreciated. The maintainers actively manage the issues list, and try to highlight issues suitable for newcomers. The project follows the typical GitHub pull request model. See CONTRIBUTING.md for more details. Before starting any work, please either comment on an existing issue, or file a new one.

Documentation

Overview

Package dep is a prototype dependency management library.

Index

• Constants
• func BackupVendor(vpath, suffix string) (string, error)
• func ValidateProjectRoots(c *Ctx, m *Manifest, sm gps.SourceManager) error
• type Analyzer
  • func (a Analyzer) DeriveManifestAndLock(path string, n gps.ProjectRoot) (gps.Manifest, gps.Lock, error)
  • func (a Analyzer) HasDepMetadata(path string) bool
  • func (a Analyzer) Info() gps.ProjectAnalyzerInfo
• type Ctx
  • func (c *Ctx) AbsForImport(path string) (string, error)
  • func (c *Ctx) DetectProjectGOPATH(p *Project) (string, error)
  • func (c *Ctx) ImportForAbs(path string) (string, error)
  • func (c *Ctx) LoadProject() (*Project, error)
  • func (c *Ctx) SetPaths(wd string, GOPATHs ...string) error
  • func (c *Ctx) SourceManager() (*gps.SourceMgr, error)
  • func (c *Ctx) ValidateParams(sm gps.SourceManager, params gps.SolveParameters) error
• type Lock
  • func LockFromSolution(in gps.Solution) *Lock
  • func (l *Lock) HasProjectWithRoot(root gps.ProjectRoot) bool
  • func (l *Lock) InputsDigest() []byte
  • func (l *Lock) MarshalTOML() ([]byte, error)
  • func (l *Lock) Projects() []gps.LockedProject
• type Manifest
  • func NewManifest() *Manifest
  • func (m *Manifest) DependencyConstraints() gps.ProjectConstraints
  • func (m *Manifest) HasConstraintsOn(root gps.ProjectRoot) bool
  • func (m *Manifest) IgnoredPackages() *pkgtree.IgnoredRuleset
  • func (m *Manifest) MarshalTOML() ([]byte, error)
  • func (m *Manifest) Overrides() gps.ProjectConstraints
  • func (m *Manifest) RequiredPackages() map[string]bool
• type Project
  • func (p *Project) MakeParams() gps.SolveParameters
  • func (p *Project) ParseRootPackageTree() (pkgtree.PackageTree, error)
  • func (p *Project) SetRoot(root string) error
• type SafeWriter
  • func NewSafeWriter(manifest *Manifest, oldLock, newLock *Lock, vendor VendorBehavior) (*SafeWriter, error)
  • func (sw *SafeWriter) HasLock() bool
  • func (sw *SafeWriter) HasManifest() bool
  • func (sw *SafeWriter) PrintPreparedActions(output *log.Logger, verbose bool) error
  • func (sw *SafeWriter) Write(root string, sm gps.SourceManager, examples bool, logger *log.Logger) error
• type SolveMeta
• type VendorBehavior

Constants

const LockName = "Gopkg.lock"

LockName is the lock file name used by dep.

const ManifestName = "Gopkg.toml"

ManifestName is the manifest file name used by dep.

Functions

func BackupVendor

func BackupVendor(vpath, suffix string) (string, error)

BackupVendor looks for existing vendor directory and if it's not empty, creates a backup of it to a new directory with the provided suffix.

func ValidateProjectRoots

func ValidateProjectRoots(c *Ctx, m *Manifest, sm gps.SourceManager) error

ValidateProjectRoots validates the project roots present in manifest.

Types

type Analyzer

type Analyzer struct{}

Analyzer implements gps.ProjectAnalyzer.

func (Analyzer) DeriveManifestAndLock

func (a Analyzer) DeriveManifestAndLock(path string, n gps.ProjectRoot) (gps.Manifest, gps.Lock, error)

DeriveManifestAndLock reads and returns the manifest at path/ManifestName or nil if one is not found. The Lock is always nil for now.

func (Analyzer) HasDepMetadata

func (a Analyzer) HasDepMetadata(path string) bool

HasDepMetadata determines if a dep manifest exists at the specified path.

func (Analyzer) Info

func (a Analyzer) Info() gps.ProjectAnalyzerInfo

Info returns Analyzer's name and version info.

type Ctx

type Ctx struct {
	WorkingDir     string      // Where to execute.
	GOPATH         string      // Selected Go path, containing WorkingDir.
	GOPATHs        []string    // Other Go paths.
	Out, Err       *log.Logger // Required loggers.
	Verbose        bool        // Enables more verbose logging.
	DisableLocking bool        // When set, no lock file will be created to protect against simultaneous dep processes.
}

Ctx defines the supporting context of dep.

A properly initialized Ctx has a GOPATH containing the project root and non-nil Loggers.

ctx := &dep.Ctx{
	WorkingDir: GOPATH + "/src/project/root",
	GOPATH: GOPATH,
	Out: log.New(os.Stdout, "", 0),
	Err: log.New(os.Stderr, "", 0),
}

Ctx.DetectProjectGOPATH() helps with setting the containing GOPATH.

ctx.GOPATH, err := Ctx.DetectProjectGOPATH(project)
if err != nil {
	// Could not determine which GOPATH to use for the project.
}

func (*Ctx) AbsForImport

func (c *Ctx) AbsForImport(path string) (string, error)

AbsForImport returns the absolute path for the project root including the $GOPATH. This will not work with stdlib packages and the package directory needs to exist.

func (*Ctx) DetectProjectGOPATH

func (c *Ctx) DetectProjectGOPATH(p *Project) (string, error)

DetectProjectGOPATH attempt to find the GOPATH containing the project.

If p.AbsRoot is not a symlink and is within a GOPATH, the GOPATH containing p.AbsRoot is returned.
If p.AbsRoot is a symlink and is not within any known GOPATH, the GOPATH containing p.ResolvedAbsRoot is returned.

p.AbsRoot is assumed to be a symlink if it is not the same as p.ResolvedAbsRoot.

DetectProjectGOPATH will return an error in the following cases:

If p.AbsRoot is not a symlink and is not within any known GOPATH.
If neither p.AbsRoot nor p.ResolvedAbsRoot are within a known GOPATH.
If both p.AbsRoot and p.ResolvedAbsRoot are within the same GOPATH.
If p.AbsRoot and p.ResolvedAbsRoot are each within a different GOPATH.

func (*Ctx) ImportForAbs

func (c *Ctx) ImportForAbs(path string) (string, error)

ImportForAbs returns the import path for an absolute project path by trimming the `$GOPATH/src/` prefix. Returns an error for paths equal to, or without this prefix.

func (*Ctx) LoadProject

func (c *Ctx) LoadProject() (*Project, error)

LoadProject starts from the current working directory and searches up the directory tree for a project root. The search stops when a file with the name ManifestName (Gopkg.toml, by default) is located.

The Project contains the parsed manifest as well as a parsed lock file, if present. The import path is calculated as the remaining path segment below Ctx.GOPATH/src.

func (*Ctx) SetPaths

func (c *Ctx) SetPaths(wd string, GOPATHs ...string) error

SetPaths sets the WorkingDir and GOPATHs fields. If GOPATHs is empty, then the GOPATH environment variable (or the default GOPATH) is used instead.

func (*Ctx) SourceManager

func (c *Ctx) SourceManager() (*gps.SourceMgr, error)

SourceManager produces an instance of gps's built-in SourceManager initialized to log to the receiver's logger.

func (*Ctx) ValidateParams

func (c *Ctx) ValidateParams(sm gps.SourceManager, params gps.SolveParameters) error

ValidateParams ensure that solving can be completed with the specified params.

type Lock

type Lock struct {
	SolveMeta SolveMeta
	P         []gps.LockedProject
}

Lock holds lock file data and implements gps.Lock.

func LockFromSolution

func LockFromSolution(in gps.Solution) *Lock

LockFromSolution converts a gps.Solution to dep's representation of a lock.

Data is defensively copied wherever necessary to ensure the resulting *lock shares no memory with the original lock.

func (*Lock) HasProjectWithRoot

func (l *Lock) HasProjectWithRoot(root gps.ProjectRoot) bool

HasProjectWithRoot checks if the lock contains a project with the provided ProjectRoot.

This check is O(n) in the number of projects.

func (*Lock) InputsDigest

func (l *Lock) InputsDigest() []byte

InputsDigest returns the hash of inputs which produced this lock data.

func (*Lock) MarshalTOML

func (l *Lock) MarshalTOML() ([]byte, error)

MarshalTOML serializes this lock into TOML via an intermediate raw form.

func (*Lock) Projects

func (l *Lock) Projects() []gps.LockedProject

Projects returns the list of LockedProjects contained in the lock data.

type Manifest

type Manifest struct {
	Constraints gps.ProjectConstraints
	Ovr         gps.ProjectConstraints
	Ignored     []string
	Required    []string
}

Manifest holds manifest file data and implements gps.RootManifest.

func NewManifest

func NewManifest() *Manifest

NewManifest instantiates a new manifest.

func (*Manifest) DependencyConstraints

func (m *Manifest) DependencyConstraints() gps.ProjectConstraints

DependencyConstraints returns a list of project-level constraints.

func (*Manifest) HasConstraintsOn

func (m *Manifest) HasConstraintsOn(root gps.ProjectRoot) bool

HasConstraintsOn checks if the manifest contains either constraints or overrides on the provided ProjectRoot.

func (*Manifest) IgnoredPackages

func (m *Manifest) IgnoredPackages() *pkgtree.IgnoredRuleset

IgnoredPackages returns a set of import paths to ignore.

func (*Manifest) MarshalTOML

func (m *Manifest) MarshalTOML() ([]byte, error)

MarshalTOML serializes this manifest into TOML via an intermediate raw form.

func (*Manifest) Overrides

func (m *Manifest) Overrides() gps.ProjectConstraints

Overrides returns a list of project-level override constraints.

func (*Manifest) RequiredPackages

func (m *Manifest) RequiredPackages() map[string]bool

RequiredPackages returns a set of import paths to require.

type Project

type Project struct {
	// AbsRoot is the absolute path to the root directory of the project.
	AbsRoot string
	// ResolvedAbsRoot is the resolved absolute path to the root directory of the project.
	// If AbsRoot is not a symlink, then ResolvedAbsRoot should equal AbsRoot.
	ResolvedAbsRoot string
	// ImportRoot is the import path of the project's root directory.
	ImportRoot gps.ProjectRoot
	Manifest   *Manifest
	Lock       *Lock // Optional
}

A Project holds a Manifest and optional Lock for a project.

func (*Project) MakeParams

func (p *Project) MakeParams() gps.SolveParameters

MakeParams is a simple helper to create a gps.SolveParameters without setting any nils incorrectly.

func (*Project) ParseRootPackageTree

func (p *Project) ParseRootPackageTree() (pkgtree.PackageTree, error)

ParseRootPackageTree analyzes the root project's disk contents to create a PackageTree, trimming out packages that are not relevant for root projects along the way.

func (*Project) SetRoot

func (p *Project) SetRoot(root string) error

SetRoot sets the project AbsRoot and ResolvedAbsRoot. If root is a not symlink, ResolvedAbsRoot will be set to root.

type SafeWriter

type SafeWriter struct {
	Manifest *Manifest
	// contains filtered or unexported fields
}

SafeWriter transactionalizes writes of manifest, lock, and vendor dir, both individually and in any combination, into a pseudo-atomic action with transactional rollback.

It is not impervious to errors (writing to disk is hard), but it should guard against non-arcane failure conditions.

func NewSafeWriter

func NewSafeWriter(manifest *Manifest, oldLock, newLock *Lock, vendor VendorBehavior) (*SafeWriter, error)

NewSafeWriter sets up a SafeWriter to write a set of manifest, lock, and vendor tree.

- If manifest is provided, it will be written to the standard manifest file name beneath root.

- If newLock is provided, it will be written to the standard lock file name beneath root.

- If vendor is VendorAlways, or is VendorOnChanged and the locks are different, the vendor directory will be written beneath root based on newLock.

- If oldLock is provided without newLock, error.

- If vendor is VendorAlways without a newLock, error.

func (*SafeWriter) HasLock

func (sw *SafeWriter) HasLock() bool

HasLock checks if a Lock is present in the SafeWriter

func (*SafeWriter) HasManifest

func (sw *SafeWriter) HasManifest() bool

HasManifest checks if a Manifest is present in the SafeWriter

func (*SafeWriter) PrintPreparedActions

func (sw *SafeWriter) PrintPreparedActions(output *log.Logger, verbose bool) error

PrintPreparedActions logs the actions a call to Write would perform.

func (*SafeWriter) Write

func (sw *SafeWriter) Write(root string, sm gps.SourceManager, examples bool, logger *log.Logger) error

Write saves some combination of config yaml, lock, and a vendor tree. root is the absolute path of root dir in which to write. sm is only required if vendor is being written.

It first writes to a temp dir, then moves them in place if and only if all the write operations succeeded. It also does its best to roll back if any moves fail. This mostly guarantees that dep cannot exit with a partial write that would leave an undefined state on disk.

type SolveMeta

type SolveMeta struct {
	InputsDigest    []byte
	AnalyzerName    string
	AnalyzerVersion int
	SolverName      string
	SolverVersion   int
}

SolveMeta holds solver meta data.

type VendorBehavior

type VendorBehavior int

VendorBehavior defines when the vendor directory should be written.

const (
	// VendorOnChanged indicates that the vendor directory should be written when the lock is new or changed.
	VendorOnChanged VendorBehavior = iota
	// VendorAlways forces the vendor directory to always be written.
	VendorAlways
	// VendorNever indicates the vendor directory should never be written.
	VendorNever
)