modfetch

Documentation

Index

• Variables
• func CachePath(ctx context.Context, m module.Version, suffix string) (string, error)
• func Download(ctx context.Context, mod module.Version) (dir string, err error)
• func DownloadDir(ctx context.Context, m module.Version) (string, error)
• func DownloadZip(ctx context.Context, mod module.Version) (zipfile string, err error)
• func GoMod(ctx context.Context, path, rev string) ([]byte, error)
• func GoModFile(ctx context.Context, path, version string) (string, error)
• func GoModSum(ctx context.Context, path, version string) (string, error)
• func HaveSum(mod module.Version) bool
• func LegacyGoMod(modPath string) []byte
• func RecordedSum(mod module.Version) (sum string, ok bool)
• func RemoveAll(dir string) error
• func Reset()
• func SideLock(ctx context.Context) (unlock func(), err error)
• func Sum(ctx context.Context, mod module.Version) string
• func TidyGoSum(keep map[module.Version]bool) (before, after []byte)
• func TrimGoSum(keep map[module.Version]bool)
• func TryProxies(f func(proxy string) error) error
• func WriteGoSum(ctx context.Context, keep map[module.Version]bool, readonly bool) error
• type DownloadDirPartialError
  • func (e *DownloadDirPartialError) Error() string
  • func (e *DownloadDirPartialError) Is(err error) bool
• type Repo
  • func Lookup(ctx context.Context, proxy, path string) Repo
• type RevInfo
  • func InfoFile(ctx context.Context, path, version string) (*RevInfo, string, error)
• type Versions

Variables

var ErrGoSumDirty = errors.New("updates to go.sum needed, disabled by -mod=readonly")

var ErrToolchain = errors.New("internal error: invalid operation on toolchain module")

var GoSumFile string // path to go.sum; set by package modload

var HelpGoproxy = &base.Command{
	UsageLine: "goproxy",
	Short:     "module proxy protocol",
	Long: `
A Go module proxy is any web server that can respond to GET requests for
URLs of a specified form. The requests have no query parameters, so even
a site serving from a fixed file system (including a file:/// URL)
can be a module proxy.

For details on the GOPROXY protocol, see
https://golang.org/ref/mod#goproxy-protocol.
`,
}

var HelpModuleAuth = &base.Command{
	UsageLine: "module-auth",
	Short:     "module authentication using go.sum",
	Long: `
When the go command downloads a module zip file or go.mod file into the
module cache, it computes a cryptographic hash and compares it with a known
value to verify the file hasn't changed since it was first downloaded. Known
hashes are stored in a file in the module root directory named go.sum. Hashes
may also be downloaded from the checksum database depending on the values of
GOSUMDB, GOPRIVATE, and GONOSUMDB.

For details, see https://golang.org/ref/mod#authenticating.
`,
}

var HelpPrivate = &base.Command{
	UsageLine: "private",
	Short:     "configuration for downloading non-public code",
	Long: `
The go command defaults to downloading modules from the public Go module
mirror at proxy.golang.org. It also defaults to validating downloaded modules,
regardless of source, against the public Go checksum database at sum.golang.org.
These defaults work well for publicly available source code.

The GOPRIVATE environment variable controls which modules the go command
considers to be private (not available publicly) and should therefore not use
the proxy or checksum database. The variable is a comma-separated list of
glob patterns (in the syntax of Go's path.Match) of module path prefixes.
For example,

	GOPRIVATE=*.corp.example.com,rsc.io/private

causes the go command to treat as private any module with a path prefix
matching either pattern, including git.corp.example.com/xyzzy, rsc.io/private,
and rsc.io/private/quux.

For fine-grained control over module download and validation, the GONOPROXY
and GONOSUMDB environment variables accept the same kind of glob list
and override GOPRIVATE for the specific decision of whether to use the proxy
and checksum database, respectively.

For example, if a company ran a module proxy serving private modules,
users would configure go using:

	GOPRIVATE=*.corp.example.com
	GOPROXY=proxy.example.com
	GONOPROXY=none

The GOPRIVATE variable is also used to define the "public" and "private"
patterns for the GOVCS variable; see 'go help vcs'. For that usage,
GOPRIVATE applies even in GOPATH mode. In that case, it matches import paths
instead of module paths.

The 'go env -w' command (see 'go help env') can be used to set these variables
for future go command invocations.

For more details, see https://golang.org/ref/mod#private-modules.
`,
}

var WorkspaceGoSumFiles []string // path to module go.sums in workspace; set by package modload

Functions

func CachePath

func CachePath(ctx context.Context, m module.Version, suffix string) (string, error)

func Download

func Download(ctx context.Context, mod module.Version) (dir string, err error)

Download downloads the specific module version to the local download cache and returns the name of the directory corresponding to the root of the module's file tree.

func DownloadDir

func DownloadDir(ctx context.Context, m module.Version) (string, error)

DownloadDir returns the directory to which m should have been downloaded. An error will be returned if the module path or version cannot be escaped. An error satisfying errors.Is(err, fs.ErrNotExist) will be returned along with the directory if the directory does not exist or if the directory is not completely populated.

func DownloadZip

func DownloadZip(ctx context.Context, mod module.Version) (zipfile string, err error)

DownloadZip downloads the specific module version to the local zip cache and returns the name of the zip file.

func GoMod

func GoMod(ctx context.Context, path, rev string) ([]byte, error)

GoMod is like Lookup(ctx, path).GoMod(rev) but avoids the repository path resolution in Lookup if the result is already cached on local disk.

func GoModFile

func GoModFile(ctx context.Context, path, version string) (string, error)

GoModFile is like GoMod but returns the name of the file containing the cached information.

func GoModSum

func GoModSum(ctx context.Context, path, version string) (string, error)

GoModSum returns the go.sum entry for the module version's go.mod file. (That is, it returns the entry listed in go.sum as "path version/go.mod".)

func HaveSum

func HaveSum(mod module.Version) bool

HaveSum returns true if the go.sum file contains an entry for mod. The entry's hash must be generated with a known hash algorithm. mod.Version may have a "/go.mod" suffix to distinguish sums for .mod and .zip files.

func LegacyGoMod

func LegacyGoMod(modPath string) []byte

LegacyGoMod generates a fake go.mod file for a module that doesn't have one. The go.mod file contains a module directive and nothing else: no go version, no requirements.

We used to try to build a go.mod reflecting pre-existing package management metadata files, but the conversion was inherently imperfect (because those files don't have exactly the same semantics as go.mod) and, when done for dependencies in the middle of a build, impossible to correct. So we stopped.

func RecordedSum

func RecordedSum(mod module.Version) (sum string, ok bool)

RecordedSum returns the sum if the go.sum file contains an entry for mod. The boolean reports true if an entry was found or false if no entry found or two conflicting sums are found. The entry's hash must be generated with a known hash algorithm. mod.Version may have a "/go.mod" suffix to distinguish sums for .mod and .zip files.

func RemoveAll

func RemoveAll(dir string) error

RemoveAll removes a directory written by Download or Unzip, first applying any permission changes needed to do so.

func Reset

func Reset()

Reset resets globals in the modfetch package, so previous loads don't affect contents of go.sum files.

func SideLock

func SideLock(ctx context.Context) (unlock func(), err error)

SideLock locks a file within the module cache that previously guarded edits to files outside the cache, such as go.sum and go.mod files in the user's working directory. If err is nil, the caller MUST eventually call the unlock function.

func Sum

func Sum(ctx context.Context, mod module.Version) string

Sum returns the checksum for the downloaded copy of the given module, if present in the download cache.

func TidyGoSum

func TidyGoSum(keep map[module.Version]bool) (before, after []byte)

TidyGoSum returns a tidy version of the go.sum file. A missing go.sum file is treated as if empty.

func TrimGoSum

func TrimGoSum(keep map[module.Version]bool)

TrimGoSum trims go.sum to contain only the modules needed for reproducible builds.

keep is used to check whether a sum should be retained in go.mod. It should have entries for both module content sums and go.mod sums (version ends with "/go.mod").

func TryProxies

func TryProxies(f func(proxy string) error) error

TryProxies iterates f over each configured proxy (including "noproxy" and "direct" if applicable) until f returns no error or until f returns an error that is not equivalent to fs.ErrNotExist on a proxy configured not to fall back on errors.

TryProxies then returns that final error.

If GOPROXY is set to "off", TryProxies invokes f once with the argument "off".

func WriteGoSum

func WriteGoSum(ctx context.Context, keep map[module.Version]bool, readonly bool) error

WriteGoSum writes the go.sum file if it needs to be updated.

keep is used to check whether a newly added sum should be saved in go.sum. It should have entries for both module content sums and go.mod sums (version ends with "/go.mod"). Existing sums will be preserved unless they have been marked for deletion with TrimGoSum.

Types

type DownloadDirPartialError

type DownloadDirPartialError struct {
	Dir string
	Err error
}

DownloadDirPartialError is returned by DownloadDir if a module directory exists but was not completely populated.

DownloadDirPartialError is equivalent to fs.ErrNotExist.

func (*DownloadDirPartialError) Error

func (e *DownloadDirPartialError) Error() string

func (*DownloadDirPartialError) Is

func (e *DownloadDirPartialError) Is(err error) bool

type Repo

type Repo interface {
	// ModulePath returns the module path.
	ModulePath() string

	// CheckReuse checks whether the validation criteria in the origin
	// are still satisfied on the server corresponding to this module.
	// If so, the caller can reuse any cached Versions or RevInfo containing
	// this origin rather than redownloading those from the server.
	CheckReuse(ctx context.Context, old *codehost.Origin) error

	// Versions lists all known versions with the given prefix.
	// Pseudo-versions are not included.
	//
	// Versions should be returned sorted in semver order
	// (implementations can use semver.Sort).
	//
	// Versions returns a non-nil error only if there was a problem
	// fetching the list of versions: it may return an empty list
	// along with a nil error if the list of matching versions
	// is known to be empty.
	//
	// If the underlying repository does not exist,
	// Versions returns an error matching errors.Is(_, os.NotExist).
	Versions(ctx context.Context, prefix string) (*Versions, error)

	// Stat returns information about the revision rev.
	// A revision can be any identifier known to the underlying service:
	// commit hash, branch, tag, and so on.
	Stat(ctx context.Context, rev string) (*RevInfo, error)

	// Latest returns the latest revision on the default branch,
	// whatever that means in the underlying source code repository.
	// It is only used when there are no tagged versions.
	Latest(ctx context.Context) (*RevInfo, error)

	// GoMod returns the go.mod file for the given version.
	GoMod(ctx context.Context, version string) (data []byte, err error)

	// Zip writes a zip file for the given version to dst.
	Zip(ctx context.Context, dst io.Writer, version string) error
}

A Repo represents a repository storing all versions of a single module. It must be safe for simultaneous use by multiple goroutines.

func Lookup

func Lookup(ctx context.Context, proxy, path string) Repo

Lookup returns the module with the given module path, fetched through the given proxy.

The distinguished proxy "direct" indicates that the path should be fetched from its origin, and "noproxy" indicates that the patch should be fetched directly only if GONOPROXY matches the given path.

For the distinguished proxy "off", Lookup always returns a Repo that returns a non-nil error for every method call.

A successful return does not guarantee that the module has any defined versions.

type RevInfo

type RevInfo struct {
	Version string    // suggested version string for this revision
	Time    time.Time // commit time

	// These fields are used for Stat of arbitrary rev,
	// but they are not recorded when talking about module versions.
	Name  string `json:"-"` // complete ID in underlying repository
	Short string `json:"-"` // shortened ID, for use in pseudo-version

	Origin *codehost.Origin `json:",omitempty"` // provenance for reuse
}

A RevInfo describes a single revision in a module repository.

func InfoFile

func InfoFile(ctx context.Context, path, version string) (*RevInfo, string, error)

InfoFile is like Lookup(ctx, path).Stat(version) but also returns the name of the file containing the cached information.

type Versions

type Versions struct {
	Origin *codehost.Origin `json:",omitempty"` // origin information for reuse

	List []string // semver versions
}

A Versions describes the available versions in a module repository.