cas

Documentation

Index

• type CachingDirectoryFetcherEvictionSet
• type CachingDirectoryFetcherKey
• type DirectoryFetcher
  • func NewBlobAccessDirectoryFetcher(blobAccess blobstore.BlobAccess, maximumDirectorySizeBytes int, ...) DirectoryFetcher
  • func NewCachingDirectoryFetcher(base DirectoryFetcher, digestKeyFormat digest.KeyFormat, maximumCount int, ...) DirectoryFetcher
  • func NewCachingDirectoryFetcherFromConfiguration(configuration *pb.CachingDirectoryFetcherConfiguration, base DirectoryFetcher) (DirectoryFetcher, error)
  • func NewSuspendingDirectoryFetcher(base DirectoryFetcher, suspendable clock.Suspendable) DirectoryFetcher
• type DirectoryWalker
  • func NewDecomposedDirectoryWalker(fetcher DirectoryFetcher, digest digest.Digest) DirectoryWalker
• type FileFetcher
  • func NewBlobAccessFileFetcher(blobAccess blobstore.BlobAccess) FileFetcher
  • func NewHardlinkingFileFetcher(base FileFetcher, cacheDirectory filesystem.Directory, maxFiles int, ...) FileFetcher

Types

type CachingDirectoryFetcherEvictionSet

type CachingDirectoryFetcherEvictionSet = eviction.Set[CachingDirectoryFetcherKey]

CachingDirectoryFetcherEvictionSet is the eviction set type that is accepted by NewCachingDirectoryFetcher().

type CachingDirectoryFetcherKey

type CachingDirectoryFetcherKey struct {
	DigestKey  string
	IsTreeRoot bool
}

CachingDirectoryFetcherKey is the key type that is used by CachingDirectoryFetcher.

A separate boolean field is used to distinguish Directory objects that are roots of Tree objects from the others. This is needed because the digests of these objects don't match with that of the resulting Directory object, while for the others it does. Using the same key space for all objects collectively would be insecure.

type DirectoryFetcher

type DirectoryFetcher interface {
	GetDirectory(ctx context.Context, directoryDigest digest.Digest) (*remoteexecution.Directory, error)
	GetTreeRootDirectory(ctx context.Context, treeDigest digest.Digest) (*remoteexecution.Directory, error)
	GetTreeChildDirectory(ctx context.Context, treeDigest, childDigest digest.Digest) (*remoteexecution.Directory, error)
}

DirectoryFetcher is responsible for fetching Directory messages from the Content Addressable Storage (CAS). These describe the layout of a single directory in a build action's input root.

func NewBlobAccessDirectoryFetcher

func NewBlobAccessDirectoryFetcher(blobAccess blobstore.BlobAccess, maximumDirectorySizeBytes int, maximumTreeSizeBytes int64) DirectoryFetcher

NewBlobAccessDirectoryFetcher creates a DirectoryFetcher that reads Directory objects from a BlobAccess based store.

func NewCachingDirectoryFetcher

func NewCachingDirectoryFetcher(base DirectoryFetcher, digestKeyFormat digest.KeyFormat, maximumCount int, maximumSizeBytes int64, evictionSet CachingDirectoryFetcherEvictionSet) DirectoryFetcher

NewCachingDirectoryFetcher creates an adapter for DirectoryFetcher that caches up a fixed number of unmarshalled objects in memory. This reduces the amount of time spent unmarshaling messages, and may reduce the amount of network traffic generated.

func NewCachingDirectoryFetcherFromConfiguration

func NewCachingDirectoryFetcherFromConfiguration(configuration *pb.CachingDirectoryFetcherConfiguration, base DirectoryFetcher) (DirectoryFetcher, error)

NewCachingDirectoryFetcherFromConfiguration creates a new CachingDirectoryFetcher based on parameters provided in a configuration file.

func NewSuspendingDirectoryFetcher

func NewSuspendingDirectoryFetcher(base DirectoryFetcher, suspendable clock.Suspendable) DirectoryFetcher

NewSuspendingDirectoryFetcher is a decorator for DirectoryFetcher that simply forwards all methods. Before and after each call, it suspends and resumes a clock.Suspendable object, respectively.

This decorator is used in combination with SuspendableClock, allowing FUSE/NFSv4-based workers to compensate the execution timeout of build actions for any time spent loading directory contents of the input root.

type DirectoryWalker

type DirectoryWalker interface {
	// GetDirectory() returns the contents of the current directory.
	GetDirectory(ctx context.Context) (*remoteexecution.Directory, error)

	// GetChild() can be used obtain a new DirectoryWalker instance
	// that corresponds to one of the children of this directory.
	GetChild(digest digest.Digest) DirectoryWalker

	// GetDescription() gives a textual description of the
	// DirectoryWalker, which may be useful for logging purposes.
	GetDescription() string

	// GetContainingDigest() returns the digest of the Content
	// Addressable Storage object that holds this directory.
	//
	// In the case of plain Directory objects, this function returns
	// the digest provided to GetChild(). In the case of Tree
	// objects, the digest of the containing Tree is returned, which
	// differs from the digest provided to GetChild().
	GetContainingDigest() digest.Digest
}

DirectoryWalker is identical to a DirectoryFetcher, except that it is bound to a specific instance of a directory.

The goal of this interface is to provide uniform access to Directory messages, regardless of the way they are stored in the Content Addressable Storage (i.e., as separate objects, or as part of a Tree message).

func NewDecomposedDirectoryWalker

func NewDecomposedDirectoryWalker(fetcher DirectoryFetcher, digest digest.Digest) DirectoryWalker

NewDecomposedDirectoryWalker creates a DirectoryWalker that assumes that all Directory messages are stored as separate objects in the Content Addressable Storage (CAS). This is the case for input roots of build actions.

type FileFetcher

type FileFetcher interface {
	GetFile(ctx context.Context, digest digest.Digest, directory filesystem.Directory, name path.Component, isExecutable bool) error
}

FileFetcher is responsible for fetching files from the Content Addressable Storage (CAS), storing its contents inside a file on disk.

func NewBlobAccessFileFetcher

func NewBlobAccessFileFetcher(blobAccess blobstore.BlobAccess) FileFetcher

NewBlobAccessFileFetcher creates a FileFetcher that reads files fom a BlobAccess based store.

func NewHardlinkingFileFetcher

func NewHardlinkingFileFetcher(base FileFetcher, cacheDirectory filesystem.Directory, maxFiles int, maxSize int64, evictionSet eviction.Set[string]) FileFetcher

NewHardlinkingFileFetcher is an adapter for FileFetcher that stores files in an internal directory. After successfully downloading files at the target location, they are hardlinked into the cache. Future calls for the same file will hardlink them from the cache to the target location. This reduces the amount of network traffic needed.