Documentation ¶
Overview ¶
Package artifact contains a solution for storing and fetching versioned blobs of data that are resolved on demand.
By default, a configuration directory called .artifact is searched for up the directory roots. It looks for a file called config.json in it. It defines where to store resolved files as well as where to fetch blobs from. In addition, when unspecified, versioned blobs are stored alongside the configuration file in tree.json which is a directory tree mapping files to content addresses. The system uses concepts from content-addressable storage in order to simplify storage and offer deduplication.
Index ¶
- Constants
- Variables
- func AtomicStore(path string, r io.Reader, hash string) (err error)
- func IsNotFoundError(err error) bool
- func MustNewPath(to string) string
- func MustPath(to string) string
- func NewArtifactNotFoundHashError(hash string) error
- func NewArtifactNotFoundPathError(path string) error
- func NewPath(to string) (string, error)
- func Path(to string) (string, error)
- func TestSetupGlobalCache(t *testing.T) (string, func())
- type Cache
- type Config
- type FileSystemStoreConfig
- type GoogleStorageStoreConfig
- type NotFoundError
- type Status
- type Store
- type StoreConfig
- type StoreType
- type TreeNode
- type TreeNodeExternal
- type TreeNodeTree
Constants ¶
const ( ConfigName = "config.json" TreeName = "tree.json" )
The artifact file names.
const ( StoreTypeFileSystem = StoreType("fs") StoreTypeGoogleStorage = StoreType("google_storage") )
The set of known store types.
const DefaultSourcePullSizeLimitBytes = 1 << 22
DefaultSourcePullSizeLimitBytes is the limit where if a normal pull happens, a file larger than this size will not be pulled down from source unless pull with ignoring the limit is used.
Variables ¶
var ( // DotDir is the location that artifact uses for itself. DotDir = ".artifact" // DefaultCachePath is the default relative location to store all cached // files (by hash). DefaultCachePath = "cache" )
var ErrConfigNotFound = errors.Errorf("%q not found on system", ConfigName)
ErrConfigNotFound is used when the configuration file cannot be found anywhere.
var Logger = golog.NewDevelopmentLogger("artifact")
Logger is the global logger to use for artifact.
Functions ¶
func AtomicStore ¶ added in v0.0.5
AtomicStore writes reader contents to a temp file and then renames to path, ensuring safer, atomic file writes.
func IsNotFoundError ¶ added in v0.0.3
IsNotFoundError returns if the given error is any kind of artifact not found error.
func MustNewPath ¶
MustNewPath works like NewPath but panics if any error occurs. Useful in tests.
func NewArtifactNotFoundHashError ¶ added in v0.0.3
NewArtifactNotFoundHashError returns an error for when an artifact is not found by its hash.
func NewArtifactNotFoundPathError ¶ added in v0.0.3
NewArtifactNotFoundPathError returns an error for when an artifact is not found by its path.
func Path ¶
Path returns the local file system path to the given artifact path. It errors if it does not exist or cannot be ensured to exist.
func TestSetupGlobalCache ¶
TestSetupGlobalCache usage implies this test package can *not* have any tests run in parallel without risking incorrect usage of the global cache. Be advised.
Types ¶
type Cache ¶
type Cache interface { Store // NewPath returns where a user visible asset would belong NewPath(to string) string // Ensure guarantees that a user visible path is populated with // the cached artifact. This can error if the path is unknown // or a failure happens retrieving it from underlying Store. Ensure(path string, ignoreLimit bool) (string, error) // Remove removes the given path from the tree and root if present // but not from the cache. Use clean with cache true to clear out // the cache. Remove(path string) error // Clean makes sure that the user visible assets reflect 1:1 // the versioned assets in the tree. Clean() error // WriteThroughUser makes sure that the user visible assets not // yet versioned are added to the tree and "written through" to // any stores responsible for caching. WriteThroughUser() error // Status inspects the root and returns a git like status of what is to // be added. Status() (*Status, error) // Close must be called in order to clean up any in use resources. Close() error }
A Cache is similar to a store in functionality but it also has capabilities to refer to artifacts by their designated path name and not by a hash. In addition, it understands how to update the underlying stores that the cache is based off of as well as the user visible, versioned assets.
func GlobalCache ¶
GlobalCache returns a cache to be used globally based on an automatically discovered configuration file. If the configuration file cannot be found, a noop implementation of the cache is created. In the future this should possibly automatically create the configuration file.
type Config ¶
type Config struct { // Cache is where the hashed files should live. If unset, it defaults // to DefaultCachePath. Cache string // Root is where the files represented by the tree should live. These // reflect files in the cache but is based on the current tree. If unset, // it defaults to a data directory within DefaultCachePath. Root string // SourceStore is the configuration for where artifacts are remotely pushed // to and pulled from. SourceStore StoreConfig // SourcePullSizeLimit is the limit where if a normal pull happens, // a file larger than this size will not be pulled down from source // unless pull with ignoring the limit is used. If unset, DefaultSourcePullSizeLimitBytes // is used. SourcePullSizeLimit int // Ignore is a list of simple file names to ignore when scanning through // the root. Ignore []string // contains filtered or unexported fields }
A Config describes how artifact should function.
func LoadConfig ¶
LoadConfig attempts to automatically load an artifact config by searching for the default configuration file upwards in the file system.
func LoadConfigFromFile ¶
LoadConfigFromFile loads a Config from the given path. It also searches for an adjacent tree file (not required to exist).
func (*Config) Lookup ¶
Lookup looks an artifact up by its path and returns its associated node if it exists.
func (*Config) RemovePath ¶
RemovePath removes nodes that fall into the given path.
func (*Config) StoreHash ¶
StoreHash associates a path to the given node hash. The path is able to overwrite any existing one, so this method can be destructive if an external node were to replace an internal one.
func (*Config) UnmarshalJSON ¶
UnmarshalJSON unmarshals the config from JSON data.
type FileSystemStoreConfig ¶
type FileSystemStoreConfig struct {
Path string `json:"path"`
}
FileSystemStoreConfig is for configuring a local file system based Store.
func (*FileSystemStoreConfig) Type ¶
func (c *FileSystemStoreConfig) Type() StoreType
Type returns that this is a file system Store.
type GoogleStorageStoreConfig ¶
type GoogleStorageStoreConfig struct {
Bucket string `json:"bucket"`
}
GoogleStorageStoreConfig is for configuring a Google based Store that has its credentials automatically looked up.
func (*GoogleStorageStoreConfig) Type ¶
func (c *GoogleStorageStoreConfig) Type() StoreType
Type returns that this is a Google storage Store.
type NotFoundError ¶ added in v0.0.3
type NotFoundError struct {
// contains filtered or unexported fields
}
An NotFoundError is used when an artifact can not be found.
func (*NotFoundError) Error ¶ added in v0.0.3
func (e *NotFoundError) Error() string
Error returns an error specific to the way the artifact was searched for.
type Status ¶
Status describes how the tree has changed. We don't show removed because a removal is a direct operation done on the tree that is immediately reflected in source control.
type Store ¶
type Store interface { Contains(hash string) error Load(hash string) (io.ReadCloser, error) Store(hash string, r io.Reader) error Close() error }
A Store is responsible for loading and storing artifacts by their hashes and content.
func NewStore ¶
func NewStore(config StoreConfig) (Store, error)
NewStore returns a new store based on the given config. It errors if making the store fails or the underlying type has no constructor.
type StoreConfig ¶
type StoreConfig interface { // Type returns the type of the Store. Type() StoreType }
A StoreConfig describes how to configure and set up a Store.
type TreeNode ¶
type TreeNode struct {
// contains filtered or unexported fields
}
A TreeNode represents a node in an artifact tree. The tree is a hierarchy of artifacts that mimics a file system.
func (*TreeNode) IsInternal ¶
IsInternal returns if this node is an internal node.
func (*TreeNode) MarshalJSON ¶
MarshalJSON marshals the node out into JSON.
func (*TreeNode) UnmarshalJSON ¶
UnmarshalJSON unmarshals JSON into a specific tree node that may be internal or external.
type TreeNodeExternal ¶
A TreeNodeExternal is an external node representing the location of an artifact identified by its content hash.
type TreeNodeTree ¶
A TreeNodeTree is an internal node with mappings to other nodes.
Source Files ¶
Directories ¶
Path | Synopsis |
---|---|
cmd
|
|
artifact
Package main provides the artifact CLI for importing and exporting artifacts.
|
Package main provides the artifact CLI for importing and exporting artifacts. |
Package tools implements the sub-commands for the artifact CLI.
|
Package tools implements the sub-commands for the artifact CLI. |