storage

package module
v1.12.11 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 20, 2019 License: Apache-2.0 Imports: 38 Imported by: 464

README

storage is a Go library which aims to provide methods for storing filesystem layers, container images, and containers. A containers-storage CLI wrapper is also included for manual and scripting use.

To build the CLI wrapper, use 'make binary'.

Operations which use VMs expect to launch them using 'vagrant', defaulting to using its 'libvirt' provider. The boxes used are also available for the 'virtualbox' provider, and can be selected by setting $VAGRANT_PROVIDER to 'virtualbox' before kicking off the build.

The library manages three types of items: layers, images, and containers.

A layer is a copy-on-write filesystem which is notionally stored as a set of changes relative to its parent layer, if it has one. A given layer can only have one parent, but any layer can be the parent of multiple layers. Layers which are parents of other layers should be treated as read-only.

An image is a reference to a particular layer (its top layer), along with other information which the library can manage for the convenience of its caller. This information typically includes configuration templates for running a binary contained within the image's layers, and may include cryptographic signatures. Multiple images can reference the same layer, as the differences between two images may not be in their layer contents.

A container is a read-write layer which is a child of an image's top layer, along with information which the library can manage for the convenience of its caller. This information typically includes configuration information for running the specific container. Multiple containers can be derived from a single image.

Layers, images, and containers are represented primarily by 32 character hexadecimal IDs, but items of each kind can also have one or more arbitrary names attached to them, which the library will automatically resolve to IDs when they are passed in to API calls which expect IDs.

The library can store what it calls metadata for each of these types of items. This is expected to be a small piece of data, since it is cached in memory and stored along with the library's own bookkeeping information.

Additionally, the library can store one or more of what it calls big data for images and containers. This is a named chunk of larger data, which is only in memory when it is being read from or being written to its own disk file.

Contributing Information about contributing to this project.

Documentation

Index

Constants

View Source 
const (
	// ImageDigestManifestBigDataNamePrefix is a prefix of big data item
	// names which we consider to be manifests, used for computing a
	// "digest" value for the image as a whole, by which we can locate the
	// image later.
	ImageDigestManifestBigDataNamePrefix = "manifest"
	// ImageDigestBigDataKey is provided for compatibility with older
	// versions of the image library.  It will be removed in the future.
	ImageDigestBigDataKey = "manifest"
)

Variables

View Source 
var (
	// ErrContainerUnknown indicates that there was no container with the specified name or ID.
	ErrContainerUnknown = errors.New("container not known")
	// ErrImageUnknown indicates that there was no image with the specified name or ID.
	ErrImageUnknown = errors.New("image not known")
	// ErrParentUnknown indicates that we didn't record the ID of the parent of the specified layer.
	ErrParentUnknown = errors.New("parent of layer not known")
	// ErrLayerUnknown indicates that there was no layer with the specified name or ID.
	ErrLayerUnknown = errors.New("layer not known")
	// ErrLoadError indicates that there was an initialization error.
	ErrLoadError = errors.New("error loading storage metadata")
	// ErrDuplicateID indicates that an ID which is to be assigned to a new item is already being used.
	ErrDuplicateID = errors.New("that ID is already in use")
	// ErrDuplicateName indicates that a name which is to be assigned to a new item is already being used.
	ErrDuplicateName = errors.New("that name is already in use")
	// ErrParentIsContainer is returned when a caller attempts to create a layer as a child of a container's layer.
	ErrParentIsContainer = errors.New("would-be parent layer is a container")
	// ErrNotAContainer is returned when the caller attempts to delete a container that isn't a container.
	ErrNotAContainer = errors.New("identifier is not a container")
	// ErrNotAnImage is returned when the caller attempts to delete an image that isn't an image.
	ErrNotAnImage = errors.New("identifier is not an image")
	// ErrNotALayer is returned when the caller attempts to delete a layer that isn't a layer.
	ErrNotALayer = errors.New("identifier is not a layer")
	// ErrNotAnID is returned when the caller attempts to read or write metadata from an item that doesn't exist.
	ErrNotAnID = errors.New("identifier is not a layer, image, or container")
	// ErrLayerHasChildren is returned when the caller attempts to delete a layer that has children.
	ErrLayerHasChildren = errors.New("layer has children")
	// ErrLayerUsedByImage is returned when the caller attempts to delete a layer that is an image's top layer.
	ErrLayerUsedByImage = errors.New("layer is in use by an image")
	// ErrLayerUsedByContainer is returned when the caller attempts to delete a layer that is a container's layer.
	ErrLayerUsedByContainer = errors.New("layer is in use by a container")
	// ErrImageUsedByContainer is returned when the caller attempts to delete an image that is a container's image.
	ErrImageUsedByContainer = errors.New("image is in use by a container")
	// ErrIncompleteOptions is returned when the caller attempts to initialize a Store without providing required information.
	ErrIncompleteOptions = errors.New("missing necessary StoreOptions")
	// ErrSizeUnknown is returned when the caller asks for the size of a big data item, but the Store couldn't determine the answer.
	ErrSizeUnknown = errors.New("size is not known")
	// ErrStoreIsReadOnly is returned when the caller makes a call to a read-only store that would require modifying its contents.
	ErrStoreIsReadOnly = errors.New("called a write method on a read-only store")
	// ErrDuplicateImageNames indicates that the read-only store uses the same name for multiple images.
	ErrDuplicateImageNames = errors.New("read-only image store assigns the same name to multiple images")
	// ErrDuplicateLayerNames indicates that the read-only store uses the same name for multiple layers.
	ErrDuplicateLayerNames = errors.New("read-only layer store assigns the same name to multiple layers")
	// ErrInvalidBigDataName indicates that the name for a big data item is not acceptable; it may be empty.
	ErrInvalidBigDataName = errors.New("not a valid name for a big data item")
	// ErrDigestUnknown indicates that we were unable to compute the digest of a specified item.
	ErrDigestUnknown = errors.New("could not compute digest of item")
	// ErrLayerNotMounted is returned when the requested information can only be computed for a mounted layer, and the layer is not mounted.
	ErrLayerNotMounted = errors.New("layer is not mounted")
)

Functions

func DefaultConfigFile 

func DefaultConfigFile(rootless bool) (string, error)

DefaultConfigFile returns the path to the storage config file used

func GetDefaultMountOptions 

func GetDefaultMountOptions() ([]string, error)

GetDefaultMountOptions returns the default mountoptions defined in container/storage

func GetMountOptions 

func GetMountOptions(driver string, graphDriverOptions []string) ([]string, error)

GetMountOptions returns the mountoptions for the specified driver and graphDriverOptions

func GetRootlessRuntimeDir 

func GetRootlessRuntimeDir(rootlessUid int) (string, error)

GetRootlessRuntimeDir returns the runtime directory when running as non root

func ReloadConfigurationFile 

func ReloadConfigurationFile(configFile string, storeOptions *StoreOptions)

ReloadConfigurationFile parses the specified configuration file and overrides the configuration in storeOptions.

Types

type Container 

type Container struct {
	// ID is either one which was specified at create-time, or a random
	// value which was generated by the library.
	ID string `json:"id"`

	// Names is an optional set of user-defined convenience values.  The
	// container can be referred to by its ID or any of its names.  Names
	// are unique among containers.
	Names []string `json:"names,omitempty"`

	// ImageID is the ID of the image which was used to create the container.
	ImageID string `json:"image"`

	// LayerID is the ID of the read-write layer for the container itself.
	// It is assumed that the image's top layer is the parent of the container's
	// read-write layer.
	LayerID string `json:"layer"`

	// Metadata is data we keep for the convenience of the caller.  It is not
	// expected to be large, since it is kept in memory.
	Metadata string `json:"metadata,omitempty"`

	// BigDataNames is a list of names of data items that we keep for the
	// convenience of the caller.  They can be large, and are only in
	// memory when being read from or written to disk.
	BigDataNames []string `json:"big-data-names,omitempty"`

	// BigDataSizes maps the names in BigDataNames to the sizes of the data
	// that has been stored, if they're known.
	BigDataSizes map[string]int64 `json:"big-data-sizes,omitempty"`

	// BigDataDigests maps the names in BigDataNames to the digests of the
	// data that has been stored, if they're known.
	BigDataDigests map[string]digest.Digest `json:"big-data-digests,omitempty"`

	// Created is the datestamp for when this container was created.  Older
	// versions of the library did not track this information, so callers
	// will likely want to use the IsZero() method to verify that a value
	// is set before using it.
	Created time.Time `json:"created,omitempty"`

	// UIDMap and GIDMap are used for setting up a container's root
	// filesystem for use inside of a user namespace where UID mapping is
	// being used.
	UIDMap []idtools.IDMap `json:"uidmap,omitempty"`
	GIDMap []idtools.IDMap `json:"gidmap,omitempty"`

	Flags map[string]interface{} `json:"flags,omitempty"`
}

A Container is a reference to a read-write layer with metadata.

func (*Container) MarshalJSON 

func (j *Container) MarshalJSON() ([]byte, error)

MarshalJSON marshal bytes to json - template

func (*Container) MarshalJSONBuf 

func (j *Container) MarshalJSONBuf(buf fflib.EncodingBuffer) error

MarshalJSONBuf marshal buff to json - template

func (*Container) MountLabel 

func (c *Container) MountLabel() string

func (*Container) MountOpts 

func (c *Container) MountOpts() []string

func (*Container) ProcessLabel 

func (c *Container) ProcessLabel() string

func (*Container) UnmarshalJSON 

func (j *Container) UnmarshalJSON(input []byte) error

UnmarshalJSON umarshall json - template of ffjson

func (*Container) UnmarshalJSONFFLexer 

func (j *Container) UnmarshalJSONFFLexer(fs *fflib.FFLexer, state fflib.FFParseState) error

UnmarshalJSONFFLexer fast json unmarshall - template ffjson

type ContainerBigDataStore 

type ContainerBigDataStore interface {
	ROBigDataStore
	// SetBigData stores a (potentially large) piece of data associated
	// with this ID.
	SetBigData(id, key string, data []byte) error
}

A ContainerBigDataStore wraps up how we store big-data associated with containers.

type ContainerOptions 

type ContainerOptions struct {
	// IDMappingOptions specifies the type of ID mapping which should be
	// used for this container's layer.  If nothing is specified, the
	// container's layer will inherit settings from the image's top layer
	// or, if it is not being created based on an image, the Store object.
	IDMappingOptions
	LabelOpts []string
	Flags     map[string]interface{}
	MountOpts []string
}

ContainerOptions is used for passing options to a Store's CreateContainer() method.

type ContainerStore 

type ContainerStore interface {
	FileBasedStore
	MetadataStore
	ContainerBigDataStore
	FlaggableStore

	// Create creates a container that has a specified ID (or generates a
	// random one if an empty value is supplied) and optional names,
	// based on the specified image, using the specified layer as its
	// read-write layer.
	// The maps in the container's options structure are recorded for the
	// convenience of the caller, nothing more.
	Create(id string, names []string, image, layer, metadata string, options *ContainerOptions) (*Container, error)

	// SetNames updates the list of names associated with the container
	// with the specified ID.
	SetNames(id string, names []string) error

	// Get retrieves information about a container given an ID or name.
	Get(id string) (*Container, error)

	// Exists checks if there is a container with the given ID or name.
	Exists(id string) bool

	// Delete removes the record of the container.
	Delete(id string) error

	// Wipe removes records of all containers.
	Wipe() error

	// Lookup attempts to translate a name to an ID.  Most methods do this
	// implicitly.
	Lookup(name string) (string, error)

	// Containers returns a slice enumerating the known containers.
	Containers() ([]Container, error)
}

ContainerStore provides bookkeeping for information about Containers.

type DiffOptions 

type DiffOptions struct {
	// Compression, if set overrides the default compressor when generating a diff.
	Compression *archive.Compression
}

DiffOptions override the default behavior of Diff() methods.

func (*DiffOptions) MarshalJSON 

func (j *DiffOptions) MarshalJSON() ([]byte, error)

MarshalJSON marshal bytes to json - template

func (*DiffOptions) MarshalJSONBuf 

func (j *DiffOptions) MarshalJSONBuf(buf fflib.EncodingBuffer) error

MarshalJSONBuf marshal buff to json - template

func (*DiffOptions) UnmarshalJSON 

func (j *DiffOptions) UnmarshalJSON(input []byte) error

UnmarshalJSON umarshall json - template of ffjson

func (*DiffOptions) UnmarshalJSONFFLexer 

func (j *DiffOptions) UnmarshalJSONFFLexer(fs *fflib.FFLexer, state fflib.FFParseState) error

UnmarshalJSONFFLexer fast json unmarshall - template ffjson

type FileBasedStore 

type FileBasedStore interface {
	ROFileBasedStore
	RWFileBasedStore
}

FileBasedStore wraps up the common methods of various types of file-based data stores that we implement.

type FlaggableStore 

type FlaggableStore interface {
	// ClearFlag removes a named flag from an item in the store.
	ClearFlag(id string, flag string) error

	// SetFlag sets a named flag and its value on an item in the store.
	SetFlag(id string, flag string, value interface{}) error
}

A FlaggableStore can have flags set and cleared on items which it manages.

type IDMappingOptions 

type IDMappingOptions struct {
	// UIDMap and GIDMap are used for setting up a layer's root filesystem
	// for use inside of a user namespace where ID mapping is being used.
	// If HostUIDMapping/HostGIDMapping is true, no mapping of the
	// respective type will be used.  Otherwise, if UIDMap and/or GIDMap
	// contain at least one mapping, one or both will be used.  By default,
	// if neither of those conditions apply, if the layer has a parent
	// layer, the parent layer's mapping will be used, and if it does not
	// have a parent layer, the mapping which was passed to the Store
	// object when it was initialized will be used.
	HostUIDMapping bool
	HostGIDMapping bool
	UIDMap         []idtools.IDMap
	GIDMap         []idtools.IDMap
}

IDMappingOptions are used for specifying how ID mapping should be set up for a layer or container.

func ParseIDMapping 

func ParseIDMapping(UIDMapSlice, GIDMapSlice []string, subUIDMap, subGIDMap string) (*IDMappingOptions, error)

ParseIDMapping takes idmappings and subuid and subgid maps and returns a storage mapping

type Image 

type Image struct {
	// ID is either one which was specified at create-time, or a random
	// value which was generated by the library.
	ID string `json:"id"`

	// Digest is a digest value that we can use to locate the image, if one
	// was specified at creation-time.
	Digest digest.Digest `json:"digest,omitempty"`

	// Digests is a list of digest values of the image's manifests, and
	// possibly a manually-specified value, that we can use to locate the
	// image.  If Digest is set, its value is also in this list.
	Digests []digest.Digest `json:"-"`

	// Names is an optional set of user-defined convenience values.  The
	// image can be referred to by its ID or any of its names.  Names are
	// unique among images, and are often the text representation of tagged
	// or canonical references.
	Names []string `json:"names,omitempty"`

	// TopLayer is the ID of the topmost layer of the image itself, if the
	// image contains one or more layers.  Multiple images can refer to the
	// same top layer.
	TopLayer string `json:"layer,omitempty"`

	// MappedTopLayers are the IDs of alternate versions of the top layer
	// which have the same contents and parent, and which differ from
	// TopLayer only in which ID mappings they use.  When the image is
	// to be removed, they should be removed before the TopLayer, as the
	// graph driver may depend on that.
	MappedTopLayers []string `json:"mapped-layers,omitempty"`

	// Metadata is data we keep for the convenience of the caller.  It is not
	// expected to be large, since it is kept in memory.
	Metadata string `json:"metadata,omitempty"`

	// BigDataNames is a list of names of data items that we keep for the
	// convenience of the caller.  They can be large, and are only in
	// memory when being read from or written to disk.
	BigDataNames []string `json:"big-data-names,omitempty"`

	// BigDataSizes maps the names in BigDataNames to the sizes of the data
	// that has been stored, if they're known.
	BigDataSizes map[string]int64 `json:"big-data-sizes,omitempty"`

	// BigDataDigests maps the names in BigDataNames to the digests of the
	// data that has been stored, if they're known.
	BigDataDigests map[string]digest.Digest `json:"big-data-digests,omitempty"`

	// Created is the datestamp for when this image was created.  Older
	// versions of the library did not track this information, so callers
	// will likely want to use the IsZero() method to verify that a value
	// is set before using it.
	Created time.Time `json:"created,omitempty"`

	// ReadOnly is true if this image resides in a read-only layer store.
	ReadOnly bool `json:"-"`

	Flags map[string]interface{} `json:"flags,omitempty"`
}

An Image is a reference to a layer and an associated metadata string.

func (*Image) MarshalJSON 

func (j *Image) MarshalJSON() ([]byte, error)

MarshalJSON marshal bytes to json - template

func (*Image) MarshalJSONBuf 

func (j *Image) MarshalJSONBuf(buf fflib.EncodingBuffer) error

MarshalJSONBuf marshal buff to json - template

func (*Image) UnmarshalJSON 

func (j *Image) UnmarshalJSON(input []byte) error

UnmarshalJSON umarshall json - template of ffjson

func (*Image) UnmarshalJSONFFLexer 

func (j *Image) UnmarshalJSONFFLexer(fs *fflib.FFLexer, state fflib.FFParseState) error

UnmarshalJSONFFLexer fast json unmarshall - template ffjson

type ImageOptions 

type ImageOptions struct {
	// CreationDate, if not zero, will override the default behavior of marking the image as having been
	// created when CreateImage() was called, recording CreationDate instead.
	CreationDate time.Time
	// Digest is a hard-coded digest value that we can use to look up the image.  It is optional.
	Digest digest.Digest
}

ImageOptions is used for passing options to a Store's CreateImage() method.

type ImageStore 

type ImageStore interface {
	ROImageStore
	RWFileBasedStore
	RWMetadataStore
	RWImageBigDataStore
	FlaggableStore

	// Create creates an image that has a specified ID (or a random one) and
	// optional names, using the specified layer as its topmost (hopefully
	// read-only) layer.  That layer can be referenced by multiple images.
	Create(id string, names []string, layer, metadata string, created time.Time, searchableDigest digest.Digest) (*Image, error)

	// SetNames replaces the list of names associated with an image with the
	// supplied values.  The values are expected to be valid normalized
	// named image references.
	SetNames(id string, names []string) error

	// Delete removes the record of the image.
	Delete(id string) error

	// Wipe removes records of all images.
	Wipe() error
}

ImageStore provides bookkeeping for information about Images.

type Layer 

type Layer struct {
	// ID is either one which was specified at create-time, or a random
	// value which was generated by the library.
	ID string `json:"id"`

	// Names is an optional set of user-defined convenience values.  The
	// layer can be referred to by its ID or any of its names.  Names are
	// unique among layers.
	Names []string `json:"names,omitempty"`

	// Parent is the ID of a layer from which this layer inherits data.
	Parent string `json:"parent,omitempty"`

	// Metadata is data we keep for the convenience of the caller.  It is not
	// expected to be large, since it is kept in memory.
	Metadata string `json:"metadata,omitempty"`

	// MountLabel is an SELinux label which should be used when attempting to mount
	// the layer.
	MountLabel string `json:"mountlabel,omitempty"`

	// MountPoint is the path where the layer is mounted, or where it was most
	// recently mounted.  This can change between subsequent Unmount() and
	// Mount() calls, so the caller should consult this value after Mount()
	// succeeds to find the location of the container's root filesystem.
	MountPoint string `json:"-"`

	// MountCount is used as a reference count for the container's layer being
	// mounted at the mount point.
	MountCount int `json:"-"`

	// Created is the datestamp for when this layer was created.  Older
	// versions of the library did not track this information, so callers
	// will likely want to use the IsZero() method to verify that a value
	// is set before using it.
	Created time.Time `json:"created,omitempty"`

	// CompressedDigest is the digest of the blob that was last passed to
	// ApplyDiff() or Put(), as it was presented to us.
	CompressedDigest digest.Digest `json:"compressed-diff-digest,omitempty"`

	// CompressedSize is the length of the blob that was last passed to
	// ApplyDiff() or Put(), as it was presented to us.  If
	// CompressedDigest is not set, this should be treated as if it were an
	// uninitialized value.
	CompressedSize int64 `json:"compressed-size,omitempty"`

	// UncompressedDigest is the digest of the blob that was last passed to
	// ApplyDiff() or Put(), after we decompressed it.  Often referred to
	// as a DiffID.
	UncompressedDigest digest.Digest `json:"diff-digest,omitempty"`

	// UncompressedSize is the length of the blob that was last passed to
	// ApplyDiff() or Put(), after we decompressed it.  If
	// UncompressedDigest is not set, this should be treated as if it were
	// an uninitialized value.
	UncompressedSize int64 `json:"diff-size,omitempty"`

	// CompressionType is the type of compression which we detected on the blob
	// that was last passed to ApplyDiff() or Put().
	CompressionType archive.Compression `json:"compression,omitempty"`

	// Flags is arbitrary data about the layer.
	Flags map[string]interface{} `json:"flags,omitempty"`

	// UIDMap and GIDMap are used for setting up a layer's contents
	// for use inside of a user namespace where UID mapping is being used.
	UIDMap []idtools.IDMap `json:"uidmap,omitempty"`
	GIDMap []idtools.IDMap `json:"gidmap,omitempty"`

	// ReadOnly is true if this layer resides in a read-only layer store.
	ReadOnly bool `json:"-"`
}

A Layer is a record of a copy-on-write layer that's stored by the lower level graph driver.

func (*Layer) MarshalJSON 

func (j *Layer) MarshalJSON() ([]byte, error)

MarshalJSON marshal bytes to json - template

func (*Layer) MarshalJSONBuf 

func (j *Layer) MarshalJSONBuf(buf fflib.EncodingBuffer) error

MarshalJSONBuf marshal buff to json - template

func (*Layer) UnmarshalJSON 

func (j *Layer) UnmarshalJSON(input []byte) error

UnmarshalJSON umarshall json - template of ffjson

func (*Layer) UnmarshalJSONFFLexer 

func (j *Layer) UnmarshalJSONFFLexer(fs *fflib.FFLexer, state fflib.FFParseState) error

UnmarshalJSONFFLexer fast json unmarshall - template ffjson

type LayerOptions 

type LayerOptions struct {
	// IDMappingOptions specifies the type of ID mapping which should be
	// used for this layer.  If nothing is specified, the layer will
	// inherit settings from its parent layer or, if it has no parent
	// layer, the Store object.
	IDMappingOptions
	// TemplateLayer is the ID of a layer whose contents will be used to
	// initialize this layer.  If set, it should be a child of the layer
	// which we want to use as the parent of the new layer.
	TemplateLayer string
}

LayerOptions is used for passing options to a Store's CreateLayer() and PutLayer() methods.

type LayerStore 

type LayerStore interface {
	ROLayerStore
	RWFileBasedStore
	RWMetadataStore
	FlaggableStore

	// Create creates a new layer, optionally giving it a specified ID rather than
	// a randomly-generated one, either inheriting data from another specified
	// layer or the empty base layer.  The new layer can optionally be given names
	// and have an SELinux label specified for use when mounting it.  Some
	// underlying drivers can accept a "size" option.  At this time, most
	// underlying drivers do not themselves distinguish between writeable
	// and read-only layers.
	Create(id string, parent *Layer, names []string, mountLabel string, options map[string]string, moreOptions *LayerOptions, writeable bool) (*Layer, error)

	// CreateWithFlags combines the functions of Create and SetFlag.
	CreateWithFlags(id string, parent *Layer, names []string, mountLabel string, options map[string]string, moreOptions *LayerOptions, writeable bool, flags map[string]interface{}) (layer *Layer, err error)

	// Put combines the functions of CreateWithFlags and ApplyDiff.
	Put(id string, parent *Layer, names []string, mountLabel string, options map[string]string, moreOptions *LayerOptions, writeable bool, flags map[string]interface{}, diff io.Reader) (*Layer, int64, error)

	// SetNames replaces the list of names associated with a layer with the
	// supplied values.
	SetNames(id string, names []string) error

	// Delete deletes a layer with the specified name or ID.
	Delete(id string) error

	// Wipe deletes all layers.
	Wipe() error

	// Mount mounts a layer for use.  If the specified layer is the parent of other
	// layers, it should not be written to.  An SELinux label to be applied to the
	// mount can be specified to override the one configured for the layer.
	// The mappings used by the container can be specified.
	Mount(id string, options drivers.MountOpts) (string, error)

	// Unmount unmounts a layer when it is no longer in use.
	Unmount(id string, force bool) (bool, error)

	// Mounted returns number of times the layer has been mounted.
	Mounted(id string) (int, error)

	// ParentOwners returns the UIDs and GIDs of parents of the layer's mountpoint
	// for which the layer's UID and GID maps don't contain corresponding entries.
	ParentOwners(id string) (uids, gids []int, err error)

	// ApplyDiff reads a tarstream which was created by a previous call to Diff and
	// applies its changes to a specified layer.
	ApplyDiff(to string, diff io.Reader) (int64, error)
}

LayerStore wraps a graph driver, adding the ability to refer to layers by name, and keeping track of parent-child relationships, along with a list of all known layers.

type Locker 

type Locker interface {
	// Acquire a writer lock.
	Lock()

	// Acquire a writer lock recursively, allowing for recursive acquisitions
	// within the same process space.
	RecursiveLock()

	// Unlock the lock.
	Unlock()

	// Acquire a reader lock.
	RLock()

	// Touch records, for others sharing the lock, that the caller was the
	// last writer.  It should only be called with the lock held.
	Touch() error

	// Modified() checks if the most recent writer was a party other than the
	// last recorded writer.  It should only be called with the lock held.
	Modified() (bool, error)

	// TouchedSince() checks if the most recent writer modified the file (likely using Touch()) after the specified time.
	TouchedSince(when time.Time) bool

	// IsReadWrite() checks if the lock file is read-write
	IsReadWrite() bool

	// Locked() checks if lock is locked for writing by a thread in this process
	Locked() bool
}

A Locker represents a file lock where the file is used to cache an identifier of the last party that made changes to whatever's being protected by the lock.

func GetLockfile 

func GetLockfile(path string) (Locker, error)

GetLockfile opens a read-write lock file, creating it if necessary. The Locker object may already be locked if the path has already been requested by the current process.

func GetROLockfile 

func GetROLockfile(path string) (Locker, error)

GetROLockfile opens a read-only lock file, creating it if necessary. The Locker object may already be locked if the path has already been requested by the current process.

type MetadataStore 

type MetadataStore interface {
	ROMetadataStore
	RWMetadataStore
}

MetadataStore wraps up methods for getting and setting metadata associated with IDs.

type ROBigDataStore 

type ROBigDataStore interface {
	// BigData retrieves a (potentially large) piece of data associated with
	// this ID, if it has previously been set.
	BigData(id, key string) ([]byte, error)

	// BigDataSize retrieves the size of a (potentially large) piece of
	// data associated with this ID, if it has previously been set.
	BigDataSize(id, key string) (int64, error)

	// BigDataDigest retrieves the digest of a (potentially large) piece of
	// data associated with this ID, if it has previously been set.
	BigDataDigest(id, key string) (digest.Digest, error)

	// BigDataNames() returns a list of the names of previously-stored pieces of
	// data.
	BigDataNames(id string) ([]string, error)
}

An ROBigDataStore wraps up the read-only big-data related methods of the various types of file-based lookaside stores that we implement.

type ROFileBasedStore 

type ROFileBasedStore interface {
	Locker

	// Load reloads the contents of the store from disk.  It should be called
	// with the lock held.
	Load() error
}

ROFileBasedStore wraps up the methods of the various types of file-based data stores that we implement which are needed for both read-only and read-write files.

type ROImageStore 

type ROImageStore interface {
	ROFileBasedStore
	ROMetadataStore
	ROBigDataStore

	// Exists checks if there is an image with the given ID or name.
	Exists(id string) bool

	// Get retrieves information about an image given an ID or name.
	Get(id string) (*Image, error)

	// Lookup attempts to translate a name to an ID.  Most methods do this
	// implicitly.
	Lookup(name string) (string, error)

	// Images returns a slice enumerating the known images.
	Images() ([]Image, error)

	// ByDigest returns a slice enumerating the images which have either an
	// explicitly-set digest, or a big data item with a name that starts
	// with ImageDigestManifestBigDataNamePrefix, which matches the
	// specified digest.
	ByDigest(d digest.Digest) ([]*Image, error)
}

ROImageStore provides bookkeeping for information about Images.

type ROLayerStore 

type ROLayerStore interface {
	ROFileBasedStore
	ROMetadataStore

	// Exists checks if a layer with the specified name or ID is known.
	Exists(id string) bool

	// Get retrieves information about a layer given an ID or name.
	Get(id string) (*Layer, error)

	// Status returns an slice of key-value pairs, suitable for human consumption,
	// relaying whatever status information the underlying driver can share.
	Status() ([][2]string, error)

	// Changes returns a slice of Change structures, which contain a pathname
	// (Path) and a description of what sort of change (Kind) was made by the
	// layer (either ChangeModify, ChangeAdd, or ChangeDelete), relative to a
	// specified layer.  By default, the layer's parent is used as a reference.
	Changes(from, to string) ([]archive.Change, error)

	// Diff produces a tarstream which can be applied to a layer with the contents
	// of the first layer to produce a layer with the contents of the second layer.
	// By default, the parent of the second layer is used as the first
	// layer, so it need not be specified.  Options can be used to override
	// default behavior, but are also not required.
	Diff(from, to string, options *DiffOptions) (io.ReadCloser, error)

	// DiffSize produces an estimate of the length of the tarstream which would be
	// produced by Diff.
	DiffSize(from, to string) (int64, error)

	// Size produces a cached value for the uncompressed size of the layer,
	// if one is known, or -1 if it is not known.  If the layer can not be
	// found, it returns an error.
	Size(name string) (int64, error)

	// Lookup attempts to translate a name to an ID.  Most methods do this
	// implicitly.
	Lookup(name string) (string, error)

	// LayersByCompressedDigest returns a slice of the layers with the
	// specified compressed digest value recorded for them.
	LayersByCompressedDigest(d digest.Digest) ([]Layer, error)

	// LayersByUncompressedDigest returns a slice of the layers with the
	// specified uncompressed digest value recorded for them.
	LayersByUncompressedDigest(d digest.Digest) ([]Layer, error)

	// Layers returns a slice of the known layers.
	Layers() ([]Layer, error)
}

ROLayerStore wraps a graph driver, adding the ability to refer to layers by name, and keeping track of parent-child relationships, along with a list of all known layers.

type ROMetadataStore 

type ROMetadataStore interface {
	// Metadata reads metadata associated with an item with the specified ID.
	Metadata(id string) (string, error)
}

ROMetadataStore wraps a method for reading metadata associated with an ID.

type RWFileBasedStore 

type RWFileBasedStore interface {
	// Save saves the contents of the store to disk.  It should be called with
	// the lock held, and Touch() should be called afterward before releasing the
	// lock.
	Save() error
}

RWFileBasedStore wraps up the methods of various types of file-based data stores that we implement using read-write files.

type RWImageBigDataStore 

type RWImageBigDataStore interface {
	// SetBigData stores a (potentially large) piece of data associated
	// with this ID.
	// Pass github.com/containers/image/manifest.Digest as digestManifest
	// to allow ByDigest to find images by their correct digests.
	SetBigData(id, key string, data []byte, digestManifest func([]byte) (digest.Digest, error)) error
}

A RWImageBigDataStore wraps up how we store big-data associated with images.

type RWMetadataStore 

type RWMetadataStore interface {
	// SetMetadata updates the metadata associated with the item with the specified ID.
	SetMetadata(id, metadata string) error
}

RWMetadataStore wraps a method for setting metadata associated with an ID.

type Store 

type Store interface {
	// RunRoot, GraphRoot, GraphDriverName, and GraphOptions retrieve
	// settings that were passed to GetStore() when the object was created.
	RunRoot() string
	GraphRoot() string
	GraphDriverName() string
	GraphOptions() []string
	UIDMap() []idtools.IDMap
	GIDMap() []idtools.IDMap

	// GraphDriver obtains and returns a handle to the graph Driver object used
	// by the Store.
	GraphDriver() (drivers.Driver, error)

	// CreateLayer creates a new layer in the underlying storage driver,
	// optionally having the specified ID (one will be assigned if none is
	// specified), with the specified layer (or no layer) as its parent,
	// and with optional names.  (The writeable flag is ignored.)
	CreateLayer(id, parent string, names []string, mountLabel string, writeable bool, options *LayerOptions) (*Layer, error)

	// PutLayer combines the functions of CreateLayer and ApplyDiff,
	// marking the layer for automatic removal if applying the diff fails
	// for any reason.
	//
	// Note that we do some of this work in a child process.  The calling
	// process's main() function needs to import our pkg/reexec package and
	// should begin with something like this in order to allow us to
	// properly start that child process:
	//   if reexec.Init() {
	//       return
	//   }
	PutLayer(id, parent string, names []string, mountLabel string, writeable bool, options *LayerOptions, diff io.Reader) (*Layer, int64, error)

	// CreateImage creates a new image, optionally with the specified ID
	// (one will be assigned if none is specified), with optional names,
	// referring to a specified image, and with optional metadata.  An
	// image is a record which associates the ID of a layer with a
	// additional bookkeeping information which the library stores for the
	// convenience of its caller.
	CreateImage(id string, names []string, layer, metadata string, options *ImageOptions) (*Image, error)

	// CreateContainer creates a new container, optionally with the
	// specified ID (one will be assigned if none is specified), with
	// optional names, using the specified image's top layer as the basis
	// for the container's layer, and assigning the specified ID to that
	// layer (one will be created if none is specified).  A container is a
	// layer which is associated with additional bookkeeping information
	// which the library stores for the convenience of its caller.
	CreateContainer(id string, names []string, image, layer, metadata string, options *ContainerOptions) (*Container, error)

	// Metadata retrieves the metadata which is associated with a layer,
	// image, or container (whichever the passed-in ID refers to).
	Metadata(id string) (string, error)

	// SetMetadata updates the metadata which is associated with a layer,
	// image, or container (whichever the passed-in ID refers to) to match
	// the specified value.  The metadata value can be retrieved at any
	// time using Metadata, or using Layer, Image, or Container and reading
	// the object directly.
	SetMetadata(id, metadata string) error

	// Exists checks if there is a layer, image, or container which has the
	// passed-in ID or name.
	Exists(id string) bool

	// Status asks for a status report, in the form of key-value pairs,
	// from the underlying storage driver.  The contents vary from driver
	// to driver.
	Status() ([][2]string, error)

	// Delete removes the layer, image, or container which has the
	// passed-in ID or name.  Note that no safety checks are performed, so
	// this can leave images with references to layers which do not exist,
	// and layers with references to parents which no longer exist.
	Delete(id string) error

	// DeleteLayer attempts to remove the specified layer.  If the layer is the
	// parent of any other layer, or is referred to by any images, it will return
	// an error.
	DeleteLayer(id string) error

	// DeleteImage removes the specified image if it is not referred to by
	// any containers.  If its top layer is then no longer referred to by
	// any other images and is not the parent of any other layers, its top
	// layer will be removed.  If that layer's parent is no longer referred
	// to by any other images and is not the parent of any other layers,
	// then it, too, will be removed.  This procedure will be repeated
	// until a layer which should not be removed, or the base layer, is
	// reached, at which point the list of removed layers is returned.  If
	// the commit argument is false, the image and layers are not removed,
	// but the list of layers which would be removed is still returned.
	DeleteImage(id string, commit bool) (layers []string, err error)

	// DeleteContainer removes the specified container and its layer.  If
	// there is no matching container, or if the container exists but its
	// layer does not, an error will be returned.
	DeleteContainer(id string) error

	// Wipe removes all known layers, images, and containers.
	Wipe() error

	// Mount attempts to mount a layer, image, or container for access, and
	// returns the pathname if it succeeds.
	// Note if the mountLabel == "", the default label for the container
	// will be used.
	//
	// Note that we do some of this work in a child process.  The calling
	// process's main() function needs to import our pkg/reexec package and
	// should begin with something like this in order to allow us to
	// properly start that child process:
	//   if reexec.Init() {
	//       return
	//   }
	Mount(id, mountLabel string) (string, error)

	// Unmount attempts to unmount a layer, image, or container, given an ID, a
	// name, or a mount path. Returns whether or not the layer is still mounted.
	Unmount(id string, force bool) (bool, error)

	// Mounted returns number of times the layer has been mounted.
	Mounted(id string) (int, error)

	// Changes returns a summary of the changes which would need to be made
	// to one layer to make its contents the same as a second layer.  If
	// the first layer is not specified, the second layer's parent is
	// assumed.  Each Change structure contains a Path relative to the
	// layer's root directory, and a Kind which is either ChangeAdd,
	// ChangeModify, or ChangeDelete.
	Changes(from, to string) ([]archive.Change, error)

	// DiffSize returns a count of the size of the tarstream which would
	// specify the changes returned by Changes.
	DiffSize(from, to string) (int64, error)

	// Diff returns the tarstream which would specify the changes returned
	// by Changes.  If options are passed in, they can override default
	// behaviors.
	Diff(from, to string, options *DiffOptions) (io.ReadCloser, error)

	// ApplyDiff applies a tarstream to a layer.  Information about the
	// tarstream is cached with the layer.  Typically, a layer which is
	// populated using a tarstream will be expected to not be modified in
	// any other way, either before or after the diff is applied.
	//
	// Note that we do some of this work in a child process.  The calling
	// process's main() function needs to import our pkg/reexec package and
	// should begin with something like this in order to allow us to
	// properly start that child process:
	//   if reexec.Init() {
	//       return
	//   }
	ApplyDiff(to string, diff io.Reader) (int64, error)

	// LayersByCompressedDigest returns a slice of the layers with the
	// specified compressed digest value recorded for them.
	LayersByCompressedDigest(d digest.Digest) ([]Layer, error)

	// LayersByUncompressedDigest returns a slice of the layers with the
	// specified uncompressed digest value recorded for them.
	LayersByUncompressedDigest(d digest.Digest) ([]Layer, error)

	// LayerSize returns a cached approximation of the layer's size, or -1
	// if we don't have a value on hand.
	LayerSize(id string) (int64, error)

	// LayerParentOwners returns the UIDs and GIDs of owners of parents of
	// the layer's mountpoint for which the layer's UID and GID maps (if
	// any are defined) don't contain corresponding IDs.
	LayerParentOwners(id string) ([]int, []int, error)

	// Layers returns a list of the currently known layers.
	Layers() ([]Layer, error)

	// Images returns a list of the currently known images.
	Images() ([]Image, error)

	// Containers returns a list of the currently known containers.
	Containers() ([]Container, error)

	// Names returns the list of names for a layer, image, or container.
	Names(id string) ([]string, error)

	// SetNames changes the list of names for a layer, image, or container.
	// Duplicate names are removed from the list automatically.
	SetNames(id string, names []string) error

	// ListImageBigData retrieves a list of the (possibly large) chunks of
	// named data associated with an image.
	ListImageBigData(id string) ([]string, error)

	// ImageBigData retrieves a (possibly large) chunk of named data
	// associated with an image.
	ImageBigData(id, key string) ([]byte, error)

	// ImageBigDataSize retrieves the size of a (possibly large) chunk
	// of named data associated with an image.
	ImageBigDataSize(id, key string) (int64, error)

	// ImageBigDataDigest retrieves the digest of a (possibly large) chunk
	// of named data associated with an image.
	ImageBigDataDigest(id, key string) (digest.Digest, error)

	// SetImageBigData stores a (possibly large) chunk of named data
	// associated with an image.  Pass
	// github.com/containers/image/manifest.Digest as digestManifest to
	// allow ImagesByDigest to find images by their correct digests.
	SetImageBigData(id, key string, data []byte, digestManifest func([]byte) (digest.Digest, error)) error

	// ImageSize computes the size of the image's layers and ancillary data.
	ImageSize(id string) (int64, error)

	// ListContainerBigData retrieves a list of the (possibly large) chunks of
	// named data associated with a container.
	ListContainerBigData(id string) ([]string, error)

	// ContainerBigData retrieves a (possibly large) chunk of named data
	// associated with a container.
	ContainerBigData(id, key string) ([]byte, error)

	// ContainerBigDataSize retrieves the size of a (possibly large)
	// chunk of named data associated with a container.
	ContainerBigDataSize(id, key string) (int64, error)

	// ContainerBigDataDigest retrieves the digest of a (possibly large)
	// chunk of named data associated with a container.
	ContainerBigDataDigest(id, key string) (digest.Digest, error)

	// SetContainerBigData stores a (possibly large) chunk of named data
	// associated with a container.
	SetContainerBigData(id, key string, data []byte) error

	// ContainerSize computes the size of the container's layer and ancillary
	// data.  Warning:  this is a potentially expensive operation.
	ContainerSize(id string) (int64, error)

	// Layer returns a specific layer.
	Layer(id string) (*Layer, error)

	// Image returns a specific image.
	Image(id string) (*Image, error)

	// ImagesByTopLayer returns a list of images which reference the specified
	// layer as their top layer.  They will have different IDs and names
	// and may have different metadata, big data items, and flags.
	ImagesByTopLayer(id string) ([]*Image, error)

	// ImagesByDigest returns a list of images which contain a big data item
	// named ImageDigestBigDataKey whose contents have the specified digest.
	ImagesByDigest(d digest.Digest) ([]*Image, error)

	// Container returns a specific container.
	Container(id string) (*Container, error)

	// ContainerByLayer returns a specific container based on its layer ID or
	// name.
	ContainerByLayer(id string) (*Container, error)

	// ContainerDirectory returns a path of a directory which the caller
	// can use to store data, specific to the container, which the library
	// does not directly manage.  The directory will be deleted when the
	// container is deleted.
	ContainerDirectory(id string) (string, error)

	// SetContainerDirectoryFile is a convenience function which stores
	// a piece of data in the specified file relative to the container's
	// directory.
	SetContainerDirectoryFile(id, file string, data []byte) error

	// FromContainerDirectory is a convenience function which reads
	// the contents of the specified file relative to the container's
	// directory.
	FromContainerDirectory(id, file string) ([]byte, error)

	// ContainerRunDirectory returns a path of a directory which the
	// caller can use to store data, specific to the container, which the
	// library does not directly manage.  The directory will be deleted
	// when the host system is restarted.
	ContainerRunDirectory(id string) (string, error)

	// SetContainerRunDirectoryFile is a convenience function which stores
	// a piece of data in the specified file relative to the container's
	// run directory.
	SetContainerRunDirectoryFile(id, file string, data []byte) error

	// FromContainerRunDirectory is a convenience function which reads
	// the contents of the specified file relative to the container's run
	// directory.
	FromContainerRunDirectory(id, file string) ([]byte, error)

	// ContainerParentOwners returns the UIDs and GIDs of owners of parents
	// of the container's layer's mountpoint for which the layer's UID and
	// GID maps (if any are defined) don't contain corresponding IDs.
	ContainerParentOwners(id string) ([]int, []int, error)

	// Lookup returns the ID of a layer, image, or container with the specified
	// name or ID.
	Lookup(name string) (string, error)

	// Shutdown attempts to free any kernel resources which are being used
	// by the underlying driver.  If "force" is true, any mounted (i.e., in
	// use) layers are unmounted beforehand.  If "force" is not true, then
	// layers being in use is considered to be an error condition.  A list
	// of still-mounted layers is returned along with possible errors.
	Shutdown(force bool) (layers []string, err error)

	// Version returns version information, in the form of key-value pairs, from
	// the storage package.
	Version() ([][2]string, error)

	// GetDigestLock returns digest-specific Locker.
	GetDigestLock(digest.Digest) (Locker, error)
}

Store wraps up the various types of file-based stores that we use into a singleton object that initializes and manages them all together.

func GetStore 

func GetStore(options StoreOptions) (Store, error)

GetStore attempts to find an already-created Store object matching the specified location and graph driver, and if it can't, it creates and initializes a new Store object, and the underlying storage that it controls.

If StoreOptions `options` haven't been fully populated, then DefaultStoreOptions are used.

These defaults observe environment variables:

  • `STORAGE_DRIVER` for the name of the storage driver to attempt to use
  • `STORAGE_OPTS` for the string of options to pass to the driver

Note that we do some of this work in a child process. The calling process's main() function needs to import our pkg/reexec package and should begin with something like this in order to allow us to properly start that child process: 

if reexec.Init() {
    return
}

type StoreOptions 

type StoreOptions struct {
	// RunRoot is the filesystem path under which we can store run-time
	// information, such as the locations of active mount points, that we
	// want to lose if the host is rebooted.
	RunRoot string `json:"runroot,omitempty"`
	// GraphRoot is the filesystem path under which we will store the
	// contents of layers, images, and containers.
	GraphRoot string `json:"root,omitempty"`
	// GraphDriverName is the underlying storage driver that we'll be
	// using.  It only needs to be specified the first time a Store is
	// initialized for a given RunRoot and GraphRoot.
	GraphDriverName string `json:"driver,omitempty"`
	// GraphDriverOptions are driver-specific options.
	GraphDriverOptions []string `json:"driver-options,omitempty"`
	// UIDMap and GIDMap are used for setting up a container's root filesystem
	// for use inside of a user namespace where UID mapping is being used.
	UIDMap []idtools.IDMap `json:"uidmap,omitempty"`
	GIDMap []idtools.IDMap `json:"gidmap,omitempty"`
}

StoreOptions is used for passing initialization options to GetStore(), for initializing a Store object and the underlying storage that it controls.

func DefaultStoreOptions 

func DefaultStoreOptions(rootless bool, rootlessUid int) (StoreOptions, error)

DefaultStoreOptions returns the default storage ops for containers

func DefaultStoreOptionsAutoDetectUID 

func DefaultStoreOptionsAutoDetectUID() (StoreOptions, error)

DefaultStoreOptionsAutoDetectUID returns the default storage ops for containers

Source Files

View all Source files

Directories

Path Synopsis
cmd
containers-storage
aufs
btrfs
copy
devmapper
graphtest
overlay
overlayutils
quota
register
vfs
windows
zfs
pkg
archive
chrootarchive
config
devicemapper
directory
dmesg
fileutils
fsutils
homedir
idtools
ioutils
locker
Package locker provides a mechanism for creating finer-grained locking to help free up more global locks to handle other tasks.
 Package locker provides a mechanism for creating finer-grained locking to help free up more global locks to handle other tasks.
longpath
loopback
mflag
mflag/example
mount
ostree
parsers
Package parsers provides helper functions to parse and validate different type of string.
 Package parsers provides helper functions to parse and validate different type of string.
parsers/kernel
Package kernel provides helper function to get, parse and compare kernel versions for different platforms.
 Package kernel provides helper function to get, parse and compare kernel versions for different platforms.
parsers/operatingsystem
Package operatingsystem provides helper function to get the operating system name for different platforms.
 Package operatingsystem provides helper function to get the operating system name for different platforms.
pools
Package pools provides a collection of pools which provide various data types with buffers.
 Package pools provides a collection of pools which provide various data types with buffers.
promise
reexec
stringid
Package stringid provides helper functions for dealing with string identifiers
 Package stringid provides helper functions for dealing with string identifiers
stringutils
Package stringutils provides helper functions for dealing with strings.
 Package stringutils provides helper functions for dealing with strings.
system
truncindex
Package truncindex provides a general 'index tree', used by Docker in order to be able to reference containers by only a few unambiguous characters of their id.
 Package truncindex provides a general 'index tree', used by Docker in order to be able to reference containers by only a few unambiguous characters of their id.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.