store

Documentation

Index

• Constants
• Variables
• type Blob
• type BlobData
• type DB
  • func NewDB(dbdir string) (*DB, error)
  • func (db *DB) Do(fns ...txfunc) error
• type Data
• type Store
  • func NewStore(storeDir string) (*Store, error)
  • func (s *Store) Close() error
  • func (s *Store) DeleteBlob(digest string) error
  • func (s *Store) DeleteBlobData(digest, dataType string) error
  • func (s *Store) DeleteData(key, dataType string) error
  • func (s *Store) GC() error
  • func (s *Store) GetBlobData(digest, dataType string) (*BlobData, error)
  • func (s *Store) GetBlobInfo(digest string) (*Blob, error)
  • func (s *Store) GetData(key, dataType string) (*Data, error)
  • func (s *Store) ListAllDataKeys(dataType string) ([]string, error)
  • func (s *Store) ListBlobs(mediaType string) ([]string, error)
  • func (s *Store) ReadBlob(digest string) (io.ReadCloser, error)
  • func (s *Store) ResolveDigest(inDigest string) (string, error)
  • func (s *Store) SetBlobData(digest string, data []byte, dataType string) error
  • func (s *Store) SetData(key string, value []byte, dataType, digest string) error
  • func (s *Store) TmpDir() (string, error)
  • func (s *Store) TmpFile() (*os.File, error)
  • func (s *Store) WriteBlob(r io.ReadSeeker, mediaType string) (string, error)

Constants

const (
	DbFilename = "ql.db"
)

Variables

var (
	ErrDigestNotFound  = errors.New("no digest found")
	ErrMultipleDigests = errors.New("found multiple digests")
	// ErrRemovingBlobContents defines an error removing non transactional (diskv) blob contents
	// When this happen there's the possibility that the store is left in an
	// unclean state (for example with some stale files). This will be fixed by store GC
	ErrRemovingBlobContents = errors.New("failed to remove blob disk contents")
)

Types

type Blob

type Blob struct {
	Digest    string
	MediaType string
}

Blob is used to index a Blob.

type BlobData

type BlobData []byte

type DB

type DB struct {
	// contains filtered or unexported fields
}

func NewDB

func NewDB(dbdir string) (*DB, error)

func (*DB) Do

func (db *DB) Do(fns ...txfunc) error

Do take an exclusive flock, opens the db, executes DoTx and then Closes the DB It's the unique function exported since it's the one that takes the exclusive flock.

type Data

type Data struct {
	Key      string
	Value    []byte
	DataType string
	Digest   string
}

Data is used to index data.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store encapsulates a content-addressable-storage for storing blobs on disk. It's an indexed CAS with the ability to augment a blob with additional data and a key value store (with optional references to a blob)

func NewStore

func NewStore(storeDir string) (*Store, error)

func (*Store) Close

func (s *Store) Close() error

Close closes a Store opened with NewStore().

func (*Store) DeleteBlob

func (s *Store) DeleteBlob(digest string) error

DeleteBlob removes a blob. It firstly removes the blob infos from the db, then it tries to remove the non transactional data. If some error occurs removing some non transactional data a ErrRemovingBlobContents is returned. The user may ignore this error since a followin GC will try to remove it and the blob won't be available with the other functions.

func (*Store) DeleteBlobData

func (s *Store) DeleteBlobData(digest, dataType string) error

DelBlobData removes additional data of dataType from a blob

func (*Store) DeleteData

func (s *Store) DeleteData(key, dataType string) error

DeleteData removes additional data with the specified key and dataType

func (*Store) GC

func (s *Store) GC() error

GC garbage collects stale blobs contents (blobs not appering in the db) (this can happen if they aren't removed for whatever reason) and remove unused keylocks

func (*Store) GetBlobData

func (s *Store) GetBlobData(digest, dataType string) (*BlobData, error)

GetBlobData gets additional data of dataType from a blob

func (*Store) GetBlobInfo

func (s *Store) GetBlobInfo(digest string) (*Blob, error)

GetBlobInfo return blob info for the provided blob digest If the blob doesn't exists the return value will be nil

func (*Store) GetData

func (s *Store) GetData(key, dataType string) (*Data, error)

GetData gets additional data from a blob

func (*Store) ListAllDataKeys

func (s *Store) ListAllDataKeys(dataType string) ([]string, error)

ListData returns a list of all data for the given dataType

func (*Store) ListBlobs

func (s *Store) ListBlobs(mediaType string) ([]string, error)

ListBlobs returns a list of all blob digest of the given mediaType

func (*Store) ReadBlob

func (s *Store) ReadBlob(digest string) (io.ReadCloser, error)

ReadBlob return an io.ReadCloser for the requested digest

func (*Store) ResolveDigest

func (s *Store) ResolveDigest(inDigest string) (string, error)

ResolveDigest resolves a partial digest (of format `sha256-0c45e8c0ab2`) to a full digest by considering the digest prefix and using the store for resolution.

func (*Store) SetBlobData

func (s *Store) SetBlobData(digest string, data []byte, dataType string) error

SetBlobData set additional data of dataType to a blob It'll be removed when the blob is removed

func (*Store) SetData

func (s *Store) SetData(key string, value []byte, dataType, digest string) error

SetData adds additional data of dataType. The data has an unique key to easily list and search by key If digest is not nil it can reference a blob digest and it'll be removed when the blob is removed. This can be useful to add custom data to the store. For example: * oci references where key is the ref name and data is an oci descriptor * remote caching information for an appc ACI store where key is the download URL, data contains additional data (like http caching information) and if references a digest to an ACI blob. TODO(sgotti) add a version field and a ErrConcurrentUpdate as a facility for clients to do optimistic locking? it should be useful to handle data migration without chaning the dataType

func (*Store) TmpDir

func (s *Store) TmpDir() (string, error)

TmpDir creates and returns dir local to the same filesystem as the Store, or any error encountered

func (*Store) TmpFile

func (s *Store) TmpFile() (*os.File, error)

TmpFile returns an *os.File local to the same filesystem as the Store, or any error encountered

func (*Store) WriteBlob

func (s *Store) WriteBlob(r io.ReadSeeker, mediaType string) (string, error)

WriteBlob takes a blob encapsulated in an io.Reader, calculates its digest and then stores it in the blob store mediaType can be used to define a blob mediaType for later fetching of all the blobs my mediaType