Documentation ¶
Index ¶
- Constants
- Variables
- func PutAll(ctx context.Context, store KeyValueStore, items ...ListItem) (err error)
- type BlobInfo
- type BlobReader
- type BlobRef
- type BlobWriter
- type Blobs
- type FormatVersion
- type Items
- type IterateOptions
- type Iterator
- type IteratorFunc
- type Key
- type KeyValueStore
- type Keys
- type ListItem
- type ListOptions
- type StaticIterator
- type Value
- type Values
Constants ¶
const Delimiter = '/'
Delimiter separates nested paths in storage
const LookupLimit = 1000
LookupLimit is enforced by storage implementations
Variables ¶
var ErrEmptyKey = errs.Class("empty key")
ErrEmptyKey is returned when an empty key is used in Put or in CompareAndSwap
var ErrEmptyQueue = errs.Class("empty queue")
ErrEmptyQueue is returned when attempting to Dequeue from an empty queue
var ErrInvalidBlobRef = errs.Class("invalid blob ref")
ErrInvalidBlobRef is returned when an blob reference is invalid
var ErrKeyNotFound = errs.Class("key not found")
ErrKeyNotFound used when something doesn't exist
var ErrLimitExceeded = errors.New("limit exceeded")
ErrLimitExceeded is returned when request limit is exceeded
var ErrValueChanged = errs.Class("value changed")
ErrValueChanged is returned when the current value of the key does not match the oldValue in CompareAndSwap
Functions ¶
Types ¶
type BlobInfo ¶ added in v0.18.0
type BlobInfo interface { // BlobRef returns the relevant BlobRef for the blob BlobRef() BlobRef // StorageFormatVersion indicates the storage format version used to store the piece StorageFormatVersion() FormatVersion // FullPath gives the full path to the on-disk blob file FullPath(ctx context.Context) (string, error) // Stat does a stat on the on-disk blob file Stat(ctx context.Context) (os.FileInfo, error) }
BlobInfo allows lazy inspection of a blob and its underlying file during iteration with WalkNamespace-type methods
type BlobReader ¶
type BlobReader interface { io.Reader io.ReaderAt io.Seeker io.Closer // Size returns the size of the blob Size() (int64, error) // StorageFormatVersion returns the storage format version associated with the blob. StorageFormatVersion() FormatVersion }
BlobReader is an interface that groups Read, ReadAt, Seek and Close.
type BlobWriter ¶
type BlobWriter interface { io.Writer io.Seeker // Cancel discards the blob. Cancel(context.Context) error // Commit ensures that the blob is readable by others. Commit(context.Context) error // Size returns the size of the blob Size() (int64, error) // StorageFormatVersion returns the storage format version associated with the blob. StorageFormatVersion() FormatVersion }
BlobWriter defines the interface that must be satisfied for a general blob storage provider. BlobWriter instances are returned by the Create() method on Blobs instances.
type Blobs ¶
type Blobs interface { // Create creates a new blob that can be written // optionally takes a size argument for performance improvements, -1 is unknown size Create(ctx context.Context, ref BlobRef, size int64) (BlobWriter, error) // Open opens a reader with the specified namespace and key Open(ctx context.Context, ref BlobRef) (BlobReader, error) // OpenWithStorageFormat opens a reader for the already-located blob, avoiding the potential // need to check multiple storage formats to find the blob. OpenWithStorageFormat(ctx context.Context, ref BlobRef, formatVer FormatVersion) (BlobReader, error) // Delete deletes the blob with the namespace and key Delete(ctx context.Context, ref BlobRef) error // DeleteWithStorageFormat deletes a blob of a specific storage format DeleteWithStorageFormat(ctx context.Context, ref BlobRef, formatVer FormatVersion) error // Trash marks a file for pending deletion Trash(ctx context.Context, ref BlobRef) error // RestoreTrash restores all files in the trash for a given namespace and returns the keys restored RestoreTrash(ctx context.Context, namespace []byte) ([][]byte, error) // EmptyTrash removes all files in trash that were moved to trash prior to trashedBefore and returns the total bytes emptied and keys deleted EmptyTrash(ctx context.Context, namespace []byte, trashedBefore time.Time) (int64, [][]byte, error) // Stat looks up disk metadata on the blob file Stat(ctx context.Context, ref BlobRef) (BlobInfo, error) // StatWithStorageFormat looks up disk metadata for the blob file with the given storage format // version. This avoids the potential need to check multiple storage formats for the blob // when the format is already known. StatWithStorageFormat(ctx context.Context, ref BlobRef, formatVer FormatVersion) (BlobInfo, error) // FreeSpace return how much free space is available to the blobstore FreeSpace() (int64, error) // SpaceUsedForTrash returns the total space used by the trash SpaceUsedForTrash(ctx context.Context) (int64, error) // SpaceUsedForBlobs adds up how much is used in all namespaces SpaceUsedForBlobs(ctx context.Context) (int64, error) // SpaceUsedForBlobsInNamespace adds up how much is used in the given namespace SpaceUsedForBlobsInNamespace(ctx context.Context, namespace []byte) (int64, error) // ListNamespaces finds all namespaces in which keys might currently be stored. ListNamespaces(ctx context.Context) ([][]byte, error) // WalkNamespace executes walkFunc for each locally stored blob, stored with // storage format V1 or greater, in the given namespace. If walkFunc returns a non-nil // error, WalkNamespace will stop iterating and return the error immediately. The ctx // parameter is intended to allow canceling iteration early. WalkNamespace(ctx context.Context, namespace []byte, walkFunc func(BlobInfo) error) error // Close closes the blob store and any resources associated with it. Close() error }
Blobs is a blob storage interface.
architecture: Database
type FormatVersion ¶ added in v0.18.0
type FormatVersion int
FormatVersion represents differing storage format version values. Different Blobs implementors might interpret different FormatVersion values differently, but they share a type so that there can be a common StorageFormatVersion() call on the interface.
Changes in FormatVersion might affect how a Blobs or BlobReader or BlobWriter instance works, or they might only be relevant to some higher layer. A FormatVersion must be specified when writing a new blob, and the blob storage interface must store that value with the blob somehow, so that the same FormatVersion is returned later when reading that stored blob.
type Items ¶
type Items []ListItem
Items keeps all ListItem
func ListV2 ¶
func ListV2(ctx context.Context, store KeyValueStore, opts ListOptions) (result Items, more bool, err error)
ListV2 lists all keys corresponding to ListOptions limit is capped to LookupLimit
more indicates if the result was truncated. If false then the result []ListItem includes all requested keys. If true then the caller must call List again to get more results by setting `StartAfter` appropriately.
func SelectPrefixed ¶
SelectPrefixed keeps only items that have prefix items will be reused and modified TODO: remove this
func SortAndCollapse ¶
SortAndCollapse sorts items and combines elements based on Delimiter items will be reused and modified TODO: remove this
type IterateOptions ¶
type IterateOptions struct { // Prefix ensure Prefix Key // First will be the first item iterator returns or the next item (previous when reverse) First Key // Recurse, do not collapse items based on Delimiter Recurse bool }
IterateOptions contains options for iterator
type Iterator ¶
type Iterator interface { // Next prepares the next list item. // It returns true on success, or false if there is no next result row or an error happened while preparing it. Next(ctx context.Context, item *ListItem) bool }
Iterator iterates over a sequence of ListItems
type IteratorFunc ¶
IteratorFunc implements basic iterator
type Key ¶
type Key []byte
Key is the type for the keys in a `KeyValueStore`
func (Key) MarshalBinary ¶
MarshalBinary implements the encoding.BinaryMarshaler interface for the Key type
type KeyValueStore ¶
type KeyValueStore interface { // Put adds a value to store Put(context.Context, Key, Value) error // Get gets a value to store Get(context.Context, Key) (Value, error) // GetAll gets all values from the store GetAll(context.Context, Keys) (Values, error) // Delete deletes key and the value Delete(context.Context, Key) error // List lists all keys starting from start and upto limit items List(ctx context.Context, start Key, limit int) (Keys, error) // Iterate iterates over items based on opts Iterate(ctx context.Context, opts IterateOptions, fn func(context.Context, Iterator) error) error // CompareAndSwap atomically compares and swaps oldValue with newValue CompareAndSwap(ctx context.Context, key Key, oldValue, newValue Value) error // Close closes the store Close() error }
KeyValueStore describes key/value stores like redis and boltdb
type Keys ¶
type Keys []Key
Keys is the type for a slice of keys in a `KeyValueStore`
func ListKeys ¶
ListKeys returns keys starting from first and upto limit limit is capped to LookupLimit
func (Keys) ByteSlices ¶
ByteSlices converts a `Keys` struct to a slice of byte-slices (i.e. `[][]byte`)
type ListOptions ¶
type ListOptions struct { Prefix Key StartAfter Key // StartAfter is relative to Prefix Recursive bool IncludeValue bool Limit int }
ListOptions are items that are optional for the LIST method
type StaticIterator ¶
StaticIterator implements an iterator over list of items
Directories ¶
Path | Synopsis |
---|---|
schema
Code generated for package schema by go-bindata DO NOT EDIT.
|
Code generated for package schema by go-bindata DO NOT EDIT. |
redisserver
Package redisserver is package for starting a redis test server
|
Package redisserver is package for starting a redis test server |