Documentation
¶
Overview ¶
Package storage defines the low-level store (and store manager) interface
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Address ¶
type Address uint64
Address used to refer to a message.
This is a 64-bit value that can be thought of as an 'internal key' that is specific to the low-level storage) that can be used to retrieve the message faster/more optimally than with the "key". The "Seek(..)" call could be used to translate the actual message Key to an Address.
const EOX Address = (1 << 64) - 2
EOX indicates the End Of eXtent (no more messages in the extent); this value needs to be larger than any valid address
const InvalidAddr Address = (1 << 64) - 1
InvalidAddr is a special value used to return an address that is "invalid"
const MaxAddr Address = (1 << 64) - 2
MaxAddr is equal to EOX, and is the largest valid address + 1
const MinAddr Address = 0
MinAddr is the smallest valid address for a message
type ExtentInfo ¶ added in v1.26.0
type ExtentInfo struct {
Size int64 // estimated size of extent on disk
Modified int64 // time extent was modified (in unix-nano)
}
ExtentInfo contains information about an extent
type ExtentStore ¶
type ExtentStore interface {
// Put inserts message into the extent against the (optional) 'key'
// Args:
// key: key to associate with value
// - for timer-queues, this is a combination of the delivery
// time (enqueue-time + delay) and seqNum
// - for append-only, this could just be the msg seq-num
// (either way, the assumption is the key is _unique_ (does
// not overwrite))
// val: byte array of message payload; StoreHost is expected to
// serialize the message
//
// Returns:
// addr: address of the message that was just inserted.
// this is returned only if available; this could be
// InvalidAddr, in case it isnt' readily available for
// consumption (like in chunky)
// err: error
Put(key Key, val Value) (addr Address, err error)
Sync()
// Get takes in the "address" of the message to read
// Args:
// addr: Address of the message to retrieve
//
// Returns:
// key: key of the message
// val: value (payload) of the message
// next: Address of the very next message (if available)
// err: error, if any
Get(addr Address) (key Key, val Value, nextAddr Address, nextKey Key, err error)
// GetMany returns a batch of max 'numMsgs' between [addr, maxAddr)
// Args:
// numMsgs: max number of messages to return
// addr: Address of the first message to retrieve
// maxAddr: upper bound of the address for messages to be returned;
// can be 'EOX' for to get all messages starting from 'addr'.
//
// Returns:
// msgs: array of key-value pair of messages in successive order
// of addresses
// nextAddr: Address of the very next message that follows the last
// one returned in 'msgs'
// nextKey: Key of the very next message that follows the last one
// returned in 'msgs'
// err: error, if any
GetMany(addr Address, numMsgs int32, maxAddr Address) (msgs []KeyValue, nextAddr Address, nextKey Key, err error)
// Next returns the address of the next message (in address order)
// Args:
// addr: Address of the message whose 'next' is being queried. This
// could refer to an address that does not exist, in which case
// the expectation is to return the address of the first
// message whose address is greater than the given address.
//
// Returns:
// nextAddr: Address of the message after 'addr'
// nextKey: Key of the next message (corresponding to nextAddr)
// err: error, if any
Next(addr Address) (nextAddr Address, nextKey Key, err error)
// SeekCeiling returns the address of the message for the given 'key' or the
// first one following it (basically message with key ">=" given key).
// If there are no such key exists, it would return an error, with
// 'addr == EOX' and 'retKey == InvalidKey'.
// Args:
// ceilKey: key of the message to 'ceil' to
//
// Returns:
// addr: address of the message whose key is equal or greater than
// the specified key
// key: key of the message at 'addr' (could be equal to 'key' or
// greater than it)
// err: error, if any
//
SeekCeiling(ceilKey Key) (addr Address, key Key, err error)
// SeekFloor returns the address of the message for the given 'key' or the
// first one following it (basically message with key "<=" given key).
// If there are no such key exists, it would return an error, with
// 'addr == MinAddr' and 'retKey == InvalidKey'.
// Args:
// floorKey: key of the message to find 'floor' for
//
// Returns:
// addr: address of the message whose key is equal or less than
// the specified key
// key: key of the message at 'addr' (could be equal to 'key' or
// less than it)
// err: error, if any
//
SeekFloor(floorKey Key) (addr Address, key Key, err error)
// SeekFirst returns the address and key of the first message available
// Args:
// none
//
// Returns:
// addr: address of the first message
// key: key of the first message
// err: error, if any
SeekFirst() (addr Address, key Key, err error)
// SeekLast returns the address and key of the last message available
// Args:
// none
//
// Returns:
// addr: address of the last message
// key: key of the last message
// err: error, if any
SeekLast() (addr Address, key Key, err error)
// GetKey returns the key of the message at the given address. In a
// sense, it is the inverse of the 'Seek' operation, in that it finds
// the key from the address.
// Args:
// addr: the message whose key is sought
//
// Returns:
// key: key of the message
// err: error, if any
//
GetKey(addr Address) (key Key, err error)
// Purge deletes all messages whose address is less than or equal to the
// given address (including the message with the given address). Once a
// purge is accepted, no messages less than or equal to the given
// address should be visible
// Args:
// addr: address upto which all the messages are to be purged
//
// Returns:
// nextAddr: the address of the next available message, if exists
// nextKey: the key of the next available message, if exists
// err: error, if any
Purge(endAddr Address) (nextAddr Address, nextKey Key, err error)
// DeleteExtent deletes all messages and data associated with the extent
// when the extent is closed. In other words, marks this extent for delete
// on close.
DeleteExtent()
// Close does necessary clean-up to close this extent
// Args:
// none
//
// Returns:
// none
Close()
// CloseExtentDB is closes the handle when opened using OpenExtentDB
// Args:
// none
//
// Returns:
// none
CloseExtentDB()
}
ExtentStore is the interface used to read/write messages for an extent
type ExtentUUID ¶
ExtentUUID is identifies the extent
func (ExtentUUID) String ¶
func (t ExtentUUID) String() string
String implements the Stringer interface for ExtentUUID
type Key ¶
type Key uint64
Key to associate the message with
This is a 64-bit value created by StoreHost:
- for timer-queues, this could be a combination of seq-num and delivery
time (enqueue-time + delay) to ensure that the key is unique.
- for append-only streams, this could be a combination of enqueue-time
& seq-num (the key needs to be inserted in increasing order for
'chunky' for the "Seek()" operation to work efficiently)
const InvalidKey Key = (1 << 64) - 1
InvalidKey indicates an invalid (or unknown) key
type KeyPattern ¶
type KeyPattern int
KeyPattern is an enum indicating the pattern of the keys that will be used to store values into the extent.
const ( // RandomKeys indicates that the keys could be in completely // random order. RandomKeys KeyPattern // IncreasingKeys indicates that the keys used to store // values will be in strictly increasing order; ie, each key // associatd with a value will be strictly greater than that // used for the previous write. IncreasingKeys )
func (KeyPattern) String ¶
func (t KeyPattern) String() string
String implements the Stringer interface for KeyPattern
type NotifyFunc ¶
NotifyFunc is the callback used to notify when new messages are available to read. It includes the "key" of the newly available message and its address. For extents with "RandomKeys" pattern, this callback will be called for every single message. For those with "IncreasingKeys" pattern, the callback may *not* be called for every message -- when the callback is called, the provided key would be the "high-water mark" of the available values. NotifyFunc is also called when messages in an extent have been deleted; in this case it would be passed a key equal to 'InvalidKey'.
type StoreManager ¶
type StoreManager interface {
// OpenExtent opens DB and initializes the state for a given extent.
// The extent is opened so it is available for read or write.
// (FIXME: This should probably take in some manager-specific
// configuration parameters, as an empty struct/interface?)
OpenExtent(id ExtentUUID, pattern KeyPattern, notify NotifyFunc, failIfNotExist bool) (ExtentStore, error)
// ListExtents returns list of available extents
ListExtents() ([]ExtentUUID, error)
// GetExtentInfo returns information about an extent
GetExtentInfo(id ExtentUUID) (*ExtentInfo, error)
}
StoreManager is the storage interface implemented by the low-level stores (currently, "Chunky" (append-only) and "Rockstor" (primarily, for indexed but can be used for append-only)
Source Files
Directories
Click to show internal directories.
Click to hide internal directories.