api

package
v0.2400.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 13, 2024 License: Apache-2.0 Imports: 11 Imported by: 0

Documentation

Overview

Package api provides a persistent node database interface for MKVS trees.

Index

Constants

View Source 
const ModuleName = "storage/mkvs/db"

ModuleName is the module name.

View Source 
const TypedHashSize = hash.Size + 1

TypedHashSize is the size of the TypedHash.

Variables

View Source 
var (
	// ErrNodeNotFound indicates that a node with the specified hash couldn't be found
	// in the database.
	ErrNodeNotFound = errors.New(ModuleName, 1, "mkvs: node not found in node db")
	// ErrWriteLogNotFound indicates that a write log for the specified storage hashes
	// couldn't be found.
	ErrWriteLogNotFound = errors.New(ModuleName, 2, "mkvs: write log not found in node db")
	// ErrNotFinalized indicates that the operation requires a version to be finalized
	// but the version is not yet finalized.
	ErrNotFinalized = errors.New(ModuleName, 3, "mkvs: version is not yet finalized")
	// ErrAlreadyFinalized indicates that the given version has already been finalized.
	ErrAlreadyFinalized = errors.New(ModuleName, 4, "mkvs: version has already been finalized")
	// ErrVersionNotFound indicates that the given version cannot be found.
	ErrVersionNotFound = errors.New(ModuleName, 5, "mkvs: version not found")
	// ErrPreviousVersionMismatch indicates that the version given for the old root does
	// not match the previous version.
	ErrPreviousVersionMismatch = errors.New(ModuleName, 6, "mkvs: previous version mismatch")
	// ErrVersionWentBackwards indicates that the new version is earlier than an already
	// inserted version.
	ErrVersionWentBackwards = errors.New(ModuleName, 7, "mkvs: version went backwards")
	// ErrRootNotFound indicates that the given root cannot be found.
	ErrRootNotFound = errors.New(ModuleName, 8, "mkvs: root not found")
	// ErrRootMustFollowOld indicates that the passed new root does not follow old root.
	ErrRootMustFollowOld = errors.New(ModuleName, 9, "mkvs: root must follow old root")
	// ErrBadNamespace indicates that the passed namespace does not match what is
	// actually contained within the database.
	ErrBadNamespace = errors.New(ModuleName, 10, "mkvs: bad namespace")
	// ErrNotEarliest indicates that the given version is not the earliest version.
	ErrNotEarliest = errors.New(ModuleName, 11, "mkvs: version is not the earliest version")
	// ErrReadOnly indicates that a write operation failed due to a read-only database.
	ErrReadOnly = errors.New(ModuleName, 12, "mkvs: read-only database")
	// ErrMultipartInProgress indicates that a multipart restore operation is already
	// in progress.
	ErrMultipartInProgress = errors.New(ModuleName, 13, "mkvs: multipart already in progress")
	// ErrInvalidMultipartVersion indicates that a Finalize, NewBatch or Commit was called with a version
	// that doesn't match the current multipart restore as set with StartMultipartRestore.
	ErrInvalidMultipartVersion = errors.New(ModuleName, 14, "mkvs: operation called with different version than current multipart version")
	// ErrUpgradeInProgress indicates that a database upgrade was started by the upgrader tool and the
	// database is therefore unusable. Run the upgrade tool to finish upgrading.
	ErrUpgradeInProgress = errors.New(ModuleName, 15, "mkvs: database upgrade in progress")
	// ErrCannotPruneLatestVersion indicates that the caller attempted to prune the latest finalized
	// version which would leave the database without any finalized versions.
	ErrCannotPruneLatestVersion = errors.New(ModuleName, 16, "mkvs: cannot prune latest version")
)

Functions

func ReviveHashedDBWriteLogs 

func ReviveHashedDBWriteLogs(
	ctx context.Context,
	logGetter func() (node.Root, HashedDBWriteLog, error),
	valueGetter func(node.Root, hash.Hash) (*node.LeafNode, error),
	closer func(),
) (writelog.Iterator, error)

ReviveHashedDBWriteLogs is a helper for hashed database backends that converts a HashedDBWriteLog into a WriteLog.

The provided logGetter will be called first to fetch the next write log to convert. If it returns a nil write log, iteration terminates.

Then the provided valueGetter will be called for each log entry to fetch each of the values in the write log.

After iteration has finished, closer will be called.

func RootTypes added in v0.2400.0 

func RootTypes() []node.RootType

RootTypes returns all supported root types.

func RootTypesWithPolicy added in v0.2400.0 

func RootTypesWithPolicy(policyFn func(*RootPolicy) bool) (types []node.RootType)

RootTypesWithPolicy returns all root types where the given policy predicate evaluates to true.

func Visit 

func Visit(ctx context.Context, ndb NodeDB, root node.Root, visitor NodeVisitor) error

Visit traverses the tree in DFS order using the passed visitor. The traversal is a pre-order DFS where the node is visited first, then its leaf (if any) and then its children (first left then right).

Different to the Visit method in the MKVS tree, this uses the NodeDB API directly to traverse the tree to avoid the overhead of keeping the cache.

Types

type BaseBatch 

type BaseBatch struct {
	// contains filtered or unexported fields
}

BaseBatch encapsulates basic functionality of a batch so it doesn't need to be reimplemented by each concrete batch implementation.

func (*BaseBatch) Commit 

func (b *BaseBatch) Commit(node.Root) error

func (*BaseBatch) OnCommit 

func (b *BaseBatch) OnCommit(hook func())

type Batch 

type Batch interface {
	// MaybeStartSubtree returns a new subtree instance that can be used for
	// persisting nodes under a given root.
	//
	// Depth is the depth of the node that subtreeRoot points to.
	MaybeStartSubtree(subtree Subtree, depth node.Depth, subtreeRoot *node.Pointer) Subtree

	// OnCommit registers a hook to run after a successful commit.
	OnCommit(hook func())

	// PutWriteLog stores the specified write log into the batch.
	PutWriteLog(writeLog writelog.WriteLog, logAnnotations writelog.Annotations) error

	// RemoveNodes marks nodes for eventual garbage collection.
	RemoveNodes(nodes []*node.Pointer) error

	// Commit commits the batch.
	Commit(root node.Root) error

	// Reset resets the batch for another use.
	Reset()
}

Batch is a NodeDB-specific batch implementation.

type Config 

type Config struct {
	// DB is the path to the database.
	DB string

	// NoFsync will disable fsync() where possible.
	NoFsync bool

	// MemoryOnly will make the storage memory-only (if the backend supports it).
	MemoryOnly bool

	// ReadOnly will make the storage read-only.
	ReadOnly bool

	// Namespace is the namespace contained within the database.
	Namespace common.Namespace

	// MaxCacheSize is the maximum in-memory cache size for the database.
	MaxCacheSize int64

	// DiscardWriteLogs will cause all write logs to be discarded.
	DiscardWriteLogs bool
}

Config is the node database backend configuration.

type Factory added in v0.2400.0 

type Factory interface {
	// New creates a new node database.
	New(cfg *Config) (NodeDB, error)

	// Name returns the name of this node database factory.
	Name() string
}

Factory is a node database factory interface that can create new databases.

type HashedDBLogEntry 

type HashedDBLogEntry struct {
	Key          []byte
	InsertedHash *hash.Hash
}

HashedDBLogEntry is a single write log entry for HashedDBWriteLog.

type HashedDBWriteLog 

type HashedDBWriteLog []HashedDBLogEntry

HashedDBWriteLog is a write log helper for database backends that can reference nodes by hash.

func MakeHashedDBWriteLog 

func MakeHashedDBWriteLog(writeLog writelog.WriteLog, annotations writelog.Annotations) HashedDBWriteLog

MakeHashedDBWriteLog converts the given write log and annotations into a serializable slice with hash node references.

type NodeDB 

type NodeDB interface {
	// GetNode looks up a node in the database.
	GetNode(root node.Root, ptr *node.Pointer) (node.Node, error)

	// GetWriteLog retrieves a write log between two storage instances from the database.
	GetWriteLog(ctx context.Context, startRoot, endRoot node.Root) (writelog.Iterator, error)

	// GetLatestVersion returns the most recent version in the node database.
	//
	// The boolean flag signifies whether any version exists to disambiguate version zero.
	GetLatestVersion() (uint64, bool)

	// GetEarliestVersion returns the earliest version in the node database.
	GetEarliestVersion() uint64

	// GetRootsForVersion returns a list of roots stored under the given version.
	GetRootsForVersion(version uint64) ([]node.Root, error)

	// StartMultipartInsert prepares the database for a batch insert job from multiple chunks.
	// Batches from this call onwards will keep track of inserted nodes so that they can be
	// deleted if the job fails for any reason.
	StartMultipartInsert(version uint64) error

	// AbortMultipartInsert cleans up the node insertion log that was kept since the last
	// StartMultipartInsert operation. The log will be cleared and the associated nodes can
	// be either removed (if the insertion failed) or left intact (if it was successful).
	//
	// It is not an error to call this method more than once.
	AbortMultipartInsert() error

	// NewBatch starts a new batch.
	//
	// The chunk argument specifies whether the given batch is being used to import a chunk of an
	// existing root. Chunks may contain unresolved pointers (e.g., pointers that point to hashes
	// which are not present in the database). Committing a chunk batch will prevent the version
	// from being finalized.
	NewBatch(oldRoot node.Root, version uint64, chunk bool) (Batch, error)

	// HasRoot checks whether the given root exists.
	HasRoot(root node.Root) bool

	// Finalize finalizes the version comprising the passed list of finalized roots.
	// All non-finalized roots can be discarded.
	Finalize(roots []node.Root) error

	// Prune removes all roots recorded under the given version.
	//
	// Only the earliest version can be pruned, passing any other version will result in an error.
	Prune(version uint64) error

	// Size returns the size of the database in bytes.
	Size() (int64, error)

	// Sync syncs the database to disk. This is useful if the NoFsync option is used to explicitly
	// perform a sync.
	Sync() error

	// Close closes the database.
	Close()
}

NodeDB is the persistence layer used for persisting the in-memory tree.

func NewNopNodeDB 

func NewNopNodeDB() (NodeDB, error)

NewNopNodeDB creates a new no-op node database.

type NodeVisitor 

type NodeVisitor func(context.Context, node.Node) bool

NodeVisitor is a function that visits a given node and returns true to continue traversal of child nodes or false to stop.

type RootPolicy added in v0.2400.0 

type RootPolicy struct {
	// NoChildRoots means that roots of this type cannot have any children, each version must create
	// a root from scratch and the root is not available in the next version.
	NoChildRoots bool
}

RootPolicy is the storage policy for a given root type.

func PolicyForRoot added in v0.2400.0 

func PolicyForRoot(root node.Root) *RootPolicy

PolicyForRoot returns the storage policy for the given root.

type Subtree 

type Subtree interface {
	// PutNode persists a node in the NodeDB.
	//
	// Depth is the node depth not bit depth.
	PutNode(depth node.Depth, ptr *node.Pointer) error

	// VisitCleanNode is called for any clean node encountered during commit
	// for which no further processing will be done (as it is marked clean).
	//
	// The specific NodeDB implementation may wish to do further processing.
	//
	// Depth is the node depth not bit depth.
	VisitCleanNode(depth node.Depth, ptr *node.Pointer, parent *node.Pointer) error

	// VisitDirtyNode is called before processing any children of a dirty node encountered during
	// commit or checkpoint restore operation.
	//
	// The specific NodeDB implementation may wish to do further processing.
	//
	// Depth is the node depth not bit depth.
	VisitDirtyNode(depth node.Depth, ptr *node.Pointer, parent *node.Pointer) error

	// Commit marks the subtree as complete.
	Commit() error
}

Subtree is a NodeDB-specific subtree implementation.

type TypedHash added in v0.2400.0 

type TypedHash [TypedHashSize]byte

TypedHash is a node hash prefixed with its root type.

func TypedHashFromParts added in v0.2400.0 

func TypedHashFromParts(typ node.RootType, hash hash.Hash) (h TypedHash)

TypedHashFromParts creates a new typed hash with the parts given.

func TypedHashFromRoot added in v0.2400.0 

func TypedHashFromRoot(root node.Root) (h TypedHash)

TypedHashFromRoot creates a new typed hash corresponding to the given storage root.

func (*TypedHash) Equal added in v0.2400.0 

func (h *TypedHash) Equal(cmp *TypedHash) bool

Equal compares vs another hash for equality.

func (*TypedHash) FromParts added in v0.2400.0 

func (h *TypedHash) FromParts(typ node.RootType, hash hash.Hash)

FromParts returns the typed hash composed of the given type and hash.

func (*TypedHash) Hash added in v0.2400.0 

func (h *TypedHash) Hash() (rh hash.Hash)

Hash returns the hash portion of the typed hash.

func (*TypedHash) MarshalBinary added in v0.2400.0 

func (h *TypedHash) MarshalBinary() (data []byte, err error)

MarshalBinary encodes a typed hash into binary form.

func (TypedHash) MarshalText added in v0.2400.0 

func (h TypedHash) MarshalText() (data []byte, err error)

MarshalText encodes a Hash into text form.

func (TypedHash) String added in v0.2400.0 

func (h TypedHash) String() string

String returns the string representation of a typed hash.

func (*TypedHash) Type added in v0.2400.0 

func (h *TypedHash) Type() node.RootType

Type returns the storage type of the root corresponding to this typed hash.

func (*TypedHash) UnmarshalBinary added in v0.2400.0 

func (h *TypedHash) UnmarshalBinary(data []byte) error

UnmarshalBinary decodes a binary marshaled hash.

func (*TypedHash) UnmarshalHex added in v0.2400.0 

func (h *TypedHash) UnmarshalHex(text string) error

UnmarshalHex deserializes a hexadecimal text string into the given type.

func (*TypedHash) UnmarshalText added in v0.2400.0 

func (h *TypedHash) UnmarshalText(text []byte) error

UnmarshalText decodes a text marshaled Hash.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.