storage

package
v0.0.0-...-f1c747c Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 11, 2017 License: Apache-2.0 Imports: 11 Imported by: 0

README

Storage layer

The interface, various concrete implementations, and any associated components live here. Currently, there is only one storage implementation:

  • MySQL/MariaDB, which lives in mysql/.

The design is such that both LogStorage and MapStorage models reuse a shared TreeStorage model which can store arbitrary nodes in a tree.

Anyone poking around in here should be aware that there are some subtle wrinkles introduced by the fact that Log trees grow upwards (i.e. the Log considers nodes at level 0 to be the leaves), and in contrast the Map considers the leaves to be at level 255 (and the root at 0), this is based on the HStar2 algorithm.

TreeStorage

Nodes

Nodes within the tree are each given a unique NodeID (see storage/types.go), this ID can be thought of as the binary path (0 for left, 1 for right) from the root of the tree down to the node in question (or, equivalently, as the binary representation of the node's horizonal index into the tree layer at the depth of the node.)

TODO(al): pictures!

Subtrees

The TreeStorage model does not, in fact, store all the internal nodes of the tree; it divides the tree into subtrees of depth 8 and stores the data for each subtree as a single unit. Within these subtrees, only the (subtree-relative) "leaf" nodes are actually written to disk, the internal structure of the subtrees is re-calculated when the subtree is read from disk.

Doing this compaction saves a considerable amout of on-disk space, and at least for the MySQL storage implementation, results in a ~20% speed increase.

History

Updates to the tree storage are performed in a batched fashion (i.e. some unit of update which provides a self-consistent view of the tree - e.g.:

  • n append leaf operations along with internal node updates for the LogStorage, and tagged with their sequence number.
  • n set value operations along with internal node updates for the MapStorage.

These batched updates are termed treeRevisions, and nodes updated within each revision are tagged with a monotonically incrementing sequence number.

In this fashion, the storage model records all historical revisions of the tree.

To perform lookups at a particular treeRevision, the TreeStorage simply requests nodes from disk which are associated with the given NodeID and whose treeRevsions are <= the desired revision.

Currently there's no mechanism to safely garbage collect obsolete nodes so storage grows without bound. This will be addressed at some point in the future.

Updates to the tree

The current treeRevision is defined to be the one referenced by the latest Signed [Map|Log] Head (if there is no SignedHead, then the current treeRevision is -1.)

Updates to the tree are performed in the future (i.e. at currentTreeRevision + 1), and, to allow for reasonable performance, are not required to be atomic (i.e. a tree update may partially fail), however for this to work, higher layers using the storage layer must guarantee that failed tree updates are later re-tried with either precisely the same set of node chages, or a superset there-of, in order to ensure integrity of the tree.

We intend to enforce this contract within the treeStorage layer at some point in the future.

LogStorage

TODO(al): flesh this out

LogStorage builds upon TreeStorage and additionally provides a means of storing log leaves, and SignedTreeHeads, and an API for sequencing new leaves into the tree.

MapStorage

TODO(al): flesh this out

MapStorage builds upon TreeStorage and additionally provides a means of storing map values, and SignedMapHeads.

Documentation

Overview

Package storage provides general interfaces to Trillian storage layers.

Index

Constants

View Source 
const (
	DuplicateLeaf = iota
)

Integer types to distinguish storage errors that might need to be mapped at a higher level.

Variables

This section is empty.

Functions

func NewTreeID 

func NewTreeID() (int64, error)

NewTreeID generates a random, positive, non-zero tree ID.

func ValidateTreeForCreation 

func ValidateTreeForCreation(tree *trillian.Tree) error

ValidateTreeForCreation returns nil if tree is valid for insertion, error otherwise. See the documentation on trillian.Tree for reference on which values are valid.

func ValidateTreeForUpdate 

func ValidateTreeForUpdate(storedTree, newTree *trillian.Tree) error

ValidateTreeForUpdate returns nil if newTree is valid for update, error otherwise. The newTree is compared to the storedTree to determine if readonly fields have been changed. It's assumed that storage-generated fields, such as update_time, have not yet changed when this method is called. See the documentation on trillian.Tree for reference on which fields may be changed and what is considered valid for each of them.

Types

type AdminReader 

type AdminReader interface {
	// GetTree returns the tree corresponding to treeID or an error.
	GetTree(ctx context.Context, treeID int64) (*trillian.Tree, error)

	// ListTreeIDs returns the IDs of all trees in storage.
	// Note that there's no authorization restriction on the IDs returned,
	// so it should be used with caution in production code.
	ListTreeIDs(ctx context.Context) ([]int64, error)

	// ListTrees returns all trees in storage.
	// Note that there's no authorization restriction on the trees returned,
	// so it should be used with caution in production code.
	ListTrees(ctx context.Context) ([]*trillian.Tree, error)
}

AdminReader provides a read-only interface for tree data.

type AdminStorage 

type AdminStorage interface {
	// Snapshot starts a read-only transaction.
	// A transaction must be explicitly committed before the data read by it
	// is considered consistent.
	Snapshot(ctx context.Context) (ReadOnlyAdminTX, error)

	// Begin starts a read/write transaction.
	// A transaction must be explicitly committed before the data read by it
	// is considered consistent.
	Begin(ctx context.Context) (AdminTX, error)
}

AdminStorage represents the persistent storage of tree data.

type AdminTX 

type AdminTX interface {
	ReadOnlyAdminTX
	AdminWriter
}

AdminTX is a transaction capable of read and write operations in the AdminStorage.

type AdminWriter 

type AdminWriter interface {
	// CreateTree inserts the specified tree in storage, returning a tree
	// with all storage-generated fields set.
	// Note that treeID and timestamps will be automatically generated by
	// the storage layer, thus may be ignored by the implementation.
	// Remaining fields must be set to valid values.
	// Returns an error if the tree is invalid or creation fails.
	CreateTree(ctx context.Context, tree *trillian.Tree) (*trillian.Tree, error)

	// UpdateTree updates the specified tree in storage, returning a tree
	// with all storage-generated fields set.
	// updateFunc is called to perform the desired tree modifications. Refer
	// to trillian.Tree for details on which fields are mutable and what is
	// considered valid.
	// Returns an error if the tree is invalid or the update cannot be
	// performed.
	UpdateTree(ctx context.Context, treeID int64, updateFunc func(*trillian.Tree)) (*trillian.Tree, error)
}

AdminWriter provides a write-only interface for tree data.

type DatabaseChecker 

type DatabaseChecker interface {
	// CheckDatabaseAccessible returns nil if the database is accessible, error otherwise.
	CheckDatabaseAccessible(context.Context) error
}

DatabaseChecker performs connectivity checks on the database.

type Error 

type Error struct {
	ErrType int
	Detail  string
	Cause   error
}

Error is a typed error that the storage layer can return to give callers information about the error to decide how to handle it.

func (Error) Error 

func (s Error) Error() string

Error formats the internal details of an Error including the original cause.

type Getter 

type Getter interface {
	// Get retrieves the values associates with the keyHashes, if any, at the
	// specified revision.
	// Setting revision to -1 will fetch the latest revision.
	// The returned array of MapLeaves will only contain entries for which values
	// exist.  i.e. requesting a set of unknown keys would result in a
	// zero-length array being returned.
	Get(ctx context.Context, revision int64, keyHashes [][]byte) ([]trillian.MapLeaf, error)
}

Getter allows access to the values stored in the map.

type LeafDequeuer 

type LeafDequeuer interface {
	// DequeueLeaves will return between [0, limit] leaves from the queue.
	// Leaves which have been dequeued within a Rolled-back Tx will become available for dequeing again.
	// Leaves queued more recently than the cutoff time will not be returned. This allows for
	// guard intervals to be configured.
	DequeueLeaves(ctx context.Context, limit int, cutoffTime time.Time) ([]*trillian.LogLeaf, error)
	UpdateSequencedLeaves(ctx context.Context, leaves []*trillian.LogLeaf) error
}

LeafDequeuer provides an interface for reading previously queued leaves for integration into the tree.

type LeafQueuer 

type LeafQueuer interface {
	// QueueLeaves enqueues leaves for later integration into the tree.
	// If error is nil, the returned slice of leaves will be the same size as the
	// input, and each entry will hold:
	//  - the existing leaf entry if a duplicate has been submitted
	//  - nil otherwise.
	// Duplicates are only reported if the underlying tree does not permit duplicates, and are
	// considered duplicate if their leaf.LeafIdentityHash matches.
	QueueLeaves(ctx context.Context, leaves []*trillian.LogLeaf, queueTimestamp time.Time) ([]*trillian.LogLeaf, error)
}

LeafQueuer provides a write-only interface for the queueing (but not necessarily integration) of leaves.

type LeafReader 

type LeafReader interface {
	// GetSequencedLeafCount returns the total number of leaves that have been integrated into the
	// tree via sequencing.
	GetSequencedLeafCount(ctx context.Context) (int64, error)
	// GetLeavesByIndex returns leaf metadata and data for a set of specified sequenced leaf indexes.
	GetLeavesByIndex(ctx context.Context, leaves []int64) ([]*trillian.LogLeaf, error)
	// GetLeavesByHash looks up sequenced leaf metadata and data by their Merkle leaf hash. If the
	// tree permits duplicate leaves callers must be prepared to handle multiple results with the
	// same hash but different sequence numbers. If orderBySequence is true then the returned data
	// will be in ascending sequence number order.
	GetLeavesByHash(ctx context.Context, leafHashes [][]byte, orderBySequence bool) ([]*trillian.LogLeaf, error)
}

LeafReader provides a read only interface to stored tree leaves

type LogMetadata 

type LogMetadata interface {
	// GetActiveLogs returns a list of the IDs of all the logs that are configured in storage
	GetActiveLogIDs(ctx context.Context) ([]int64, error)
	// GetActiveLogIDsWithPendingWork returns a list of IDs of logs that have
	// pending queued leaves that need to be integrated into the log.
	GetActiveLogIDsWithPendingWork(ctx context.Context) ([]int64, error)
}

LogMetadata provides access to information about the logs in storage

type LogRootReader 

type LogRootReader interface {
	// LatestSignedLogRoot returns the most recent SignedLogRoot, if any.
	LatestSignedLogRoot(ctx context.Context) (trillian.SignedLogRoot, error)
}

LogRootReader provides an interface for reading SignedLogRoots.

type LogRootWriter 

type LogRootWriter interface {
	// StoreSignedLogRoot stores a freshly created SignedLogRoot.
	StoreSignedLogRoot(ctx context.Context, root trillian.SignedLogRoot) error
}

LogRootWriter provides an interface for storing new SignedLogRoots.

type LogStorage 

type LogStorage interface {
	ReadOnlyLogStorage

	// BeginForTree starts a transaction for the specified treeID.
	// Either Commit or Rollback must be called when the caller is finished with
	// the returned object, and values read through it should only be propagated
	// if Commit returns without error.
	BeginForTree(ctx context.Context, treeID int64) (LogTreeTX, error)
}

LogStorage should be implemented by concrete storage mechanisms which want to support Logs.

type LogTreeTX 

type LogTreeTX interface {
	TreeTX
	LogRootReader
	LogRootWriter
	LeafReader
	LeafQueuer
	LeafDequeuer
	LogMetadata
}

LogTreeTX is the transactional interface for reading/updating a Log. It extends the basic TreeTX interface with Log specific methods. After a call to Commit or Rollback implementations must be in a clean state and have released any resources owned by the LogTX. A LogTreeTX can only modify the tree specified in its creation.

type MapRootReader 

type MapRootReader interface {
	// GetSignedMapRoot returns the SignedMapRoot associated with the
	// specified revision.
	GetSignedMapRoot(ctx context.Context, revision int64) (trillian.SignedMapRoot, error)
	// LatestSignedMapRoot returns the most recently created SignedMapRoot.
	LatestSignedMapRoot(ctx context.Context) (trillian.SignedMapRoot, error)
}

MapRootReader provides access to the map roots.

type MapRootWriter 

type MapRootWriter interface {
	// StoreSignedMapRoot stores root.
	StoreSignedMapRoot(ctx context.Context, root trillian.SignedMapRoot) error
}

MapRootWriter allows the storage of new SignedMapRoots

type MapStorage 

type MapStorage interface {
	ReadOnlyMapStorage
	// BeginForTree starts a new Map transaction.
	// Either Commit or Rollback must be called when the caller is finished with
	// the returned object, and values read through it should only be propagated
	// if Commit returns without error.
	BeginForTree(ctx context.Context, treeID int64) (MapTreeTX, error)
}

MapStorage should be implemented by concrete storage mechanisms which want to support Maps

type MapTreeTX 

type MapTreeTX interface {
	TreeTX
	MapRootReader
	MapRootWriter
	Getter
	Setter
}

MapTreeTX is the transactional interface for reading/modifying a Map. It extends the basic TreeTX interface with Map specific methods. After a call to Commit or Rollback implementations must be in a clean state and have released any resources owned by the MapTX. A MapTreeTX can only read from the tree specified in its creation.

type MockAdminStorage 

type MockAdminStorage struct {
	// contains filtered or unexported fields
}

Mock of AdminStorage interface

func NewMockAdminStorage 

func NewMockAdminStorage(ctrl *gomock.Controller) *MockAdminStorage

func (*MockAdminStorage) Begin 

func (_m *MockAdminStorage) Begin(_param0 context.Context) (AdminTX, error)

func (*MockAdminStorage) EXPECT 

func (_m *MockAdminStorage) EXPECT() *_MockAdminStorageRecorder

func (*MockAdminStorage) Snapshot 

func (_m *MockAdminStorage) Snapshot(_param0 context.Context) (ReadOnlyAdminTX, error)

type MockAdminTX 

type MockAdminTX struct {
	// contains filtered or unexported fields
}

Mock of AdminTX interface

func NewMockAdminTX 

func NewMockAdminTX(ctrl *gomock.Controller) *MockAdminTX

func (*MockAdminTX) Close 

func (_m *MockAdminTX) Close() error

func (*MockAdminTX) Commit 

func (_m *MockAdminTX) Commit() error

func (*MockAdminTX) CreateTree 

func (_m *MockAdminTX) CreateTree(_param0 context.Context, _param1 *trillian.Tree) (*trillian.Tree, error)

func (*MockAdminTX) EXPECT 

func (_m *MockAdminTX) EXPECT() *_MockAdminTXRecorder

func (*MockAdminTX) GetTree 

func (_m *MockAdminTX) GetTree(_param0 context.Context, _param1 int64) (*trillian.Tree, error)

func (*MockAdminTX) IsClosed 

func (_m *MockAdminTX) IsClosed() bool

func (*MockAdminTX) ListTreeIDs 

func (_m *MockAdminTX) ListTreeIDs(_param0 context.Context) ([]int64, error)

func (*MockAdminTX) ListTrees 

func (_m *MockAdminTX) ListTrees(_param0 context.Context) ([]*trillian.Tree, error)

func (*MockAdminTX) Rollback 

func (_m *MockAdminTX) Rollback() error

func (*MockAdminTX) UpdateTree 

func (_m *MockAdminTX) UpdateTree(_param0 context.Context, _param1 int64, _param2 func(*trillian.Tree)) (*trillian.Tree, error)

type MockLogStorage 

type MockLogStorage struct {
	// contains filtered or unexported fields
}

Mock of LogStorage interface

func NewMockLogStorage 

func NewMockLogStorage(ctrl *gomock.Controller) *MockLogStorage

func (*MockLogStorage) BeginForTree 

func (_m *MockLogStorage) BeginForTree(_param0 context.Context, _param1 int64) (LogTreeTX, error)

func (*MockLogStorage) CheckDatabaseAccessible 

func (_m *MockLogStorage) CheckDatabaseAccessible(_param0 context.Context) error

func (*MockLogStorage) EXPECT 

func (_m *MockLogStorage) EXPECT() *_MockLogStorageRecorder

func (*MockLogStorage) Snapshot 

func (_m *MockLogStorage) Snapshot(_param0 context.Context) (ReadOnlyLogTX, error)

func (*MockLogStorage) SnapshotForTree 

func (_m *MockLogStorage) SnapshotForTree(_param0 context.Context, _param1 int64) (ReadOnlyLogTreeTX, error)

type MockLogTreeTX 

type MockLogTreeTX struct {
	// contains filtered or unexported fields
}

Mock of LogTreeTX interface

func NewMockLogTreeTX 

func NewMockLogTreeTX(ctrl *gomock.Controller) *MockLogTreeTX

func (*MockLogTreeTX) Close 

func (_m *MockLogTreeTX) Close() error

func (*MockLogTreeTX) Commit 

func (_m *MockLogTreeTX) Commit() error

func (*MockLogTreeTX) DequeueLeaves 

func (_m *MockLogTreeTX) DequeueLeaves(_param0 context.Context, _param1 int, _param2 time.Time) ([]*trillian.LogLeaf, error)

func (*MockLogTreeTX) EXPECT 

func (_m *MockLogTreeTX) EXPECT() *_MockLogTreeTXRecorder

func (*MockLogTreeTX) GetActiveLogIDs 

func (_m *MockLogTreeTX) GetActiveLogIDs(_param0 context.Context) ([]int64, error)

func (*MockLogTreeTX) GetActiveLogIDsWithPendingWork 

func (_m *MockLogTreeTX) GetActiveLogIDsWithPendingWork(_param0 context.Context) ([]int64, error)

func (*MockLogTreeTX) GetLeavesByHash 

func (_m *MockLogTreeTX) GetLeavesByHash(_param0 context.Context, _param1 [][]byte, _param2 bool) ([]*trillian.LogLeaf, error)

func (*MockLogTreeTX) GetLeavesByIndex 

func (_m *MockLogTreeTX) GetLeavesByIndex(_param0 context.Context, _param1 []int64) ([]*trillian.LogLeaf, error)

func (*MockLogTreeTX) GetMerkleNodes 

func (_m *MockLogTreeTX) GetMerkleNodes(_param0 context.Context, _param1 int64, _param2 []NodeID) ([]Node, error)

func (*MockLogTreeTX) GetSequencedLeafCount 

func (_m *MockLogTreeTX) GetSequencedLeafCount(_param0 context.Context) (int64, error)

func (*MockLogTreeTX) IsOpen 

func (_m *MockLogTreeTX) IsOpen() bool

func (*MockLogTreeTX) LatestSignedLogRoot 

func (_m *MockLogTreeTX) LatestSignedLogRoot(_param0 context.Context) (trillian.SignedLogRoot, error)

func (*MockLogTreeTX) QueueLeaves 

func (_m *MockLogTreeTX) QueueLeaves(_param0 context.Context, _param1 []*trillian.LogLeaf, _param2 time.Time) ([]*trillian.LogLeaf, error)

func (*MockLogTreeTX) ReadRevision 

func (_m *MockLogTreeTX) ReadRevision() int64

func (*MockLogTreeTX) Rollback 

func (_m *MockLogTreeTX) Rollback() error

func (*MockLogTreeTX) SetMerkleNodes 

func (_m *MockLogTreeTX) SetMerkleNodes(_param0 context.Context, _param1 []Node) error

func (*MockLogTreeTX) StoreSignedLogRoot 

func (_m *MockLogTreeTX) StoreSignedLogRoot(_param0 context.Context, _param1 trillian.SignedLogRoot) error

func (*MockLogTreeTX) UpdateSequencedLeaves 

func (_m *MockLogTreeTX) UpdateSequencedLeaves(_param0 context.Context, _param1 []*trillian.LogLeaf) error

func (*MockLogTreeTX) WriteRevision 

func (_m *MockLogTreeTX) WriteRevision() int64

type MockMapStorage 

type MockMapStorage struct {
	// contains filtered or unexported fields
}

Mock of MapStorage interface

func NewMockMapStorage 

func NewMockMapStorage(ctrl *gomock.Controller) *MockMapStorage

func (*MockMapStorage) BeginForTree 

func (_m *MockMapStorage) BeginForTree(_param0 context.Context, _param1 int64) (MapTreeTX, error)

func (*MockMapStorage) CheckDatabaseAccessible 

func (_m *MockMapStorage) CheckDatabaseAccessible(_param0 context.Context) error

func (*MockMapStorage) EXPECT 

func (_m *MockMapStorage) EXPECT() *_MockMapStorageRecorder

func (*MockMapStorage) Snapshot 

func (_m *MockMapStorage) Snapshot(_param0 context.Context) (ReadOnlyMapTX, error)

func (*MockMapStorage) SnapshotForTree 

func (_m *MockMapStorage) SnapshotForTree(_param0 context.Context, _param1 int64) (ReadOnlyMapTreeTX, error)

type MockMapTreeTX 

type MockMapTreeTX struct {
	// contains filtered or unexported fields
}

Mock of MapTreeTX interface

func NewMockMapTreeTX 

func NewMockMapTreeTX(ctrl *gomock.Controller) *MockMapTreeTX

func (*MockMapTreeTX) Close 

func (_m *MockMapTreeTX) Close() error

func (*MockMapTreeTX) Commit 

func (_m *MockMapTreeTX) Commit() error

func (*MockMapTreeTX) EXPECT 

func (_m *MockMapTreeTX) EXPECT() *_MockMapTreeTXRecorder

func (*MockMapTreeTX) Get 

func (_m *MockMapTreeTX) Get(_param0 context.Context, _param1 int64, _param2 [][]byte) ([]trillian.MapLeaf, error)

func (*MockMapTreeTX) GetMerkleNodes 

func (_m *MockMapTreeTX) GetMerkleNodes(_param0 context.Context, _param1 int64, _param2 []NodeID) ([]Node, error)

func (*MockMapTreeTX) GetSignedMapRoot 

func (_m *MockMapTreeTX) GetSignedMapRoot(_param0 context.Context, _param1 int64) (trillian.SignedMapRoot, error)

func (*MockMapTreeTX) IsOpen 

func (_m *MockMapTreeTX) IsOpen() bool

func (*MockMapTreeTX) LatestSignedMapRoot 

func (_m *MockMapTreeTX) LatestSignedMapRoot(_param0 context.Context) (trillian.SignedMapRoot, error)

func (*MockMapTreeTX) ReadRevision 

func (_m *MockMapTreeTX) ReadRevision() int64

func (*MockMapTreeTX) Rollback 

func (_m *MockMapTreeTX) Rollback() error

func (*MockMapTreeTX) Set 

func (_m *MockMapTreeTX) Set(_param0 context.Context, _param1 []byte, _param2 trillian.MapLeaf) error

func (*MockMapTreeTX) SetMerkleNodes 

func (_m *MockMapTreeTX) SetMerkleNodes(_param0 context.Context, _param1 []Node) error

func (*MockMapTreeTX) StoreSignedMapRoot 

func (_m *MockMapTreeTX) StoreSignedMapRoot(_param0 context.Context, _param1 trillian.SignedMapRoot) error

func (*MockMapTreeTX) WriteRevision 

func (_m *MockMapTreeTX) WriteRevision() int64

type MockReadOnlyAdminTX 

type MockReadOnlyAdminTX struct {
	// contains filtered or unexported fields
}

Mock of ReadOnlyAdminTX interface

func NewMockReadOnlyAdminTX 

func NewMockReadOnlyAdminTX(ctrl *gomock.Controller) *MockReadOnlyAdminTX

func (*MockReadOnlyAdminTX) Close 

func (_m *MockReadOnlyAdminTX) Close() error

func (*MockReadOnlyAdminTX) Commit 

func (_m *MockReadOnlyAdminTX) Commit() error

func (*MockReadOnlyAdminTX) EXPECT 

func (_m *MockReadOnlyAdminTX) EXPECT() *_MockReadOnlyAdminTXRecorder

func (*MockReadOnlyAdminTX) GetTree 

func (_m *MockReadOnlyAdminTX) GetTree(_param0 context.Context, _param1 int64) (*trillian.Tree, error)

func (*MockReadOnlyAdminTX) IsClosed 

func (_m *MockReadOnlyAdminTX) IsClosed() bool

func (*MockReadOnlyAdminTX) ListTreeIDs 

func (_m *MockReadOnlyAdminTX) ListTreeIDs(_param0 context.Context) ([]int64, error)

func (*MockReadOnlyAdminTX) ListTrees 

func (_m *MockReadOnlyAdminTX) ListTrees(_param0 context.Context) ([]*trillian.Tree, error)

func (*MockReadOnlyAdminTX) Rollback 

func (_m *MockReadOnlyAdminTX) Rollback() error

type MockReadOnlyLogTX 

type MockReadOnlyLogTX struct {
	// contains filtered or unexported fields
}

Mock of ReadOnlyLogTX interface

func NewMockReadOnlyLogTX 

func NewMockReadOnlyLogTX(ctrl *gomock.Controller) *MockReadOnlyLogTX

func (*MockReadOnlyLogTX) Close 

func (_m *MockReadOnlyLogTX) Close() error

func (*MockReadOnlyLogTX) Commit 

func (_m *MockReadOnlyLogTX) Commit() error

func (*MockReadOnlyLogTX) EXPECT 

func (_m *MockReadOnlyLogTX) EXPECT() *_MockReadOnlyLogTXRecorder

func (*MockReadOnlyLogTX) GetActiveLogIDs 

func (_m *MockReadOnlyLogTX) GetActiveLogIDs(_param0 context.Context) ([]int64, error)

func (*MockReadOnlyLogTX) GetActiveLogIDsWithPendingWork 

func (_m *MockReadOnlyLogTX) GetActiveLogIDsWithPendingWork(_param0 context.Context) ([]int64, error)

func (*MockReadOnlyLogTX) Rollback 

func (_m *MockReadOnlyLogTX) Rollback() error

type MockReadOnlyLogTreeTX 

type MockReadOnlyLogTreeTX struct {
	// contains filtered or unexported fields
}

Mock of ReadOnlyLogTreeTX interface

func NewMockReadOnlyLogTreeTX 

func NewMockReadOnlyLogTreeTX(ctrl *gomock.Controller) *MockReadOnlyLogTreeTX

func (*MockReadOnlyLogTreeTX) Close 

func (_m *MockReadOnlyLogTreeTX) Close() error

func (*MockReadOnlyLogTreeTX) Commit 

func (_m *MockReadOnlyLogTreeTX) Commit() error

func (*MockReadOnlyLogTreeTX) EXPECT 

func (_m *MockReadOnlyLogTreeTX) EXPECT() *_MockReadOnlyLogTreeTXRecorder

func (*MockReadOnlyLogTreeTX) GetLeavesByHash 

func (_m *MockReadOnlyLogTreeTX) GetLeavesByHash(_param0 context.Context, _param1 [][]byte, _param2 bool) ([]*trillian.LogLeaf, error)

func (*MockReadOnlyLogTreeTX) GetLeavesByIndex 

func (_m *MockReadOnlyLogTreeTX) GetLeavesByIndex(_param0 context.Context, _param1 []int64) ([]*trillian.LogLeaf, error)

func (*MockReadOnlyLogTreeTX) GetMerkleNodes 

func (_m *MockReadOnlyLogTreeTX) GetMerkleNodes(_param0 context.Context, _param1 int64, _param2 []NodeID) ([]Node, error)

func (*MockReadOnlyLogTreeTX) GetSequencedLeafCount 

func (_m *MockReadOnlyLogTreeTX) GetSequencedLeafCount(_param0 context.Context) (int64, error)

func (*MockReadOnlyLogTreeTX) IsOpen 

func (_m *MockReadOnlyLogTreeTX) IsOpen() bool

func (*MockReadOnlyLogTreeTX) LatestSignedLogRoot 

func (_m *MockReadOnlyLogTreeTX) LatestSignedLogRoot(_param0 context.Context) (trillian.SignedLogRoot, error)

func (*MockReadOnlyLogTreeTX) ReadRevision 

func (_m *MockReadOnlyLogTreeTX) ReadRevision() int64

func (*MockReadOnlyLogTreeTX) Rollback 

func (_m *MockReadOnlyLogTreeTX) Rollback() error

type MockReadOnlyMapTreeTX 

type MockReadOnlyMapTreeTX struct {
	// contains filtered or unexported fields
}

Mock of ReadOnlyMapTreeTX interface

func NewMockReadOnlyMapTreeTX 

func NewMockReadOnlyMapTreeTX(ctrl *gomock.Controller) *MockReadOnlyMapTreeTX

func (*MockReadOnlyMapTreeTX) Close 

func (_m *MockReadOnlyMapTreeTX) Close() error

func (*MockReadOnlyMapTreeTX) Commit 

func (_m *MockReadOnlyMapTreeTX) Commit() error

func (*MockReadOnlyMapTreeTX) EXPECT 

func (_m *MockReadOnlyMapTreeTX) EXPECT() *_MockReadOnlyMapTreeTXRecorder

func (*MockReadOnlyMapTreeTX) Get 

func (_m *MockReadOnlyMapTreeTX) Get(_param0 context.Context, _param1 int64, _param2 [][]byte) ([]trillian.MapLeaf, error)

func (*MockReadOnlyMapTreeTX) GetMerkleNodes 

func (_m *MockReadOnlyMapTreeTX) GetMerkleNodes(_param0 context.Context, _param1 int64, _param2 []NodeID) ([]Node, error)

func (*MockReadOnlyMapTreeTX) GetSignedMapRoot 

func (_m *MockReadOnlyMapTreeTX) GetSignedMapRoot(_param0 context.Context, _param1 int64) (trillian.SignedMapRoot, error)

func (*MockReadOnlyMapTreeTX) IsOpen 

func (_m *MockReadOnlyMapTreeTX) IsOpen() bool

func (*MockReadOnlyMapTreeTX) LatestSignedMapRoot 

func (_m *MockReadOnlyMapTreeTX) LatestSignedMapRoot(_param0 context.Context) (trillian.SignedMapRoot, error)

func (*MockReadOnlyMapTreeTX) ReadRevision 

func (_m *MockReadOnlyMapTreeTX) ReadRevision() int64

func (*MockReadOnlyMapTreeTX) Rollback 

func (_m *MockReadOnlyMapTreeTX) Rollback() error

type Node 

type Node struct {
	NodeID       NodeID
	Hash         []byte
	NodeRevision int64
}

Node represents a single node in a Merkle tree.

type NodeID 

type NodeID struct {
	// path is effectively a BigEndian bit set, with path[0] being the MSB
	// (identifying the root child), and successive bits identifying the lower
	// level children down to the leaf.
	Path []byte
	// PrefixLenBits is the number of MSB in Path which are considered part of
	// this NodeID.
	//
	// e.g. if Path contains two bytes, and PrefixLenBits is 9, then the 8 bits
	// in Path[0] are included, along with the lowest bit of Path[1]
	PrefixLenBits int
	PathLenBits   int
}

NodeID uniquely identifies a Node within a versioned MerkleTree.

func NewEmptyNodeID 

func NewEmptyNodeID(maxLenBits int) NodeID

NewEmptyNodeID creates a new zero-length NodeID with sufficient underlying capacity to store a maximum of maxLenBits.

func NewNodeIDForTreeCoords 

func NewNodeIDForTreeCoords(depth int64, index int64, maxPathBits int) (NodeID, error)

NewNodeIDForTreeCoords creates a new NodeID for a Tree node with a specified depth and index. This method is used exclusively by the Log, and, since the Log model grows upwards from the leaves, we modify the provided coords accordingly.

depth is the Merkle tree level: 0 = leaves, and increases upwards towards the root.

index is the horizontal index into the tree at level depth, so the returned NodeID will be zero padded on the right by depth places.

func NewNodeIDFromHash 

func NewNodeIDFromHash(h []byte) NodeID

NewNodeIDFromHash creates a new NodeID for the given Hash.

func NewNodeIDWithPrefix 

func NewNodeIDWithPrefix(prefix uint64, prefixLenBits, nodeIDLenBits, maxLenBits int) NodeID

NewNodeIDWithPrefix creates a new NodeID of nodeIDLen bits with the prefixLen MSBs set to prefix.

func (*NodeID) Bit 

func (n *NodeID) Bit(i int) uint

Bit returns 1 if the ith bit is true, and false otherwise.

func (*NodeID) CoordString 

func (n *NodeID) CoordString() string

CoordString returns a string representation assuming that the NodeID represents a tree coordinate. Using this on a NodeID for a sparse Merkle tree will give incorrect results. Intended for debugging purposes, the format could change.

func (*NodeID) Equivalent 

func (n *NodeID) Equivalent(other NodeID) bool

Equivalent return true iff the other represents the same path prefix as this NodeID.

func (*NodeID) SetBit 

func (n *NodeID) SetBit(i int, b uint)

SetBit sets the ith bit to true if b is non-zero, and false otherwise.

func (*NodeID) Siblings 

func (n *NodeID) Siblings() []NodeID

Siblings returns the siblings of the given node.

func (*NodeID) String 

func (n *NodeID) String() string

String returns a string representation of the binary value of the NodeID. The left-most bit is the MSB (i.e. nearer the root of the tree).

type NodeReader 

type NodeReader interface {
	// GetMerkleNodes looks up the set of nodes identified by ids, at treeRevision, and returns them.
	GetMerkleNodes(ctx context.Context, treeRevision int64, ids []NodeID) ([]Node, error)
}

NodeReader provides a read-only interface into the stored tree nodes.

type NodeWriter 

type NodeWriter interface {
	// SetMerkleNodes stores the provided nodes, at the transaction's writeRevision.
	SetMerkleNodes(ctx context.Context, nodes []Node) error
}

NodeWriter provides a write interface into the stored tree nodes.

type PopulateSubtreeFunc 

type PopulateSubtreeFunc func(*storagepb.SubtreeProto) error

PopulateSubtreeFunc is a function which knows how to re-populate a subtree from just its leaf nodes.

type PrepareSubtreeWriteFunc 

type PrepareSubtreeWriteFunc func(*storagepb.SubtreeProto) error

PrepareSubtreeWriteFunc is a function that carries out any required tree type specific manipulation of a subtree before it's written to storage

type ReadOnlyAdminTX 

type ReadOnlyAdminTX interface {
	AdminReader

	// Commit applies the operations performed to the underlying storage, or
	// returns an error.
	// A commit must be performed before any reads from storage are
	// considered consistent.
	Commit() error

	// Rollback aborts any performed operations, or returns an error.
	// See Close() for a way to automatically manage transactions.
	Rollback() error

	// IsClosed returns true if the transaction is closed.
	// A transaction is closed when either Commit() or Rollback() are
	// called.
	IsClosed() bool

	// Close rolls back the transaction if it's not yet closed.
	// It's advisable to call "defer tx.Close()" after the creation of
	// transaction to ensure that it's always rolled back if not explicitly
	// committed.
	Close() error
}

ReadOnlyAdminTX is a transaction capable only of read operations in the AdminStorage.

type ReadOnlyLogStorage 

type ReadOnlyLogStorage interface {
	DatabaseChecker

	// Snapshot starts a read-only transaction not tied to any particular tree.
	Snapshot(ctx context.Context) (ReadOnlyLogTX, error)

	// SnapshotForTree starts a read-only transaction for the specified treeID.
	// Commit must be called when the caller is finished with the returned object,
	// and values read through it should only be propagated if Commit returns
	// without error.
	SnapshotForTree(ctx context.Context, treeID int64) (ReadOnlyLogTreeTX, error)
}

ReadOnlyLogStorage represents a narrowed read-only view into a LogStorage.

type ReadOnlyLogTX 

type ReadOnlyLogTX interface {
	LogMetadata

	// Commit ensures the data read by the TX is consistent in the database. Only after Commit the
	// data read should be regarded as valid.
	Commit() error

	// Rollback discards the read-only TX.
	Rollback() error

	// Close attempts to Rollback the TX if it's open, it's a noop otherwise.
	Close() error
}

ReadOnlyLogTX provides a read-only view into log data. A ReadOnlyLogTX, unlike ReadOnlyLogTreeTX, is not tied to a particular tree.

type ReadOnlyLogTreeTX 

type ReadOnlyLogTreeTX interface {
	ReadOnlyTreeTX
	LeafReader
	LogRootReader
}

ReadOnlyLogTreeTX provides a read-only view into the Log data. A ReadOnlyLogTreeTX can only read from the tree specified in its creation.

type ReadOnlyMapStorage 

type ReadOnlyMapStorage interface {
	DatabaseChecker

	// Snapshot starts a read-only transaction not tied to any particular tree.
	Snapshot(ctx context.Context) (ReadOnlyMapTX, error)

	// SnapshotForTree starts a new read-only transaction.
	// Commit must be called when the caller is finished with the returned object,
	// and values read through it should only be propagated if Commit returns
	// without error.
	SnapshotForTree(ctx context.Context, treeID int64) (ReadOnlyMapTreeTX, error)
}

ReadOnlyMapStorage provides a narrow read-only view into a MapStorage.

type ReadOnlyMapTX 

type ReadOnlyMapTX interface {
	// Commit ensures the data read by the TX is consistent in the database. Only after Commit the
	// data read should be regarded as valid.
	Commit() error

	// Rollback discards the read-only TX.
	Rollback() error

	// Close attempts to Rollback the TX if it's open, it's a noop otherwise.
	Close() error
}

ReadOnlyMapTX provides a read-only view into log data. A ReadOnlyMapTX, unlike ReadOnlyMapTreeTX, is not tied to a particular tree.

type ReadOnlyMapTreeTX 

type ReadOnlyMapTreeTX interface {
	ReadOnlyTreeTX
	MapRootReader
	Getter
}

ReadOnlyMapTreeTX provides a read-only view into the Map data. A ReadOnlyMapTreeTX can only read from the tree specified in its creation.

type ReadOnlyTreeTX 

type ReadOnlyTreeTX interface {
	NodeReader

	// ReadRevision returns the tree revision that was current at the time this
	// transaction was started.
	ReadRevision() int64

	// Commit attempts to commit any reads performed under this transaction.
	Commit() error

	// Rollback aborts this transaction.
	Rollback() error

	// Close attempts to Rollback the TX if it's open, it's a noop otherwise.
	Close() error

	// Open indicates if this transaction is open. An open transaction is one for which
	// Commit() or Rollback() has never been called. Implementations must do all clean up
	// in these methods so transactions are assumed closed regardless of the reported success.
	IsOpen() bool
}

ReadOnlyTreeTX represents a read-only transaction on a TreeStorage. A ReadOnlyTreeTX can only modify the tree specified in its creation.

type Setter 

type Setter interface {
	// Set sets key to leaf
	Set(ctx context.Context, keyHash []byte, value trillian.MapLeaf) error
}

Setter allows the setting of key->value pairs on the map.

type TreeTX 

type TreeTX interface {
	ReadOnlyTreeTX
	NodeWriter

	// WriteRevision returns the tree revision that any writes through this TreeTX will be stored at.
	WriteRevision() int64
}

TreeTX represents an in-process tree-modifying transaction. The transaction must end with a call to Commit or Rollback. After a call to Commit or Rollback, all operations on the transaction will fail. After a call to Commit or Rollback implementations must be in a clean state and have released any resources owned by the TreeTX. A TreeTX can only modify the tree specified in its creation.

Source Files

View all Source files

Directories

Path Synopsis
Package cache provides subtree caching functionality.
 Package cache provides subtree caching functionality.
Package memory provides a simple in-process implementation of the tree- and log-storage interfaces.
 Package memory provides a simple in-process implementation of the tree- and log-storage interfaces.
Package mysql provides a MySQL-based storage layer implementation.
 Package mysql provides a MySQL-based storage layer implementation.
Package storagepb is a generated protocol buffer package.
 Package storagepb is a generated protocol buffer package.
Package testonly holds test-specific code for Trillian storage layers.
 Package testonly holds test-specific code for Trillian storage layers.
tools
fetch_leaves
The fetch_leaves program retrieves leaves from a tree.
 The fetch_leaves program retrieves leaves from a tree.
log_client
The log_client binary retrieves leaves from a log.
 The log_client binary retrieves leaves from a log.
queue_leaves
The queue_leaves binary queues a number of leaves for a log from a given start point with predictable hashes.
 The queue_leaves binary queues a number of leaves for a log from a given start point with predictable hashes.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.