storage

package
v1.7.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 25, 2024 License: Apache-2.0 Imports: 17 Imported by: 98

README

Storage layer

Package Layout

The interface, various concrete implementations, and any associated components live here. Interfaces and types are defined at the top level package.

Currently, there are two usable storage implementation for logs:

The MySQL / MariaDB implementation includes support for Maps. This has not yet been implemented by Cloud Spanner. There may be other storage implementations available from third parties.

These implementations are in alpha mode and are not yet ready to be used by real applications:

These implementations are for test purposes only and should not be used by real applications:

  • In-memory Storage, in the memory package.

Build tags

By default all of the storage and quota implementations are compiled in to the log server and signer binaries. These binaries can be slimmed down significantly by specifying one or more of the following build tags:

  • cloudspanner
  • crdb
  • mysql
  • postgresql
Adding a new storage implementation

To add a new storage and/or quota implementation requires:

  • each of the go:build directives in the existing files in the cmd/internal/provider directory to be made aware of the build tag for the new implementation.
  • a new file to be created for the new implementation in that same directory, whose contents follow the pattern established by the existing files.
Examples

Include all storage and quota implementations (default):

> cd cmd/trillian_log_server && go build && ls -sh trillian_log_server
62M trillian_log_server*

Include just one storage and associated quota implementation:

> cd cmd/trillian_log_server && go build -tags=crdb && ls -sh trillian_log_server
37M trillian_log_server*

Include multiple storage and associated quota implementations:

> cd cmd/trillian_log_server && go build -tags=mysql,postgresql && ls -sh trillian_log_server
40M trillian_log_server*

Notes and Caveats

The design is such that both LogStorage and MapStorage models reuse a shared TreeStorage model which can store arbitrary nodes in a tree.

Anyone poking around in here should be aware that there are some subtle wrinkles introduced by the fact that Log trees grow upwards (i.e. the Log considers nodes at level 0 to be the leaves), and in contrast the Map considers the leaves to be at level 255 (and the root at 0), this is based on the HStar2 algorithm.

TreeStorage

Nodes

Nodes within the tree are each given a unique NodeID (see storage/types.go), this ID can be thought of as the binary path (0 for left, 1 for right) from the root of the tree down to the node in question (or, equivalently, as the binary representation of the node's horizonal index into the tree layer at the depth of the node.)

TODO(al): pictures!

Subtrees

The TreeStorage model does not, in fact, store all the internal nodes of the tree; it divides the tree into subtrees of depth 8 and stores the data for each subtree as a single unit. Within these subtrees, only the (subtree-relative) "leaf" nodes are actually written to disk, the internal structure of the subtrees is re-calculated when the subtree is read from disk.

Doing this compaction saves a considerable amout of on-disk space, and at least for the MySQL storage implementation, results in a ~20% speed increase.

History

Updates to the tree storage are performed in a batched fashion (i.e. some unit of update which provides a self-consistent view of the tree - e.g.:

  • n append leaf operations along with internal node updates for the LogStorage, and tagged with their sequence number.
  • n set value operations along with internal node updates for the MapStorage.

These batched updates are termed treeRevisions, and nodes updated within each revision are tagged with a monotonically incrementing sequence number.

In this fashion, the storage model records all historical revisions of the tree.

To perform lookups at a particular treeRevision, the TreeStorage simply requests nodes from disk which are associated with the given NodeID and whose treeRevsions are <= the desired revision.

Currently there's no mechanism to safely garbage collect obsolete nodes so storage grows without bound. This will be addressed at some point in the future for Map trees. Log trees don't need garbage collection as they're required to preserve a full history.

Updates to the tree

The current treeRevision is defined to be the one referenced by the latest Signed [Map|Log] Head (if there is no SignedHead, then the current treeRevision is -1.)

Updates to the tree are performed in the future (i.e. at currentTreeRevision + 1), and, to allow for reasonable performance, are not required to be atomic (i.e. a tree update may partially fail), however for this to work, higher layers using the storage layer must guarantee that failed tree updates are later re-tried with either precisely the same set of node chages, or a superset there-of, in order to ensure integrity of the tree.

We intend to enforce this contract within the treeStorage layer at some point in the future.

LogStorage

TODO(al): flesh this out

LogStorage builds upon TreeStorage and additionally provides a means of storing log leaves, and SignedTreeHeads, and an API for sequencing new leaves into the tree.

MapStorage

TODO(al): flesh this out

MapStorage builds upon TreeStorage and additionally provides a means of storing map values, and SignedMapHeads.

Documentation

Overview

Package storage provides general interfaces to Trillian storage layers.

Package storage is a generated GoMock package.

Index

Constants

This section is empty.

Variables

View Source 
var ErrTreeNeedsInit = status.Error(codes.FailedPrecondition, "tree needs initialising")

ErrTreeNeedsInit is returned when calling methods on an uninitialised tree.

Functions

func CreateTree added in v1.0.4 

func CreateTree(ctx context.Context, admin AdminStorage, tree *trillian.Tree) (*trillian.Tree, error)

CreateTree creates a tree in storage. It's a convenience wrapper around ReadWriteTransaction and AdminWriter's CreateTree. See ReadWriteTransaction if you need to perform more than one action per transaction.

func GetTree added in v1.0.4 

func GetTree(ctx context.Context, admin AdminStorage, treeID int64) (*trillian.Tree, error)

GetTree reads a tree from storage using a snapshot transaction. It's a convenience wrapper around RunInAdminSnapshot and ReadOnlyAdminTX's GetTree. See RunInAdminSnapshot if you need to perform more than one action per transaction.

func HardDeleteTree added in v1.0.4 

func HardDeleteTree(ctx context.Context, admin AdminStorage, treeID int64) error

HardDeleteTree hard-deletes a tree from storage. It's a convenience wrapper around ReadWriteTransaction and AdminWriter's HardDeleteTree. See ReadWriteTransaction if you need to perform more than one action per transaction.

func ListTrees added in v1.0.4 

func ListTrees(ctx context.Context, admin AdminStorage, includeDeleted bool) ([]*trillian.Tree, error)

ListTrees reads trees from storage using a snapshot transaction. It's a convenience wrapper around RunInAdminSnapshot and ReadOnlyAdminTX's ListTrees. See RunInAdminSnapshot if you need to perform more than one action per transaction.

func NewTreeID 

func NewTreeID() (int64, error)

NewTreeID generates a random, positive, non-zero tree ID.

func Providers added in v1.3.12 

func Providers() []string

Providers returns a slice of all registered storage provider names.

func RegisterProvider added in v1.3.4 

func RegisterProvider(name string, sp NewProviderFunc) error

RegisterProvider registers the given storage Provider.

func RunInAdminSnapshot added in v1.0.4 

func RunInAdminSnapshot(ctx context.Context, admin AdminStorage, fn func(tx ReadOnlyAdminTX) error) error

RunInAdminSnapshot runs fn against a ReadOnlyAdminTX and commits if no error is returned.

func SoftDeleteTree added in v1.0.4 

func SoftDeleteTree(ctx context.Context, admin AdminStorage, treeID int64) (*trillian.Tree, error)

SoftDeleteTree soft-deletes a tree in storage. It's a convenience wrapper around ReadWriteTransaction and AdminWriter's SoftDeleteTree. See ReadWriteTransaction if you need to perform more than one action per transaction.

func UndeleteTree added in v1.0.4 

func UndeleteTree(ctx context.Context, admin AdminStorage, treeID int64) (*trillian.Tree, error)

UndeleteTree undeletes a tree in storage. It's a convenience wrapper around ReadWriteTransaction and AdminWriter's UndeleteTree. See ReadWriteTransaction if you need to perform more than one action per transaction.

func UpdateTree added in v1.0.4 

func UpdateTree(ctx context.Context, admin AdminStorage, treeID int64, fn func(*trillian.Tree)) (*trillian.Tree, error)

UpdateTree updates a tree in storage. It's a convenience wrapper around ReadWriteTransaction and AdminWriter's UpdateTree. See ReadWriteTransaction if you need to perform more than one action per transaction.

func ValidateTreeForCreation 

func ValidateTreeForCreation(ctx context.Context, tree *trillian.Tree) error

ValidateTreeForCreation returns nil if tree is valid for insertion, error otherwise. See the documentation on trillian.Tree for reference on which values are valid.

func ValidateTreeForUpdate 

func ValidateTreeForUpdate(ctx context.Context, storedTree, newTree *trillian.Tree) error

ValidateTreeForUpdate returns nil if newTree is valid for update, error otherwise. The newTree is compared to the storedTree to determine if readonly fields have been changed. It's assumed that storage-generated fields, such as update_time, have not yet changed when this method is called. See the documentation on trillian.Tree for reference on which fields may be changed and what is considered valid for each of them.

Types

type AdminStorage 

type AdminStorage interface {
	// Snapshot starts a read-only transaction.
	// A transaction must be explicitly committed before the data read by it
	// is considered consistent.
	Snapshot(ctx context.Context) (ReadOnlyAdminTX, error)

	// ReadWriteTransaction creates a transaction, and runs f with it.
	// Some storage implementations may retry aborted transactions, so
	// f MUST be idempotent.
	ReadWriteTransaction(ctx context.Context, f AdminTXFunc) error

	// CheckDatabaseAccessible checks whether we are able to connect to / open the
	// underlying storage.
	CheckDatabaseAccessible(ctx context.Context) error
}

AdminStorage represents the persistent storage of tree data.

type AdminTX 

type AdminTX interface {
	ReadOnlyAdminTX
	AdminWriter
}

AdminTX is a transaction capable of read and write operations in the AdminStorage.

type AdminTXFunc added in v1.0.7 

type AdminTXFunc func(context.Context, AdminTX) error

AdminTXFunc is the signature for functions passed to ReadWriteTransaction.

type AdminWriter 

type AdminWriter interface {
	// CreateTree inserts the specified tree in storage, returning a tree
	// with all storage-generated fields set.
	// Note that treeID and timestamps will be automatically generated by
	// the storage layer, thus may be ignored by the implementation.
	// Remaining fields must be set to valid values.
	// Returns an error if the tree is invalid or creation fails.
	CreateTree(ctx context.Context, tree *trillian.Tree) (*trillian.Tree, error)

	// UpdateTree updates the specified tree in storage, returning a tree
	// with all storage-generated fields set.
	// updateFunc is called to perform the desired tree modifications. Refer
	// to trillian.Tree for details on which fields are mutable and what is
	// considered valid.
	// Returns an error if the tree is invalid or the update cannot be
	// performed.
	UpdateTree(ctx context.Context, treeID int64, updateFunc func(*trillian.Tree)) (*trillian.Tree, error)

	// SoftDeleteTree soft deletes the specified tree.
	// The tree must exist and not be already soft deleted, otherwise an error is returned.
	// Soft deletion may be undone via UndeleteTree.
	SoftDeleteTree(ctx context.Context, treeID int64) (*trillian.Tree, error)

	// HardDeleteTree hard deletes (i.e. completely removes from storage) the specified tree and all
	// records related to it.
	// The tree must exist and currently be soft deleted, as per SoftDeletedTree, otherwise an error
	// is returned.
	// Hard deleted trees cannot be recovered.
	HardDeleteTree(ctx context.Context, treeID int64) error

	// UndeleteTree undeletes a soft-deleted tree.
	// The tree must exist and currently be soft deleted, as per SoftDeletedTree, otherwise an error
	// is returned.
	UndeleteTree(ctx context.Context, treeID int64) (*trillian.Tree, error)
}

AdminWriter provides a write-only interface for tree data.

type LogStorage 

type LogStorage interface {
	ReadOnlyLogStorage

	// ReadWriteTransaction starts a RW transaction on the underlying storage, and
	// calls f with it.
	// If f fails and returns an error, the storage implementation may optionally
	// retry with a new transaction, and f MUST NOT keep state across calls.
	ReadWriteTransaction(ctx context.Context, tree *trillian.Tree, f LogTXFunc) error

	// QueueLeaves enqueues leaves for later integration into the tree.
	// If error is nil, the returned slice of leaves will be the same size as the
	// input, and each entry will hold a passed-in leaf struct and a Status
	// representing the outcome for that particular leaf:
	//  * a status of OK indicates that the leaf was successfully queued.
	//  * a status of AlreadyExists indicates that the leaf was a duplicate, in this case
	//    the returned leaf data is that of the original.
	// Other status values may be returned in error cases.
	//
	// Duplicates are only reported if the underlying tree does not permit duplicates, and are
	// considered duplicate if their leaf.LeafIdentityHash matches.
	QueueLeaves(ctx context.Context, tree *trillian.Tree, leaves []*trillian.LogLeaf, queueTimestamp time.Time) ([]*trillian.QueuedLogLeaf, error)

	// AddSequencedLeaves stores the `leaves` and associates them with the log
	// positions according to their `LeafIndex` field. The indices must be
	// contiguous.
	//
	// If error is nil, the returned slice is the same size as the input, entries
	// correspond to the `leaves` in the same order. Each entry describes the
	// result of adding the corresponding leaf.
	//
	// Possible `QueuedLogLeaf.status` values with their semantics:
	//  - OK: The leaf has been successfully stored.
	//  - AlreadyExists: The storage already contains an identical leaf at the
	//    specified `LeafIndex`. That leaf is returned in `QueuedLogLeaf.leaf`.
	//  - FailedPrecondition: There is another leaf with the same `LeafIndex`,
	//    but a different value. That leaf is returned in `QueuedLogLeaf.leaf`.
	//  - OutOfRange: The leaf can not be stored at the specified `LeafIndex`.
	//    For example, the storage might not support non-sequential writes.
	//  - Internal, etc: A storage-specific error.
	//
	// TODO(pavelkalinnikov): Make returning the resulting/conflicting leaves
	// optional. Channel these options to the top-level Log API.
	// TODO(pavelkalinnikov): Not checking values of the occupied indices might
	// be a good optimization. Could also be optional.
	AddSequencedLeaves(ctx context.Context, tree *trillian.Tree, leaves []*trillian.LogLeaf, timestamp time.Time) ([]*trillian.QueuedLogLeaf, error)
}

LogStorage should be implemented by concrete storage mechanisms which want to support Logs.

type LogTXFunc added in v1.0.7 

type LogTXFunc func(context.Context, LogTreeTX) error

LogTXFunc is the func signature for passing into ReadWriteTransaction.

type LogTreeTX 

type LogTreeTX interface {
	ReadOnlyLogTreeTX

	// SetMerkleNodes writes the nodes, at the write revision.
	//
	// TODO(pavelkalinnikov): Use tiles instead, here and in GetMerkleNodes.
	SetMerkleNodes(ctx context.Context, nodes []tree.Node) error

	// StoreSignedLogRoot stores a freshly created SignedLogRoot.
	StoreSignedLogRoot(ctx context.Context, root *trillian.SignedLogRoot) error

	// DequeueLeaves returns between [0, limit] leaves to be integrated to the
	// tree.
	//
	// For LOG trees:
	// - The leaves are taken from the queue.
	// - If the Tx is rolled back, they become available for dequeueing again.
	//
	// For PREORDERED_LOG trees:
	// - The leaves are taken from the head of as yet un-integrated part of the
	//   sequenced entries, immediately following the current SignedLogRoot tree
	//   size.
	// - The operation is a no-op with regards to the sequenced entries.
	//
	// Leaves queued more recently than the cutoff time will not be returned.
	// This allows for guard intervals to be configured, and (in case of
	// PREORDERED_LOG trees) avoiding contention between log signer and writers
	// appending new entries.
	//
	// This method is not required to return fully populated LogLeaf structures,
	// but it *must* include MerkleLeafHash, QueueTimestamp, and LeafIndex (for
	// PREORDERED_LOG trees). Storage implementations might apply optimizations
	// employing this property. Consult the call sites of this method to be sure.
	DequeueLeaves(ctx context.Context, limit int, cutoff time.Time) ([]*trillian.LogLeaf, error)

	// UpdateSequencedLeaves associates the leaves with the sequence numbers
	// assigned to them.
	UpdateSequencedLeaves(ctx context.Context, leaves []*trillian.LogLeaf) error
}

LogTreeTX is the transactional interface for reading/updating a Log. After a call to Commit or Close implementations must be in a clean state and have released any resources owned by the LogTreeTX. A LogTreeTX can only modify the tree specified in its creation.

type MockAdminStorage 

type MockAdminStorage struct {
	// contains filtered or unexported fields
}

MockAdminStorage is a mock of AdminStorage interface.

func NewMockAdminStorage 

func NewMockAdminStorage(ctrl *gomock.Controller) *MockAdminStorage

NewMockAdminStorage creates a new mock instance.

func (*MockAdminStorage) CheckDatabaseAccessible 

func (m *MockAdminStorage) CheckDatabaseAccessible(arg0 context.Context) error

CheckDatabaseAccessible mocks base method.

func (*MockAdminStorage) EXPECT 

func (m *MockAdminStorage) EXPECT() *MockAdminStorageMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockAdminStorage) ReadWriteTransaction added in v1.0.7 

func (m *MockAdminStorage) ReadWriteTransaction(arg0 context.Context, arg1 AdminTXFunc) error

ReadWriteTransaction mocks base method.

func (*MockAdminStorage) Snapshot 

func (m *MockAdminStorage) Snapshot(arg0 context.Context) (ReadOnlyAdminTX, error)

Snapshot mocks base method.

type MockAdminStorageMockRecorder 

type MockAdminStorageMockRecorder struct {
	// contains filtered or unexported fields
}

MockAdminStorageMockRecorder is the mock recorder for MockAdminStorage.

func (*MockAdminStorageMockRecorder) CheckDatabaseAccessible 

func (mr *MockAdminStorageMockRecorder) CheckDatabaseAccessible(arg0 interface{}) *gomock.Call

CheckDatabaseAccessible indicates an expected call of CheckDatabaseAccessible.

func (*MockAdminStorageMockRecorder) ReadWriteTransaction added in v1.0.7 

func (mr *MockAdminStorageMockRecorder) ReadWriteTransaction(arg0, arg1 interface{}) *gomock.Call

ReadWriteTransaction indicates an expected call of ReadWriteTransaction.

func (*MockAdminStorageMockRecorder) Snapshot 

func (mr *MockAdminStorageMockRecorder) Snapshot(arg0 interface{}) *gomock.Call

Snapshot indicates an expected call of Snapshot.

type MockAdminTX 

type MockAdminTX struct {
	// contains filtered or unexported fields
}

MockAdminTX is a mock of AdminTX interface.

func NewMockAdminTX 

func NewMockAdminTX(ctrl *gomock.Controller) *MockAdminTX

NewMockAdminTX creates a new mock instance.

func (*MockAdminTX) Close 

func (m *MockAdminTX) Close() error

Close mocks base method.

func (*MockAdminTX) Commit 

func (m *MockAdminTX) Commit() error

Commit mocks base method.

func (*MockAdminTX) CreateTree 

func (m *MockAdminTX) CreateTree(arg0 context.Context, arg1 *trillian.Tree) (*trillian.Tree, error)

CreateTree mocks base method.

func (*MockAdminTX) EXPECT 

func (m *MockAdminTX) EXPECT() *MockAdminTXMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockAdminTX) GetTree 

func (m *MockAdminTX) GetTree(arg0 context.Context, arg1 int64) (*trillian.Tree, error)

GetTree mocks base method.

func (*MockAdminTX) HardDeleteTree 

func (m *MockAdminTX) HardDeleteTree(arg0 context.Context, arg1 int64) error

HardDeleteTree mocks base method.

func (*MockAdminTX) ListTrees 

func (m *MockAdminTX) ListTrees(arg0 context.Context, arg1 bool) ([]*trillian.Tree, error)

ListTrees mocks base method.

func (*MockAdminTX) SoftDeleteTree 

func (m *MockAdminTX) SoftDeleteTree(arg0 context.Context, arg1 int64) (*trillian.Tree, error)

SoftDeleteTree mocks base method.

func (*MockAdminTX) UndeleteTree 

func (m *MockAdminTX) UndeleteTree(arg0 context.Context, arg1 int64) (*trillian.Tree, error)

UndeleteTree mocks base method.

func (*MockAdminTX) UpdateTree 

func (m *MockAdminTX) UpdateTree(arg0 context.Context, arg1 int64, arg2 func(*trillian.Tree)) (*trillian.Tree, error)

UpdateTree mocks base method.

type MockAdminTXMockRecorder 

type MockAdminTXMockRecorder struct {
	// contains filtered or unexported fields
}

MockAdminTXMockRecorder is the mock recorder for MockAdminTX.

func (*MockAdminTXMockRecorder) Close 

func (mr *MockAdminTXMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close.

func (*MockAdminTXMockRecorder) Commit 

func (mr *MockAdminTXMockRecorder) Commit() *gomock.Call

Commit indicates an expected call of Commit.

func (*MockAdminTXMockRecorder) CreateTree 

func (mr *MockAdminTXMockRecorder) CreateTree(arg0, arg1 interface{}) *gomock.Call

CreateTree indicates an expected call of CreateTree.

func (*MockAdminTXMockRecorder) GetTree 

func (mr *MockAdminTXMockRecorder) GetTree(arg0, arg1 interface{}) *gomock.Call

GetTree indicates an expected call of GetTree.

func (*MockAdminTXMockRecorder) HardDeleteTree 

func (mr *MockAdminTXMockRecorder) HardDeleteTree(arg0, arg1 interface{}) *gomock.Call

HardDeleteTree indicates an expected call of HardDeleteTree.

func (*MockAdminTXMockRecorder) ListTrees 

func (mr *MockAdminTXMockRecorder) ListTrees(arg0, arg1 interface{}) *gomock.Call

ListTrees indicates an expected call of ListTrees.

func (*MockAdminTXMockRecorder) SoftDeleteTree 

func (mr *MockAdminTXMockRecorder) SoftDeleteTree(arg0, arg1 interface{}) *gomock.Call

SoftDeleteTree indicates an expected call of SoftDeleteTree.

func (*MockAdminTXMockRecorder) UndeleteTree 

func (mr *MockAdminTXMockRecorder) UndeleteTree(arg0, arg1 interface{}) *gomock.Call

UndeleteTree indicates an expected call of UndeleteTree.

func (*MockAdminTXMockRecorder) UpdateTree 

func (mr *MockAdminTXMockRecorder) UpdateTree(arg0, arg1, arg2 interface{}) *gomock.Call

UpdateTree indicates an expected call of UpdateTree.

type MockLogStorage 

type MockLogStorage struct {
	// contains filtered or unexported fields
}

MockLogStorage is a mock of LogStorage interface.

func NewMockLogStorage 

func NewMockLogStorage(ctrl *gomock.Controller) *MockLogStorage

NewMockLogStorage creates a new mock instance.

func (*MockLogStorage) AddSequencedLeaves added in v1.0.7 

func (m *MockLogStorage) AddSequencedLeaves(arg0 context.Context, arg1 *trillian.Tree, arg2 []*trillian.LogLeaf, arg3 time.Time) ([]*trillian.QueuedLogLeaf, error)

AddSequencedLeaves mocks base method.

func (*MockLogStorage) CheckDatabaseAccessible 

func (m *MockLogStorage) CheckDatabaseAccessible(arg0 context.Context) error

CheckDatabaseAccessible mocks base method.

func (*MockLogStorage) EXPECT 

func (m *MockLogStorage) EXPECT() *MockLogStorageMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockLogStorage) GetActiveLogIDs added in v1.4.0 

func (m *MockLogStorage) GetActiveLogIDs(arg0 context.Context) ([]int64, error)

GetActiveLogIDs mocks base method.

func (*MockLogStorage) QueueLeaves added in v1.0.7 

func (m *MockLogStorage) QueueLeaves(arg0 context.Context, arg1 *trillian.Tree, arg2 []*trillian.LogLeaf, arg3 time.Time) ([]*trillian.QueuedLogLeaf, error)

QueueLeaves mocks base method.

func (*MockLogStorage) ReadWriteTransaction added in v1.0.7 

func (m *MockLogStorage) ReadWriteTransaction(arg0 context.Context, arg1 *trillian.Tree, arg2 LogTXFunc) error

ReadWriteTransaction mocks base method.

func (*MockLogStorage) SnapshotForTree 

func (m *MockLogStorage) SnapshotForTree(arg0 context.Context, arg1 *trillian.Tree) (ReadOnlyLogTreeTX, error)

SnapshotForTree mocks base method.

type MockLogStorageMockRecorder 

type MockLogStorageMockRecorder struct {
	// contains filtered or unexported fields
}

MockLogStorageMockRecorder is the mock recorder for MockLogStorage.

func (*MockLogStorageMockRecorder) AddSequencedLeaves added in v1.0.7 

func (mr *MockLogStorageMockRecorder) AddSequencedLeaves(arg0, arg1, arg2, arg3 interface{}) *gomock.Call

AddSequencedLeaves indicates an expected call of AddSequencedLeaves.

func (*MockLogStorageMockRecorder) CheckDatabaseAccessible 

func (mr *MockLogStorageMockRecorder) CheckDatabaseAccessible(arg0 interface{}) *gomock.Call

CheckDatabaseAccessible indicates an expected call of CheckDatabaseAccessible.

func (*MockLogStorageMockRecorder) GetActiveLogIDs added in v1.4.0 

func (mr *MockLogStorageMockRecorder) GetActiveLogIDs(arg0 interface{}) *gomock.Call

GetActiveLogIDs indicates an expected call of GetActiveLogIDs.

func (*MockLogStorageMockRecorder) QueueLeaves added in v1.0.7 

func (mr *MockLogStorageMockRecorder) QueueLeaves(arg0, arg1, arg2, arg3 interface{}) *gomock.Call

QueueLeaves indicates an expected call of QueueLeaves.

func (*MockLogStorageMockRecorder) ReadWriteTransaction added in v1.0.7 

func (mr *MockLogStorageMockRecorder) ReadWriteTransaction(arg0, arg1, arg2 interface{}) *gomock.Call

ReadWriteTransaction indicates an expected call of ReadWriteTransaction.

func (*MockLogStorageMockRecorder) SnapshotForTree 

func (mr *MockLogStorageMockRecorder) SnapshotForTree(arg0, arg1 interface{}) *gomock.Call

SnapshotForTree indicates an expected call of SnapshotForTree.

type MockLogTreeTX 

type MockLogTreeTX struct {
	// contains filtered or unexported fields
}

MockLogTreeTX is a mock of LogTreeTX interface.

func NewMockLogTreeTX 

func NewMockLogTreeTX(ctrl *gomock.Controller) *MockLogTreeTX

NewMockLogTreeTX creates a new mock instance.

func (*MockLogTreeTX) Close 

func (m *MockLogTreeTX) Close() error

Close mocks base method.

func (*MockLogTreeTX) Commit 

func (m *MockLogTreeTX) Commit(arg0 context.Context) error

Commit mocks base method.

func (*MockLogTreeTX) DequeueLeaves 

func (m *MockLogTreeTX) DequeueLeaves(arg0 context.Context, arg1 int, arg2 time.Time) ([]*trillian.LogLeaf, error)

DequeueLeaves mocks base method.

func (*MockLogTreeTX) EXPECT 

func (m *MockLogTreeTX) EXPECT() *MockLogTreeTXMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockLogTreeTX) GetLeavesByHash 

func (m *MockLogTreeTX) GetLeavesByHash(arg0 context.Context, arg1 [][]byte, arg2 bool) ([]*trillian.LogLeaf, error)

GetLeavesByHash mocks base method.

func (*MockLogTreeTX) GetLeavesByRange added in v1.0.6 

func (m *MockLogTreeTX) GetLeavesByRange(arg0 context.Context, arg1, arg2 int64) ([]*trillian.LogLeaf, error)

GetLeavesByRange mocks base method.

func (*MockLogTreeTX) GetMerkleNodes 

func (m *MockLogTreeTX) GetMerkleNodes(arg0 context.Context, arg1 []compact.NodeID) ([]tree.Node, error)

GetMerkleNodes mocks base method.

func (*MockLogTreeTX) LatestSignedLogRoot 

func (m *MockLogTreeTX) LatestSignedLogRoot(arg0 context.Context) (*trillian.SignedLogRoot, error)

LatestSignedLogRoot mocks base method.

func (*MockLogTreeTX) SetMerkleNodes 

func (m *MockLogTreeTX) SetMerkleNodes(arg0 context.Context, arg1 []tree.Node) error

SetMerkleNodes mocks base method.

func (*MockLogTreeTX) StoreSignedLogRoot 

func (m *MockLogTreeTX) StoreSignedLogRoot(arg0 context.Context, arg1 *trillian.SignedLogRoot) error

StoreSignedLogRoot mocks base method.

func (*MockLogTreeTX) UpdateSequencedLeaves 

func (m *MockLogTreeTX) UpdateSequencedLeaves(arg0 context.Context, arg1 []*trillian.LogLeaf) error

UpdateSequencedLeaves mocks base method.

type MockLogTreeTXMockRecorder 

type MockLogTreeTXMockRecorder struct {
	// contains filtered or unexported fields
}

MockLogTreeTXMockRecorder is the mock recorder for MockLogTreeTX.

func (*MockLogTreeTXMockRecorder) Close 

func (mr *MockLogTreeTXMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close.

func (*MockLogTreeTXMockRecorder) Commit 

func (mr *MockLogTreeTXMockRecorder) Commit(arg0 interface{}) *gomock.Call

Commit indicates an expected call of Commit.

func (*MockLogTreeTXMockRecorder) DequeueLeaves 

func (mr *MockLogTreeTXMockRecorder) DequeueLeaves(arg0, arg1, arg2 interface{}) *gomock.Call

DequeueLeaves indicates an expected call of DequeueLeaves.

func (*MockLogTreeTXMockRecorder) GetLeavesByHash 

func (mr *MockLogTreeTXMockRecorder) GetLeavesByHash(arg0, arg1, arg2 interface{}) *gomock.Call

GetLeavesByHash indicates an expected call of GetLeavesByHash.

func (*MockLogTreeTXMockRecorder) GetLeavesByRange added in v1.0.6 

func (mr *MockLogTreeTXMockRecorder) GetLeavesByRange(arg0, arg1, arg2 interface{}) *gomock.Call

GetLeavesByRange indicates an expected call of GetLeavesByRange.

func (*MockLogTreeTXMockRecorder) GetMerkleNodes 

func (mr *MockLogTreeTXMockRecorder) GetMerkleNodes(arg0, arg1 interface{}) *gomock.Call

GetMerkleNodes indicates an expected call of GetMerkleNodes.

func (*MockLogTreeTXMockRecorder) LatestSignedLogRoot 

func (mr *MockLogTreeTXMockRecorder) LatestSignedLogRoot(arg0 interface{}) *gomock.Call

LatestSignedLogRoot indicates an expected call of LatestSignedLogRoot.

func (*MockLogTreeTXMockRecorder) SetMerkleNodes 

func (mr *MockLogTreeTXMockRecorder) SetMerkleNodes(arg0, arg1 interface{}) *gomock.Call

SetMerkleNodes indicates an expected call of SetMerkleNodes.

func (*MockLogTreeTXMockRecorder) StoreSignedLogRoot 

func (mr *MockLogTreeTXMockRecorder) StoreSignedLogRoot(arg0, arg1 interface{}) *gomock.Call

StoreSignedLogRoot indicates an expected call of StoreSignedLogRoot.

func (*MockLogTreeTXMockRecorder) UpdateSequencedLeaves 

func (mr *MockLogTreeTXMockRecorder) UpdateSequencedLeaves(arg0, arg1 interface{}) *gomock.Call

UpdateSequencedLeaves indicates an expected call of UpdateSequencedLeaves.

type MockReadOnlyAdminTX 

type MockReadOnlyAdminTX struct {
	// contains filtered or unexported fields
}

MockReadOnlyAdminTX is a mock of ReadOnlyAdminTX interface.

func NewMockReadOnlyAdminTX 

func NewMockReadOnlyAdminTX(ctrl *gomock.Controller) *MockReadOnlyAdminTX

NewMockReadOnlyAdminTX creates a new mock instance.

func (*MockReadOnlyAdminTX) Close 

func (m *MockReadOnlyAdminTX) Close() error

Close mocks base method.

func (*MockReadOnlyAdminTX) Commit 

func (m *MockReadOnlyAdminTX) Commit() error

Commit mocks base method.

func (*MockReadOnlyAdminTX) EXPECT 

func (m *MockReadOnlyAdminTX) EXPECT() *MockReadOnlyAdminTXMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockReadOnlyAdminTX) GetTree 

func (m *MockReadOnlyAdminTX) GetTree(arg0 context.Context, arg1 int64) (*trillian.Tree, error)

GetTree mocks base method.

func (*MockReadOnlyAdminTX) ListTrees 

func (m *MockReadOnlyAdminTX) ListTrees(arg0 context.Context, arg1 bool) ([]*trillian.Tree, error)

ListTrees mocks base method.

type MockReadOnlyAdminTXMockRecorder 

type MockReadOnlyAdminTXMockRecorder struct {
	// contains filtered or unexported fields
}

MockReadOnlyAdminTXMockRecorder is the mock recorder for MockReadOnlyAdminTX.

func (*MockReadOnlyAdminTXMockRecorder) Close 

func (mr *MockReadOnlyAdminTXMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close.

func (*MockReadOnlyAdminTXMockRecorder) Commit 

func (mr *MockReadOnlyAdminTXMockRecorder) Commit() *gomock.Call

Commit indicates an expected call of Commit.

func (*MockReadOnlyAdminTXMockRecorder) GetTree 

func (mr *MockReadOnlyAdminTXMockRecorder) GetTree(arg0, arg1 interface{}) *gomock.Call

GetTree indicates an expected call of GetTree.

func (*MockReadOnlyAdminTXMockRecorder) ListTrees 

func (mr *MockReadOnlyAdminTXMockRecorder) ListTrees(arg0, arg1 interface{}) *gomock.Call

ListTrees indicates an expected call of ListTrees.

type MockReadOnlyLogTreeTX 

type MockReadOnlyLogTreeTX struct {
	// contains filtered or unexported fields
}

MockReadOnlyLogTreeTX is a mock of ReadOnlyLogTreeTX interface.

func NewMockReadOnlyLogTreeTX 

func NewMockReadOnlyLogTreeTX(ctrl *gomock.Controller) *MockReadOnlyLogTreeTX

NewMockReadOnlyLogTreeTX creates a new mock instance.

func (*MockReadOnlyLogTreeTX) Close 

func (m *MockReadOnlyLogTreeTX) Close() error

Close mocks base method.

func (*MockReadOnlyLogTreeTX) Commit 

func (m *MockReadOnlyLogTreeTX) Commit(arg0 context.Context) error

Commit mocks base method.

func (*MockReadOnlyLogTreeTX) EXPECT 

func (m *MockReadOnlyLogTreeTX) EXPECT() *MockReadOnlyLogTreeTXMockRecorder

EXPECT returns an object that allows the caller to indicate expected use.

func (*MockReadOnlyLogTreeTX) GetLeavesByHash 

func (m *MockReadOnlyLogTreeTX) GetLeavesByHash(arg0 context.Context, arg1 [][]byte, arg2 bool) ([]*trillian.LogLeaf, error)

GetLeavesByHash mocks base method.

func (*MockReadOnlyLogTreeTX) GetLeavesByRange added in v1.0.6 

func (m *MockReadOnlyLogTreeTX) GetLeavesByRange(arg0 context.Context, arg1, arg2 int64) ([]*trillian.LogLeaf, error)

GetLeavesByRange mocks base method.

func (*MockReadOnlyLogTreeTX) GetMerkleNodes 

func (m *MockReadOnlyLogTreeTX) GetMerkleNodes(arg0 context.Context, arg1 []compact.NodeID) ([]tree.Node, error)

GetMerkleNodes mocks base method.

func (*MockReadOnlyLogTreeTX) LatestSignedLogRoot 

func (m *MockReadOnlyLogTreeTX) LatestSignedLogRoot(arg0 context.Context) (*trillian.SignedLogRoot, error)

LatestSignedLogRoot mocks base method.

type MockReadOnlyLogTreeTXMockRecorder 

type MockReadOnlyLogTreeTXMockRecorder struct {
	// contains filtered or unexported fields
}

MockReadOnlyLogTreeTXMockRecorder is the mock recorder for MockReadOnlyLogTreeTX.

func (*MockReadOnlyLogTreeTXMockRecorder) Close 

func (mr *MockReadOnlyLogTreeTXMockRecorder) Close() *gomock.Call

Close indicates an expected call of Close.

func (*MockReadOnlyLogTreeTXMockRecorder) Commit 

func (mr *MockReadOnlyLogTreeTXMockRecorder) Commit(arg0 interface{}) *gomock.Call

Commit indicates an expected call of Commit.

func (*MockReadOnlyLogTreeTXMockRecorder) GetLeavesByHash 

func (mr *MockReadOnlyLogTreeTXMockRecorder) GetLeavesByHash(arg0, arg1, arg2 interface{}) *gomock.Call

GetLeavesByHash indicates an expected call of GetLeavesByHash.

func (*MockReadOnlyLogTreeTXMockRecorder) GetLeavesByRange added in v1.0.6 

func (mr *MockReadOnlyLogTreeTXMockRecorder) GetLeavesByRange(arg0, arg1, arg2 interface{}) *gomock.Call

GetLeavesByRange indicates an expected call of GetLeavesByRange.

func (*MockReadOnlyLogTreeTXMockRecorder) GetMerkleNodes 

func (mr *MockReadOnlyLogTreeTXMockRecorder) GetMerkleNodes(arg0, arg1 interface{}) *gomock.Call

GetMerkleNodes indicates an expected call of GetMerkleNodes.

func (*MockReadOnlyLogTreeTXMockRecorder) LatestSignedLogRoot 

func (mr *MockReadOnlyLogTreeTXMockRecorder) LatestSignedLogRoot(arg0 interface{}) *gomock.Call

LatestSignedLogRoot indicates an expected call of LatestSignedLogRoot.

type NewProviderFunc added in v1.3.4 

type NewProviderFunc func(monitoring.MetricFactory) (Provider, error)

NewProviderFunc is the signature of a function which can be registered to provide instances of storage providers.

type Provider added in v1.3.4 

type Provider interface {
	// LogStorage creates and returns a LogStorage implementation.
	LogStorage() LogStorage
	// AdminStorage creates and returns a AdminStorage implementation.
	AdminStorage() AdminStorage

	// Close closes the underlying storage.
	Close() error
}

Provider is an interface which allows Trillian binaries to use different storage implementations.

func NewProvider added in v1.3.4 

func NewProvider(name string, mf monitoring.MetricFactory) (Provider, error)

NewProvider returns a new Provider instance of the type specified by name.

type ReadOnlyAdminTX 

type ReadOnlyAdminTX interface {
	// GetTree returns the tree corresponding to treeID or an error.
	GetTree(ctx context.Context, treeID int64) (*trillian.Tree, error)

	// ListTrees returns all trees in storage.
	// Note that there's no authorization restriction on the trees returned,
	// so it should be used with caution in production code.
	ListTrees(ctx context.Context, includeDeleted bool) ([]*trillian.Tree, error)

	// Commit applies the operations performed to the underlying storage. It must
	// be called before any reads from storage are considered consistent.
	Commit() error

	// Close rolls back the transaction if it wasn't committed or closed
	// previously. Resources are cleaned up regardless of the success, and the
	// transaction should not be used after it.
	Close() error
}

ReadOnlyAdminTX is a transaction capable only of read operations in the AdminStorage.

type ReadOnlyLogStorage 

type ReadOnlyLogStorage interface {
	// CheckDatabaseAccessible returns nil if the database is accessible, or an
	// error otherwise.
	CheckDatabaseAccessible(context.Context) error

	// GetActiveLogIDs returns a list of the IDs of all the logs that are
	// configured in storage and are eligible to have entries sequenced.
	GetActiveLogIDs(ctx context.Context) ([]int64, error)

	// SnapshotForTree starts a read-only transaction for the specified treeID.
	// Commit must be called when the caller is finished with the returned object,
	// and values read through it should only be propagated if Commit returns
	// without error.
	SnapshotForTree(ctx context.Context, tree *trillian.Tree) (ReadOnlyLogTreeTX, error)
}

ReadOnlyLogStorage represents a narrowed read-only view into a LogStorage.

type ReadOnlyLogTreeTX 

type ReadOnlyLogTreeTX interface {
	// Commit applies the operations performed to the underlying storage. It must
	// be called before any reads from storage are considered consistent.
	Commit(context.Context) error

	// Close rolls back the transaction if it wasn't committed or closed
	// previously. Resources are cleaned up regardless of the success, and the
	// transaction should not be used after it.
	Close() error

	// GetMerkleNodes returns tree nodes by their IDs, in the requested order.
	GetMerkleNodes(ctx context.Context, ids []compact.NodeID) ([]tree.Node, error)
	// GetLeavesByRange returns leaf data for a range of indexes. The returned
	// slice is a contiguous prefix of leaves in [start, start+count) ordered by
	// LeafIndex. It will be shorter than `count` if the requested range has
	// missing entries (e.g., it extends beyond the size of a LOG tree), or
	// `count` is too big to handle in one go.
	// For PREORDERED_LOG trees, *must* return leaves beyond the tree size if
	// they are stored, in order to allow integrating them into the tree.
	GetLeavesByRange(ctx context.Context, start, count int64) ([]*trillian.LogLeaf, error)
	// GetLeavesByHash looks up sequenced leaf metadata and data by their Merkle leaf hash. If the
	// tree permits duplicate leaves callers must be prepared to handle multiple results with the
	// same hash but different sequence numbers. If orderBySequence is true then the returned data
	// will be in ascending sequence number order.
	GetLeavesByHash(ctx context.Context, leafHashes [][]byte, orderBySequence bool) ([]*trillian.LogLeaf, error)
	// LatestSignedLogRoot returns the most recent SignedLogRoot, if any.
	LatestSignedLogRoot(ctx context.Context) (*trillian.SignedLogRoot, error)
}

ReadOnlyLogTreeTX provides a read-only view into the Log data. A ReadOnlyLogTreeTX can only read from the tree specified in its creation.

Source Files

View all Source files

Directories

Path Synopsis
Package cache provides subtree caching functionality.
 Package cache provides subtree caching functionality.
Package cloudspanner contains the Cloud Spanner storage implementation.
 Package cloudspanner contains the Cloud Spanner storage implementation.
spannerpb
Package spannerpb contains the generated protobuf code for the Spanner storage implementation.
 Package spannerpb contains the generated protobuf code for the Spanner storage implementation.
Package crdb provides a CockroachDB-based storage layer implementation.
 Package crdb provides a CockroachDB-based storage layer implementation.
Package memory provides a simple in-process implementation of the tree- and log-storage interfaces.
 Package memory provides a simple in-process implementation of the tree- and log-storage interfaces.
Package mysql provides a MySQL-based storage layer implementation.
 Package mysql provides a MySQL-based storage layer implementation.
mysqlpb
Package mysqlpb contains protobuf definitions used by the mysql implementation.
 Package mysqlpb contains protobuf definitions used by the mysql implementation.
Package postgresql provides a PostgreSQL-based storage layer implementation.
 Package postgresql provides a PostgreSQL-based storage layer implementation.
testdbpgx
Package testdbpgx creates new PostgreSQL databases for tests.
 Package testdbpgx creates new PostgreSQL databases for tests.
Package storagepb contains protobuf definitions used by various storage implementations.
 Package storagepb contains protobuf definitions used by various storage implementations.
Package testdb creates new databases for tests.
 Package testdb creates new databases for tests.
Package testonly contains utilities and helpers to use in the storage tests.
 Package testonly contains utilities and helpers to use in the storage tests.
Package tree defines types that help navigating a tree in storage.
 Package tree defines types that help navigating a tree in storage.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.