tstore

Documentation

Index

• func DisableLog()
• func NewTstoreClient(tstore *Tstore, pluginID string) plugins.TstoreClient
• func UseLogger(logger slog.Logger)
• type Tstore
  • func New(appDir, dataDir string, anp *chaincfg.Params, ...) (*Tstore, error)
  • func NewTestTstore(t *testing.T, dataDir string) *Tstore
  • func (t *Tstore) Close()
  • func (t *Tstore) Fsck(allTokens [][]byte) error
  • func (t *Tstore) Inventory() ([][]byte, error)
  • func (t *Tstore) PluginHookPost(h plugins.HookT, payload string)
  • func (t *Tstore) PluginHookPre(h plugins.HookT, payload string) error
  • func (t *Tstore) PluginRead(token []byte, pluginID, cmd, payload string) (string, error)
  • func (t *Tstore) PluginRegister(b backend.Backend, p backend.Plugin) error
  • func (t *Tstore) PluginSetup(pluginID string) error
  • func (t *Tstore) PluginWrite(token []byte, pluginID, cmd, payload string) (string, error)
  • func (t *Tstore) Plugins() []backend.Plugin
  • func (t *Tstore) Record(token []byte, version uint32) (*backend.Record, error)
  • func (t *Tstore) RecordDel(token []byte) error
  • func (t *Tstore) RecordExists(token []byte) bool
  • func (t *Tstore) RecordFreeze(token []byte, rm backend.RecordMetadata, metadata []backend.MetadataStream, ...) error
  • func (t *Tstore) RecordLatest(token []byte) (*backend.Record, error)
  • func (t *Tstore) RecordNew() ([]byte, error)
  • func (t *Tstore) RecordPartial(token []byte, version uint32, filenames []string, omitAllFiles bool) (*backend.Record, error)
  • func (t *Tstore) RecordSave(token []byte, rm backend.RecordMetadata, metadata []backend.MetadataStream, ...) error
  • func (t *Tstore) RecordState(token []byte) (backend.StateT, error)
  • func (t *Tstore) RecordTimestamps(token []byte, version uint32) (*backend.RecordTimestamps, error)
  • func (t *Tstore) Setup() error

Functions

func DisableLog

func DisableLog()

DisableLog disables all library log output. Logging output is disabled by default until either UseLogger or SetLogWriter are called.

func NewTstoreClient

func NewTstoreClient(tstore *Tstore, pluginID string) plugins.TstoreClient

NewTstoreClient returns a new TstoreClient interface that is backed by a tstoreClient structure.

func UseLogger

func UseLogger(logger slog.Logger)

UseLogger uses a specified Logger to output package logging info. This should be used in preference to SetLogWriter if the caller is also using slog.

Types

type Tstore

type Tstore struct {
	sync.RWMutex
	// contains filtered or unexported fields
}

Tstore is a data store that automatically timestamps all data saved to it onto the decred blockchain, making it possible to cryptographically prove that a piece of data existed at a specific block height. It combines a trillian log (tlog) and a key-value store. When data is saved to a tstore instance it is first saved to the key-value store then a digest of the data is appended onto the tlog tree. Tlog trees are episodically timestamped onto the decred blockchain. An inlcusion proof, i.e. the cryptographic proof that the data was included in the decred timestamp, can be retrieved for any individual piece of data saved to the tstore.

Saving only the digest of the data to tlog means that we separate the timestamp from the data itself. This allows us to remove content that is deemed undesirable from the key-value store without impacting the ability to retrieve inclusion proofs for any other pieces of data saved to tstore.

The tlog tree is append only and is treated as the source of truth. If any blobs make it into the key-value store but do not make it into the tlog tree they are considered to be orphaned and are simply ignored. We do not unwind failed calls.

func New

func New(appDir, dataDir string, anp *chaincfg.Params, tlogHost, dbHost, dbPass, dcrtimeHost, dcrtimeCert string) (*Tstore, error)

New returns a new tstore instance.

func NewTestTstore

func NewTestTstore(t *testing.T, dataDir string) *Tstore

NewTestTstore returns a tstore instance that is setup for testing.

func (*Tstore) Close

func (t *Tstore) Close()

Close performs cleanup of the tstore.

func (*Tstore) Fsck

func (t *Tstore) Fsck(allTokens [][]byte) error

Fsck performs a filesystem check on the tstore.

func (*Tstore) Inventory

func (t *Tstore) Inventory() ([][]byte, error)

Inventory returns all record tokens that are in the tstore. Its possible for a token to be returned that does not correspond to an actual record. For example, if the tlog tree was created but saving the record to the tree failed due to an unexpected error then a empty tree with exist. This function does not filter those tokens out.

func (*Tstore) PluginHookPost

func (t *Tstore) PluginHookPost(h plugins.HookT, payload string)

PluginHookPost executes a tstore backend post hook. Post hooks are hooks that are executed after the tstore backend successfully writes data to disk. These hooks give plugins the opportunity to cache data from the write.

func (*Tstore) PluginHookPre

func (t *Tstore) PluginHookPre(h plugins.HookT, payload string) error

PluginHookPre executes a tstore backend pre hook. Pre hooks are hooks that are executed prior to the tstore backend writing data to disk. These hooks give plugins the opportunity to add plugin specific validation to record methods or plugin commands that write data.

func (*Tstore) PluginRead

func (t *Tstore) PluginRead(token []byte, pluginID, cmd, payload string) (string, error)

PluginRead executes a read-only plugin command.

func (*Tstore) PluginRegister

func (t *Tstore) PluginRegister(b backend.Backend, p backend.Plugin) error

PluginRegister registers a plugin. Plugin commands and hooks can be executed on the plugin once registered.

func (*Tstore) PluginSetup

func (t *Tstore) PluginSetup(pluginID string) error

PluginSetup performs any required setup for the specified plugin.

func (*Tstore) PluginWrite

func (t *Tstore) PluginWrite(token []byte, pluginID, cmd, payload string) (string, error)

PluginWrite executes a plugin command that writes data.

func (*Tstore) Plugins

func (t *Tstore) Plugins() []backend.Plugin

Plugins returns all registered plugins for the tstore instance.

func (*Tstore) Record

func (t *Tstore) Record(token []byte, version uint32) (*backend.Record, error)

Record returns the specified version of the record.

func (*Tstore) RecordDel

func (t *Tstore) RecordDel(token []byte) error

RecordDel walks the provided tree and deletes all blobs in the store that correspond to record files. This is done for all versions and all iterations of the record. Record metadata and metadata stream blobs are not deleted.

func (*Tstore) RecordExists

func (t *Tstore) RecordExists(token []byte) bool

RecordExists returns whether a record exists.

This method only returns whether a tree exists for the provided token. It's possible for a tree to exist that does not correspond to a record in the rare case that a tree was created but an unexpected error, such as a network error, was encoutered prior to the record being saved to the tree. We ignore this edge case because:

1. A user has no way to obtain this token unless the trillian instance has been opened to the public.

2. Even if they have the token they cannot do anything with it. Any attempt to read from the tree or write to the tree will return a RecordNotFound error.

Pulling the leaves from the tree to see if a record has been saved to the tree adds a large amount of overhead to this call, which should be a very light weight. Its for this reason that we rely on the tree exists call despite the edge case.

func (*Tstore) RecordFreeze

func (t *Tstore) RecordFreeze(token []byte, rm backend.RecordMetadata, metadata []backend.MetadataStream, files []backend.File) error

RecordFreeze updates the status of a record then freezes the trillian tree to prevent any additional updates.

A tree is considered to be frozen once the record index has been saved with its Frozen field set to true. The only thing that can be appended onto a frozen tree is one additional anchor record. Once a frozen tree has been anchored, the tstore fsck function will update the status of the tree to frozen in trillian, at which point trillian will prevent any changes to the tree.

func (*Tstore) RecordLatest

func (t *Tstore) RecordLatest(token []byte) (*backend.Record, error)

RecordLatest returns the latest version of a record.

func (*Tstore) RecordNew

func (t *Tstore) RecordNew() ([]byte, error)

RecordNew creates a new record in the tstore and returns the record token that serves as the unique identifier for the record. Creating a new record means creating a tlog tree for the record. Nothing is saved to the tree yet.

func (*Tstore) RecordPartial

func (t *Tstore) RecordPartial(token []byte, version uint32, filenames []string, omitAllFiles bool) (*backend.Record, error)

RecordPartial returns a partial record. This method gives the caller fine grained control over what version and what files are returned. The only required field is the token. All other fields are optional.

Version is used to request a specific version of a record. If no version is provided then the most recent version of the record will be returned.

Filenames can be used to request specific files. If filenames is not empty then the specified files will be the only files returned.

OmitAllFiles can be used to retrieve a record without any of the record files. This supersedes the filenames argument.

func (*Tstore) RecordSave

func (t *Tstore) RecordSave(token []byte, rm backend.RecordMetadata, metadata []backend.MetadataStream, files []backend.File) error

RecordSave saves the provided record to tstore. Once the record contents have been successfully saved to tstore, a recordIndex is created for this version of the record and saved to tstore as well. The record update is not considered to be valid until the record index has been successfully saved. If the record content makes it in but the record index does not, the record content blobs are orphaned and ignored.

func (*Tstore) RecordState

func (t *Tstore) RecordState(token []byte) (backend.StateT, error)

RecordState returns the state of a record. This call does not require retrieving any blobs from the kv store. The record state can be derived from only the tlog leaves.

func (*Tstore) RecordTimestamps

func (t *Tstore) RecordTimestamps(token []byte, version uint32) (*backend.RecordTimestamps, error)

RecordTimestamps returns the timestamps for the contents of a record. Timestamps for the record metadata, metadata streams, and files are all returned.

func (*Tstore) Setup

func (t *Tstore) Setup() error

Setup performs any required work to setup the tstore instance.