sequencer

Documentation

Overview

Package sequencer reads mutations and applies them to the Trillian Map.

Index

• func PeriodicallyRun(ctx context.Context, tickch <-chan time.Time, f func(ctx context.Context))
• type Batcher
• type LogsReader
• type MapClient
  • func (c *MapClient) GetAndVerifyLatestMapRoot(ctx context.Context) (*tpb.SignedMapRoot, *types.MapRootV1, error)
  • func (c *MapClient) GetAndVerifyMapRootByRevision(ctx context.Context, rev int64) (*tpb.SignedMapRoot, *types.MapRootV1, error)
  • func (c *MapClient) SetLeavesAtRevision(ctx context.Context, rev int64, leaves []*tpb.MapLeaf, metadata []byte) (*types.MapRootV1, error)
• type Sequencer
  • func New(sequencerClient spb.KeyTransparencySequencerClient, ...) *Sequencer
  • func (s *Sequencer) RunBatchForAllDirectories(ctx context.Context) error
  • func (s *Sequencer) TrackMasterships(ctx context.Context)
• type Server
  • func NewServer(directories directory.Storage, logAdmin tpb.TrillianAdminClient, ...) *Server
  • func (s *Server) ApplyRevision(ctx context.Context, in *spb.ApplyRevisionRequest) (*spb.ApplyRevisionResponse, error)
  • func (s *Server) DefineRevisions(ctx context.Context, in *spb.DefineRevisionsRequest) (*spb.DefineRevisionsResponse, error)
  • func (s *Server) HighWatermarks(ctx context.Context, directoryID string, lastMeta *spb.MapMetadata, ...) (int32, *spb.MapMetadata, error)
  • func (s *Server) PublishRevisions(ctx context.Context, in *spb.PublishRevisionsRequest) (*spb.PublishRevisionsResponse, error)
  • func (s *Server) RunBatch(ctx context.Context, in *spb.RunBatchRequest) (*empty.Empty, error)
• type Trillian
  • func (t *Trillian) LogClient(ctx context.Context, dirID string) (trillianLog, error)
  • func (t *Trillian) MapClient(ctx context.Context, dirID string) (trillianMap, error)
• type Watermarks

Functions

func PeriodicallyRun

func PeriodicallyRun(ctx context.Context, tickch <-chan time.Time, f func(ctx context.Context))

PeriodicallyRun executes f once per tick until ctx is closed. Closing ctx will also stop any in-flight operation mid-way through.

Types

type Batcher

type Batcher interface {
	// WriteBatchSources saves the (low, high] boundaries used for each log in making this revision.
	WriteBatchSources(ctx context.Context, dirID string, rev int64, meta *spb.MapMetadata) error
	// ReadBatch returns the batch definitions for a given revision.
	ReadBatch(ctx context.Context, directoryID string, rev int64) (*spb.MapMetadata, error)
	// HighestRev returns the highest defined revision number for directoryID.
	HighestRev(ctx context.Context, directoryID string) (int64, error)
}

Batcher writes batch definitions to storage.

type LogsReader

type LogsReader interface {
	// HighWatermark returns the number of items and the highest primary
	// key up to batchSize items after start (exclusive).
	HighWatermark(ctx context.Context, directoryID string, logID, start int64,
		batchSize int32) (count int32, watermark int64, err error)

	// ListLogs returns the logIDs associated with directoryID that have their write bits set,
	// or all logIDs associated with directoryID if writable is false.
	ListLogs(ctx context.Context, directoryID string, writable bool) ([]int64, error)

	// ReadLog returns the lowest messages in the (low, high] range stored in the
	// specified log, up to batchSize.  Paginate by setting low to the
	// highest LogMessage returned in the previous page.
	ReadLog(ctx context.Context, directoryID string, logID, low, high int64,
		batchSize int32) ([]*mutator.LogMessage, error)
}

LogsReader reads messages in multiple logs.

type MapClient

type MapClient struct {
	*tclient.MapClient
}

MapClient interacts with the Trillian Map and verifies its responses.

func (*MapClient) GetAndVerifyLatestMapRoot

func (c *MapClient) GetAndVerifyLatestMapRoot(ctx context.Context) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyLatestMapRoot verifies and returns the latest map root.

func (*MapClient) GetAndVerifyMapRootByRevision

func (c *MapClient) GetAndVerifyMapRootByRevision(ctx context.Context,
	rev int64) (*tpb.SignedMapRoot, *types.MapRootV1, error)

GetAndVerifyMapRootByRevision verifies and returns a specific map root.

func (*MapClient) SetLeavesAtRevision

func (c *MapClient) SetLeavesAtRevision(ctx context.Context, rev int64,
	leaves []*tpb.MapLeaf, metadata []byte) (*types.MapRootV1, error)

SetLeavesAtRevision creates a new map revision and returns its verified root. TODO(gbelvin): Move to Trillian Map client.

type Sequencer

type Sequencer struct {
	// contains filtered or unexported fields
}

Sequencer processes mutations and sends them to the trillian map.

func New

func New(
	sequencerClient spb.KeyTransparencySequencerClient,
	mapAdmin tpb.TrillianAdminClient,
	directories directory.Storage,
	batchSize int32,
	tracker *election.Tracker,
) *Sequencer

New creates a new instance of the signer.

func (*Sequencer) RunBatchForAllDirectories

func (s *Sequencer) RunBatchForAllDirectories(ctx context.Context) error

RunBatchForAllDirectories scans the directories table for new directories and creates new receivers for directories that the sequencer is not currently receiving for.

func (*Sequencer) TrackMasterships

func (s *Sequencer) TrackMasterships(ctx context.Context)

TrackMasterships monitors resources for mastership.

type Server

type Server struct {
	// contains filtered or unexported fields
}

Server implements KeyTransparencySequencerServer.

func NewServer

func NewServer(
	directories directory.Storage,
	logAdmin tpb.TrillianAdminClient,
	mapAdmin tpb.TrillianAdminClient,
	tlog tpb.TrillianLogClient,
	tmap tpb.TrillianMapClient,
	batcher Batcher,
	logs LogsReader,
	loopback spb.KeyTransparencySequencerClient,
	metricsFactory monitoring.MetricFactory,
) *Server

NewServer creates a new KeyTransparencySequencerServer.

func (*Server) ApplyRevision

func (s *Server) ApplyRevision(ctx context.Context, in *spb.ApplyRevisionRequest) (*spb.ApplyRevisionResponse, error)

ApplyRevision applies the supplied mutations to the current map revision and creates a new revision.

func (*Server) DefineRevisions

func (s *Server) DefineRevisions(ctx context.Context,
	in *spb.DefineRevisionsRequest) (*spb.DefineRevisionsResponse, error)

DefineRevisions examines the outstanding mutations and returns a list of outstanding revisions that have not been applied.

func (*Server) HighWatermarks

func (s *Server) HighWatermarks(ctx context.Context, directoryID string, lastMeta *spb.MapMetadata,
	batchSize int32) (int32, *spb.MapMetadata, error)

HighWatermarks returns the total count across all logs and the highest watermark for each log. batchSize is a limit on the total number of items represented by the returned watermarks. TODO(gbelvin): Block until a minBatchSize has been reached or a timeout has occurred.

func (*Server) PublishRevisions

func (s *Server) PublishRevisions(ctx context.Context,
	in *spb.PublishRevisionsRequest) (*spb.PublishRevisionsResponse, error)

PublishRevisions copies the MapRoots of all known map revisions into the Log of MapRoots.

func (*Server) RunBatch

func (s *Server) RunBatch(ctx context.Context, in *spb.RunBatchRequest) (*empty.Empty, error)

RunBatch runs the full sequence of steps (for one directory) nessesary to get a mutation from the log integrated into the map. This consists of a series of idempotent steps: a) assign a batch of mutations from the logs to a map revision b) apply the batch to the map c) publish existing map roots to a log of SignedMapRoots.

type Trillian

type Trillian struct {
	// contains filtered or unexported fields
}

Trillian contains Trillian gRPC clients and metadata about them.

func (*Trillian) LogClient

func (t *Trillian) LogClient(ctx context.Context, dirID string) (trillianLog, error)

LogClient returns a verifying LogClient.

func (*Trillian) MapClient

func (t *Trillian) MapClient(ctx context.Context, dirID string) (trillianMap, error)

MapClient returns a verifying MapClient

type Watermarks

type Watermarks map[int64]int64

Watermarks is a map of watermarks by logID.