internal

Documentation

Index

• Constants
• type Agent
• type InfoProvider
• type Query
• type Scheduler
• type TableExecutor

Constants

const (
	// CheckpointCannotProceed is a placeholder indicating that the
	// Owner should not advance the global checkpoint TS just yet.
	CheckpointCannotProceed = model.Ts(0)
)

Types

type Agent

type Agent interface {
	// Tick is called periodically by the processor to drive the Agent's internal logic.
	Tick(context.Context) (*schedulepb.Barrier, error)

	// Close closes the messenger and does the necessary cleanup.
	Close() error
}

Agent is an interface for an object inside Processor that is responsible for receiving commands from the Owner. Ideally the processor should drive the Agent by Tick.

Note that Agent is not thread-safe

type InfoProvider

type InfoProvider interface {
	// IsInitialized returns a boolean indicates whether the scheduler is
	// initialized.
	IsInitialized() bool

	// GetTaskStatuses returns the task statuses.
	GetTaskStatuses() (map[model.CaptureID]*model.TaskStatus, error)
}

InfoProvider is the interface to get information about the internal states of the scheduler. We need this interface so that we can provide the information through HTTP API.

type Query

type Query struct {
	CaptureID model.CaptureID

	Resp interface{}
}

Query is for scheduler related owner job. at the moment, only for `DrainCapture`, we can use this to handle all manual schedule task. TODO: refactor `MoveTable` use Query to access the scheduler

type Scheduler

type Scheduler interface {
	// Tick is called periodically from the owner, and returns
	// updated global watermarks.
	// It is not thread-safe.
	Tick(
		ctx context.Context,

		checkpointTs model.Ts,

		currentTables []model.TableID,

		aliveCaptures map[model.CaptureID]*model.CaptureInfo,

		barrier *schedulepb.BarrierWithMinTs,
	) (watermark schedulepb.Watermark, err error)

	// MoveTable requests that a table be moved to target.
	// It is thread-safe.
	MoveTable(tableID model.TableID, target model.CaptureID)

	// Rebalance triggers a rebalance operation.
	// It is thread-safe
	Rebalance()

	// DrainCapture is used to drop all tables situated at the target capture
	// It is thread-safe.
	DrainCapture(target model.CaptureID) (int, error)

	// Close scheduler and release resource.
	// It is not thread-safe.
	Close(ctx context.Context)
}

Scheduler is an interface for scheduling tables. Since in our design, we do not record checkpoints per table, how we calculate the global watermarks (checkpoint-ts and resolved-ts) is heavily coupled with how tables are scheduled. That is why we have a scheduler interface that also reports the global watermarks.

type TableExecutor

type TableExecutor interface {
	// AddTableSpan add a new table span with `Checkpoint.CheckpointTs`
	// if `isPrepare` is true, the 1st phase of the 2 phase scheduling protocol.
	// if `isPrepare` is false, the 2nd phase.
	AddTableSpan(
		ctx context.Context, span tablepb.Span, checkpoint tablepb.Checkpoint, isPrepare bool,
	) (done bool, err error)

	// IsAddTableSpanFinished make sure the requested table span is in the proper status
	IsAddTableSpanFinished(span tablepb.Span, isPrepare bool) (done bool)

	// RemoveTableSpan remove the table, return true if the table is already removed
	RemoveTableSpan(span tablepb.Span) (done bool)
	// IsRemoveTableSpanFinished convince the table is fully stopped.
	// return false if table is not stopped
	// return true and corresponding checkpoint otherwise.
	IsRemoveTableSpanFinished(span tablepb.Span) (model.Ts, bool)

	// GetTableSpanStatus return the checkpoint and resolved ts for the given table span.
	GetTableSpanStatus(span tablepb.Span, collectStat bool) tablepb.TableStatus
}

TableExecutor is an abstraction for "Processor".

This interface is so designed that it would be the least problematic to adapt the current Processor implementation to it. TODO find a way to make the semantics easier to understand.