lightning

Documentation

Index

• func CheckpointRemove(ctx context.Context, cfg *config.Config, tableName string) error
• func CleanupMetas(ctx context.Context, cfg *config.Config, tableName string) error
• func SwitchMode(ctx context.Context, cfg *config.Config, tls *common.TLS, mode string) error
• type Lightning
  • func New(globalCfg *config.GlobalConfig) *Lightning
  • func (l *Lightning) GoServe() error
  • func (l *Lightning) Metrics() *metric.Metrics
  • func (l *Lightning) RunOnceWithOptions(taskCtx context.Context, taskCfg *config.Config, opts ...Option) error
  • func (l *Lightning) RunServer() error
  • func (l *Lightning) Status() (finished int64, total int64)
  • func (l *Lightning) Stop()
• type Option
  • func WithCheckpointStorage(s storage.ExternalStorage, cpName string) Option
  • func WithDumpFileStorage(s storage.ExternalStorage) Option
  • func WithDupIndicator(b *atomic.Bool) Option
  • func WithLogger(logger *zap.Logger) Option
  • func WithPromFactory(f promutil.Factory) Option
  • func WithPromRegistry(r promutil.Registry) Option

Functions

func CheckpointRemove

func CheckpointRemove(ctx context.Context, cfg *config.Config, tableName string) error

CheckpointRemove removes the checkpoint of the given table.

func CleanupMetas

func CleanupMetas(ctx context.Context, cfg *config.Config, tableName string) error

CleanupMetas removes the table metas of the given table.

func SwitchMode

func SwitchMode(ctx context.Context, cfg *config.Config, tls *common.TLS, mode string) error

SwitchMode switches the mode of the TiKV cluster.

Types

type Lightning

type Lightning struct {
	// contains filtered or unexported fields
}

Lightning is the main struct of the lightning package.

func New

func New(globalCfg *config.GlobalConfig) *Lightning

New creates a new Lightning instance.

func (*Lightning) GoServe

func (l *Lightning) GoServe() error

GoServe starts the HTTP server in a goroutine. The server will be closed

func (*Lightning) Metrics

func (l *Lightning) Metrics() *metric.Metrics

Metrics returns the metrics of lightning. it's inited during `run`, so might return nil.

func (*Lightning) RunOnceWithOptions

func (l *Lightning) RunOnceWithOptions(taskCtx context.Context, taskCfg *config.Config, opts ...Option) error

RunOnceWithOptions is used by binary lightning and host when using lightning as a library.

• for binary lightning, taskCtx could be context.Background which means taskCtx wouldn't be canceled directly by its cancel function, but only by Lightning.Stop or HTTP DELETE using l.cancel. No need to set Options
• for lightning as a library, taskCtx could be a meaningful context that get canceled outside, and there Options may be used:
• WithGlue: set a caller implemented glue. Otherwise, lightning will use a default glue later.
• WithDumpFileStorage: caller has opened an external storage for lightning. Otherwise, lightning will open a storage by config
• WithCheckpointStorage: caller has opened an external storage for lightning and want to save checkpoint in it. Otherwise, lightning will save checkpoint by the Checkpoint.DSN in config

func (*Lightning) RunServer

func (l *Lightning) RunServer() error

RunServer is used by binary lightning to start a HTTP server to receive import tasks.

func (*Lightning) Status

func (l *Lightning) Status() (finished int64, total int64)

Status return the sum size of file which has been imported to TiKV and the total size of source file.

func (*Lightning) Stop

func (l *Lightning) Stop()

Stop stops the lightning server.

type Option

type Option func(*options)

Option is a function that configures a lightning task.

func WithCheckpointStorage

func WithCheckpointStorage(s storage.ExternalStorage, cpName string) Option

WithCheckpointStorage sets the checkpoint name in external storage to a lightning task. Typically, the checkpoint name is set when lightning is integrated with dataflow engine by DM.

func WithDumpFileStorage

func WithDumpFileStorage(s storage.ExternalStorage) Option

WithDumpFileStorage sets the external storage to a lightning task. Typically, the external storage is set when lightning is integrated with dataflow engine by DM.

func WithDupIndicator

func WithDupIndicator(b *atomic.Bool) Option

WithDupIndicator sets a *bool to indicate duplicate detection has found duplicate data.

func WithLogger

func WithLogger(logger *zap.Logger) Option

WithLogger sets the logger to a lightning task.

func WithPromFactory

func WithPromFactory(f promutil.Factory) Option

WithPromFactory sets the prometheus factory to a lightning task.

func WithPromRegistry

func WithPromRegistry(r promutil.Registry) Option

WithPromRegistry sets the prometheus registry to a lightning task. The task metrics will be registered to the registry before the task starts and unregistered after the task ends.