errormanager

Documentation

Index

• Constants
• type DataConflictInfo
• type ErrorManager
  • func New(db *sql.DB, cfg *config.Config, logger log.Logger) *ErrorManager
  • func (em *ErrorManager) ConflictErrorsRemain() int64
  • func (em *ErrorManager) ConflictRecordsRemain() int64
  • func (em *ErrorManager) HasError() bool
  • func (em *ErrorManager) Init(ctx context.Context) error
  • func (em *ErrorManager) LogErrorDetails()
  • func (em *ErrorManager) Output() string
  • func (em *ErrorManager) RecordDataConflictError(ctx context.Context, logger log.Logger, tableName string, ...) error
  • func (em *ErrorManager) RecordDuplicate(ctx context.Context, logger log.Logger, tableName string, path string, ...) error
  • func (em *ErrorManager) RecordDuplicateCount(cnt int64) error
  • func (em *ErrorManager) RecordDuplicateOnce(ctx context.Context, logger log.Logger, tableName string, path string, ...)
  • func (em *ErrorManager) RecordErrorOnce() bool
  • func (em *ErrorManager) RecordIndexConflictError(ctx context.Context, logger log.Logger, tableName string, indexNames []string, ...) error
  • func (em *ErrorManager) RecordTypeError(ctx context.Context, logger log.Logger, tableName string, path string, ...) error
  • func (em *ErrorManager) ReplaceConflictKeys(ctx context.Context, tbl tidbtbl.Table, tableName string, ...) error
  • func (em *ErrorManager) TypeErrorsRemain() int64

Constants

const (

	// ConflictErrorTableName is the table name for duplicate detection.
	ConflictErrorTableName = "conflict_error_v3"
	// DupRecordTableName is the table name to record duplicate data that displayed to user.
	DupRecordTableName = "conflict_records"
	// ConflictViewName is the view name for presenting the union information of ConflictErrorTable and DupRecordTable.
	ConflictViewName = "conflict_view"
)

Types

type DataConflictInfo

type DataConflictInfo struct {
	RawKey   []byte
	RawValue []byte
	KeyData  string
	Row      string
}

DataConflictInfo is the information of a data conflict error.

type ErrorManager

type ErrorManager struct {
	// contains filtered or unexported fields
}

ErrorManager records errors during the import process.

func New

func New(db *sql.DB, cfg *config.Config, logger log.Logger) *ErrorManager

New creates a new error manager.

func (*ErrorManager) ConflictErrorsRemain

func (em *ErrorManager) ConflictErrorsRemain() int64

ConflictErrorsRemain returns the number of conflict errors that can be recorded.

func (*ErrorManager) ConflictRecordsRemain

func (em *ErrorManager) ConflictRecordsRemain() int64

ConflictRecordsRemain returns the number of errors that need be recorded.

func (*ErrorManager) HasError

func (em *ErrorManager) HasError() bool

HasError returns true if any error type has reached the limit

func (*ErrorManager) Init

func (em *ErrorManager) Init(ctx context.Context) error

Init creates the schemas and tables to store the task information.

func (*ErrorManager) LogErrorDetails

func (em *ErrorManager) LogErrorDetails()

LogErrorDetails return a slice of zap.Field for each error type

func (*ErrorManager) Output

func (em *ErrorManager) Output() string

Output renders a table which contains error summery for each error type.

func (*ErrorManager) RecordDataConflictError

func (em *ErrorManager) RecordDataConflictError(
	ctx context.Context,
	logger log.Logger,
	tableName string,
	conflictInfos []DataConflictInfo,
) error

RecordDataConflictError records a data conflict error.

func (*ErrorManager) RecordDuplicate

func (em *ErrorManager) RecordDuplicate(
	ctx context.Context,
	logger log.Logger,
	tableName string,
	path string,
	offset int64,
	errMsg string,
	rowID int64,
	rowData string,
) error

RecordDuplicate records a "duplicate entry" error so user can query them later. Currently, the error will not be shared for multiple lightning instances.

func (*ErrorManager) RecordDuplicateCount

func (em *ErrorManager) RecordDuplicateCount(cnt int64) error

RecordDuplicateCount reduce the counter of "duplicate entry" errors. Currently, the count will not be shared for multiple lightning instances.

func (*ErrorManager) RecordDuplicateOnce

func (em *ErrorManager) RecordDuplicateOnce(
	ctx context.Context,
	logger log.Logger,
	tableName string,
	path string,
	offset int64,
	errMsg string,
	rowID int64,
	rowData string,
)

RecordDuplicateOnce records a "duplicate entry" error so user can query them later. Currently the error will not be shared for multiple lightning instances. Different from RecordDuplicate, this function is used when conflict.strategy is "error" and will only write the first conflict error to the table.

func (*ErrorManager) RecordErrorOnce

func (em *ErrorManager) RecordErrorOnce() bool

RecordErrorOnce returns if RecordDuplicateOnce has been called. Not that this method is not atomic with RecordDuplicateOnce.

func (*ErrorManager) RecordIndexConflictError

func (em *ErrorManager) RecordIndexConflictError(
	ctx context.Context,
	logger log.Logger,
	tableName string,
	indexNames []string,
	conflictInfos []DataConflictInfo,
	rawHandles, rawRows [][]byte,
) error

RecordIndexConflictError records a index conflict error.

func (*ErrorManager) RecordTypeError

func (em *ErrorManager) RecordTypeError(
	ctx context.Context,
	logger log.Logger,
	tableName string,
	path string,
	offset int64,
	rowText string,
	encodeErr error,
) error

RecordTypeError records a type error. If the number of recorded type errors exceed the max-error count, also returns `err` directly.

func (*ErrorManager) ReplaceConflictKeys

func (em *ErrorManager) ReplaceConflictKeys(
	ctx context.Context,
	tbl tidbtbl.Table,
	tableName string,
	pool *util.WorkerPool,
	fnGetLatest func(ctx context.Context, key []byte) ([]byte, error),
	fnDeleteKeys func(ctx context.Context, key [][]byte) error,
) error

ReplaceConflictKeys query all conflicting rows (handle and their values) from the current error report and resolve them by replacing the necessary rows and reserving the others.

func (*ErrorManager) TypeErrorsRemain

func (em *ErrorManager) TypeErrorsRemain() int64

TypeErrorsRemain returns the number of type errors that can be recorded.