checkpoints

Documentation

Index

• Constants
• type Checkpoint
  • func (cp *Checkpoint) GetChunkSnapshot() (cur *Node)
  • func (cp *Checkpoint) GetCurrentSavedID() *Node
  • func (cp *Checkpoint) Init()
  • func (cp *Checkpoint) InitCurrentSavedID(n *Node)
  • func (cp *Checkpoint) Insert(node *Node)
  • func (cp *Checkpoint) LoadChunk(fileName string) (*Node, *report.Report, error)
  • func (cp *Checkpoint) SaveChunk(ctx context.Context, fileName string, cur *Node, reportInfo *report.Report) (*chunk.ChunkID, error)
• type Node
  • func (n *Node) GetBucketIndexLeft() int
  • func (n *Node) GetBucketIndexRight() int
  • func (n *Node) GetChunkIndex() int
  • func (n *Node) GetID() *chunk.ChunkID
  • func (n *Node) GetState() string
  • func (n *Node) GetTableIndex() int
  • func (n *Node) IsAdjacent(next *Node) bool
  • func (n *Node) IsLess(next *Node) bool
• type SavedState

Constants

const (
	// SuccessState
	// for chunk: means this chunk's data is equal
	// for table: means this all chunk in this table is equal(except ignore chunk)
	SuccessState = "success"

	// FailedState
	// for chunk: means this chunk's data is not equal
	// for table: means some chunks' data is not equal or some chunk check failed in this table
	FailedState = "failed"

	// IgnoreState
	// for chunk: this chunk is ignored. if it is Empty chunk, will ignore some chunk
	// for table: don't have this state
	IgnoreState = "ignore"
)

Types

type Checkpoint

type Checkpoint struct {
	// contains filtered or unexported fields
}

Checkpoint provide the ability to restart the sync-diff process from the latest previous exit point (due to error or intention).

func (*Checkpoint) GetChunkSnapshot

func (cp *Checkpoint) GetChunkSnapshot() (cur *Node)

GetChunkSnapshot get the snapshot of the minimum continuous checked chunk

func (*Checkpoint) GetCurrentSavedID

func (cp *Checkpoint) GetCurrentSavedID() *Node

func (*Checkpoint) Init

func (cp *Checkpoint) Init()

func (*Checkpoint) InitCurrentSavedID

func (cp *Checkpoint) InitCurrentSavedID(n *Node)

InitCurrentSavedID the method is only used in initialization without lock, be cautious

func (*Checkpoint) Insert

func (cp *Checkpoint) Insert(node *Node)

func (*Checkpoint) LoadChunk

func (cp *Checkpoint) LoadChunk(fileName string) (*Node, *report.Report, error)

LoadChunk loads chunk info from file `chunk`

func (*Checkpoint) SaveChunk

func (cp *Checkpoint) SaveChunk(ctx context.Context, fileName string, cur *Node, reportInfo *report.Report) (*chunk.ChunkID, error)

SaveChunk saves the chunk to file.

type Node

type Node struct {
	State string `json:"state"` // indicate the state ("success" or "failed") of the chunk

	ChunkRange *chunk.Range `json:"chunk-range"`
	IndexID    int64        `json:"index-id"`
}

func (*Node) GetBucketIndexLeft

func (n *Node) GetBucketIndexLeft() int

func (*Node) GetBucketIndexRight

func (n *Node) GetBucketIndexRight() int

func (*Node) GetChunkIndex

func (n *Node) GetChunkIndex() int

func (*Node) GetID

func (n *Node) GetID() *chunk.ChunkID

func (*Node) GetState

func (n *Node) GetState() string

func (*Node) GetTableIndex

func (n *Node) GetTableIndex() int

func (*Node) IsAdjacent

func (n *Node) IsAdjacent(next *Node) bool

IsAdjacent represents whether the next node is adjacent node. it's the important logic for checkpoint update. we need keep this node save to checkpoint in global order.

func (*Node) IsLess

func (n *Node) IsLess(next *Node) bool

IsLess represents whether the cur node is less than next node.

type SavedState

type SavedState struct {
	Chunk  *Node          `json:"chunk-info"`
	Report *report.Report `json:"report-info"`
}

SaveState contains the information of the latest checked chunk and state of `report` When sync-diff start from the checkpoint, it will load this information and continue running