comparer

Documentation

Overview

package comparer is responsible for using a FileChecksumGenerator (filechecksum) and an index to move through a file and compare it to the index, producing a FileDiffSummary

Index

• Constants
• type BlockBuffer
• type BlockMatchResult
• type BlockSpan
  • func (b BlockSpan) EndOffset(blockSize int64) int64
• type BlockSpanEnd
  • func (s BlockSpanEnd) Less(than llrb.Item) bool
  • func (s BlockSpanEnd) Position() uint
• type BlockSpanIndex
• type BlockSpanKey
  • func (k BlockSpanKey) Less(than llrb.Item) bool
  • func (s BlockSpanKey) Position() uint
• type BlockSpanList
  • func (l BlockSpanList) GetMissingBlocks(maxBlock uint) (sorted BlockSpanList)
  • func (l BlockSpanList) Len() int
  • func (l BlockSpanList) Less(i, j int) bool
  • func (l BlockSpanList) Swap(i, j int)
• type BlockSpanStart
  • func (s BlockSpanStart) Less(than llrb.Item) bool
  • func (s BlockSpanStart) Position() uint
• type Comparer
  • func (c *Comparer) StartFindMatchingBlocks(comparison io.Reader, baseOffset int64, ...) <-chan BlockMatchResult
• type Index
• type MatchMerger
  • func (merger *MatchMerger) GetMergedBlocks() (sorted BlockSpanList)
  • func (merger *MatchMerger) StartMergeResultStream(resultStream <-chan BlockMatchResult, blockSize int64)

Constants

const (
	READ_NEXT_BYTE = iota
	READ_NEXT_BLOCK
	READ_NONE
)

Types

type BlockBuffer

type BlockBuffer interface {
	Write([]byte) (int, error)
	// the last set of bytes of the size of the circular buffer
	// oldest to newest
	GetLastBlock() []byte
}

If the weak Hash object satisfies this interface, then StartFindMatchingBlocks will not allocate a circular buffer

type BlockMatchResult

type BlockMatchResult struct {
	// In case of error
	Err error

	// The offset the comparison + baseOffset
	ComparisonOffset int64

	// The block from the index that it matched
	BlockIdx uint
}

type BlockSpan

type BlockSpan struct {
	StartBlock uint
	EndBlock   uint

	// byte offset in the comparison for the match
	ComparisonStartOffset int64
}

a span of multiple blocks, from start to end, which match the blocks starting at an offset of ComparisonStartOffset

func (BlockSpan) EndOffset

func (b BlockSpan) EndOffset(blockSize int64) int64

type BlockSpanEnd

type BlockSpanEnd BlockSpan

Wraps a blockspan so that it may easily be used in llrb. Corresponds to the end of the blockspan.

func (BlockSpanEnd) Less

func (s BlockSpanEnd) Less(than llrb.Item) bool

func (BlockSpanEnd) Position

func (s BlockSpanEnd) Position() uint

type BlockSpanIndex

type BlockSpanIndex interface {
	Position() uint
}

type BlockSpanKey

type BlockSpanKey uint

Wraps a block index, allowing easy use of llrb.Get()

func (BlockSpanKey) Less

func (k BlockSpanKey) Less(than llrb.Item) bool

func (BlockSpanKey) Position

func (s BlockSpanKey) Position() uint

type BlockSpanList

type BlockSpanList []BlockSpan

func (BlockSpanList) GetMissingBlocks

func (l BlockSpanList) GetMissingBlocks(maxBlock uint) (sorted BlockSpanList)

Creates a list of spans that are missing. note that maxBlock is blockCount-1

func (BlockSpanList) Len

func (l BlockSpanList) Len() int

func (BlockSpanList) Less

func (l BlockSpanList) Less(i, j int) bool

func (BlockSpanList) Swap

func (l BlockSpanList) Swap(i, j int)

type BlockSpanStart

type BlockSpanStart BlockSpan

Wraps a blockspan so that it may easily be used in llrb. Corresponds to the start block of the blockspan.

func (BlockSpanStart) Less

func (s BlockSpanStart) Less(than llrb.Item) bool

func (BlockSpanStart) Position

func (s BlockSpanStart) Position() uint

type Comparer

type Comparer struct {
	Comparisons    int64
	WeakHashHits   int64
	StrongHashHits int64
}

func (*Comparer) StartFindMatchingBlocks

func (c *Comparer) StartFindMatchingBlocks(
	comparison io.Reader,
	baseOffset int64,
	generator *filechecksum.FileChecksumGenerator,
	referenceIndex Index,
) <-chan BlockMatchResult

type Index

type Index interface {
	FindWeakChecksum2(chk []byte) interface{}
	FindStrongChecksum2(chk []byte, weak interface{}) []chunks.ChunkChecksum
}

type MatchMerger

type MatchMerger struct {
	sync.Mutex
	// contains filtered or unexported fields
}

The result merger takes many BlockMatchResult and combines adjoining results into spans of blocks

func (*MatchMerger) GetMergedBlocks

func (merger *MatchMerger) GetMergedBlocks() (sorted BlockSpanList)

Sorted list of blocks, based on StartBlock

func (*MatchMerger) StartMergeResultStream

func (merger *MatchMerger) StartMergeResultStream(
	resultStream <-chan BlockMatchResult,
	blockSize int64,
)

Can be used on multiple streams of results simultaneously starts working asyncronously, call from the initiating goroutine