gsync

README

gSync

A Go library implementing rsync algorithm described at https://www.samba.org/~tridge/phd_thesis.pdf and in more detail at http://paperswelove.org/2017/video/camilo-aguilar-rsync-algorithm/.

Documentation

Overview

Package gsync implements a rsync-based algorithm for sending delta updates to a remote server.

Index

• Constants
• func Apply(ctx context.Context, dst io.Writer, cache io.ReaderAt, ...) error
• func LookUpTable(ctx context.Context, bc <-chan BlockSignature) (map[uint32][]BlockSignature, error)
• func Signatures(ctx context.Context, r io.Reader, shash hash.Hash) (<-chan BlockSignature, error)
• func Sync(ctx context.Context, r io.ReaderAt, shash hash.Hash, ...) (<-chan BlockOperation, error)
• type BlockOperation
• type BlockSignature

Constants

const (
	// DefaultBlockSize is the default block size.
	DefaultBlockSize = 6 * 1024 // 6kb
)

Functions

func Apply

func Apply(ctx context.Context, dst io.Writer, cache io.ReaderAt, ops <-chan BlockOperation) error

Apply reconstructs a file given a set of operations. The caller must close the ops channel or the context when done or there will be a deadlock.

func LookUpTable

func LookUpTable(ctx context.Context, bc <-chan BlockSignature) (map[uint32][]BlockSignature, error)

LookUpTable reads up blocks signatures and builds a lookup table for the client to search from when trying to decide wether to send or not a block of data.

func Signatures

func Signatures(ctx context.Context, r io.Reader, shash hash.Hash) (<-chan BlockSignature, error)

Signatures reads data blocks from reader and pipes out block signatures on the returning channel, closing it when done reading or when the context is cancelled. This function does not block and returns immediately. The caller must make sure the concrete reader instance is not nil or this function will panic.

func Sync

func Sync(ctx context.Context, r io.ReaderAt, shash hash.Hash, remote map[uint32][]BlockSignature) (<-chan BlockOperation, error)

Sync sends tokens or literal bytes to the caller in order to efficiently re-construct a remote file. Whether to send tokens or literals is determined by the remote checksums provided by the caller. This function does not block and returns immediately. Also, the remote blocks map is accessed without a mutex, so this function is expected to be called once the remote blocks map is fully populated.

The caller must make sure the concrete reader instance is not nil or this function will panic.

Types

type BlockOperation

type BlockOperation struct {
	// Index is the block index involved.
	Index uint64
	// Data is the delta to be applied to the remote file. No data means
	// the client found a matching checksum for this block, which in turn means
	// the remote end proceeds to get the block data from its local
	// copy instead.
	Data []byte
	// Error is used to report any error while sending operations.
	Error error
}

BlockOperation represents a file re-construction instruction.

type BlockSignature

type BlockSignature struct {
	// Index is the block index
	Index uint64
	// Strong refers to the strong checksum, it need not to be cryptographic.
	Strong []byte
	// Weak refers to the fast rsync rolling checksum
	Weak uint32
	// Error is used to report the error reading the file or calculating checksums.
	Error error
}

BlockSignature contains file block index and checksums.