rdiff

README

RDiff

Package rdiff provides file diff between a source and a target, expressed as a collection of operations, to be applied to the target in order to update its content to match the source.

Installation:

go get github.com/silviutanasa/rdiff

Note that the minimum supported version is Go v1.21.

Usage:

package main

import (
	"encoding/gob"
	"fmt"
	"log"
	"os"

	"github.com/silviutanasa/rdiff"
)

func main() {
	// test_target and test_source must exist
	// test_signature and test_delta must not exist
	app := rdiff.New(5)
	err := app.Signature("test_target", "test_signature")
	if err != nil {
		log.Fatal(err)
	}

	err = app.Delta("test_signature", "test_source", "test_delta")
	if err != nil {
		log.Fatal(err)
	}

	// inspect the delta output
	delta, err := os.Open("test_delta")
	if err != nil {
		log.Fatal(err)
	}
	defer delta.Close()

	var ops []rdiff.Operation
	gob.NewDecoder(delta).Decode(&ops)
	// the delta should be a []Operation
	// where the Operation is defined as follows:
	// type Operation struct {
	//	 Type       OpType
	//   // the index of the block from the target, for OpBlockNew -1 is used to enforce that the BlockIndex is not important in this case
	//	 BlockIndex int
	//   // additional literal data if the block was modified, or a new block if the Block was not matched (BlockIndex == 0)
	//	 Data       []byte
	// }
	// where the operations are described as follows: 
	// OpBlockKeep
	// OpBlockUpdate
	// OpBlockRemove means there is no match for a target block in the source
	// OpBlockRemove
	// OpBlockNew (as a convention BlockIndex will be -1, in this case, indicating that it has no purpose)
	fmt.Println(ops)
}

Documentation

Overview

Package rdiff provides file diff between a source and a target, expressed as a collection of operations to be applied to the target in order to update its content to match the source.

The public API exposes 3 operations: New, Signature and Delta

	// usage example:
	//
	// creates a new instance with a block size of 3 bytes.
	rd := rdiff.New(3)
    // target_file_path must exist prior to this call
	// signature_file_path must not exist prior to this call
	// signature_file_path content will be serialized using gob encoding
	err := rd.Signature("target_file_path", "signature_file_path")
	if err != nil {
		return err
	}
	// signature_file_path must exist prior to this call
	// source_file_path must exist prior to this call
	// delta_file_path must not exist prior to this call
	// delta_file_path content will be serialized using gob encoding
	err = rd.Delta("signature_file_path", "source_file_path", "delta_file_path")
	if err != nil {
		...
	}
	...

Index

• Constants
• type App
  • func New(blockSize int) *App
  • func (a *App) Delta(signatureFilePath string, sourceFilePath string, deltaFilePath string) error
  • func (a *App) Signature(targetFilePath string, signatureFilePath string) error
• type Block
• type OpType
• type Operation

Examples

• New

Constants

const (
	// DefaultBlockSize is the default block size value, in bytes, used by the system
	DefaultBlockSize = 700
	// MaxBlockSize is the max block size value, in bytes, used by the system
	MaxBlockSize = 1 << 17
)

const M = 65521

M is the modulo for the Adler32 hash computation

Types

type App

type App struct {
	// contains filtered or unexported fields
}

App is the application layer of the RDiff service. It exposes the public API and allows for IO interactions.

func New

func New(blockSize int) *App

New constructs the RDiff app instance and returns a pointer to it. It accepts a blockSize as input, representing the size, in bytes, for splitting the target in blocks, in order to compute the target's signature. A blockSize <=0 means the size, in bytes, ii computed dynamically.

Example

package main

import (
	"encoding/gob"
	"fmt"
	"log"
	"os"

	"github.com/silviutanasa/rdiff"
)

func main() {
	// first create both target and source files
	err := os.WriteFile("test_source.bin", []byte{12, 32, 1, 2, 3, 4, 5, 6, 7, 8}, 0666)
	defer os.Remove("test_source.bin")
	if err != nil {
		log.Fatal(err)
	}
	err = os.WriteFile("test_target.bin", []byte{1, 2, 3, 4, 5, 6, 7}, 0666)
	defer os.Remove("test_target.bin")
	if err != nil {
		log.Fatal(err)
	}

	// second process the Signature and then the Delta
	app := rdiff.New(3)
	err = app.Signature("test_target.bin", "test_signature")
	defer os.Remove("test_signature")
	if err != nil {
		log.Fatal(err)
	}

	err = app.Delta("test_signature", "test_source.bin", "test_delta")
	if err != nil {
		log.Fatal(err)
	}

	// third inspect the Delta for results
	delta, err := os.Open("test_delta")
	if err != nil {
		log.Fatal(err)
	}
	defer os.Remove("test_delta")
	defer delta.Close()

	var ops []rdiff.Operation
	gob.NewDecoder(delta).Decode(&ops)
	fmt.Println(ops)

}

Output:

[{1 0 [12 32]} {0 1 []} {2 2 []} {3 -1 [7 8]}]

func (*App) Delta

func (a *App) Delta(signatureFilePath string, sourceFilePath string, deltaFilePath string) error

Delta computes the instruction list(operations list) in order for the target to be able to update its content to match the source. The signature file(signatureFilePath) and the source file(sourceFilePath) must exist, otherwise a non-nil error is returned. The delta file(deltaFilePath) must not exist, otherwise a non-nil error is returned. The content written to deltaFilePath is serialized using gob encoding.

func (*App) Signature

func (a *App) Signature(targetFilePath string, signatureFilePath string) error

Signature computes the signature of a target file(targetFilePath) and writes it to an output file(outputFilePath) The target file(targetFileName) must exist, otherwise it returns an appropriate non-nil error. If the output file(outputFilePath) already exists, it returns an appropriate non-nil error. The content written to outputFilePath is serialized using gob encoding.

type Block

type Block struct {
	StrongHash []byte
	WeakHash   uint32
}

Block represents a chunk of data(bytes) used by the target to split its data.

type OpType

type OpType byte

OpType represents a block operation/instruction, useful to recompute the target, based on source.

const (
	// OpBlockKeep means a block is unchanged and should be kept as it is.
	OpBlockKeep OpType = iota
	// OpBlockUpdate means there is a match in the target, and it contains extra data(literal) compared to the initial block
	OpBlockUpdate
	// OpBlockRemove means there is no match for a target block in the source
	OpBlockRemove
	// OpBlockNew means there is a literal block in the source that doesn't have any match in the target - new data
	OpBlockNew
)

type Operation

type Operation struct {
	Type OpType
	// the index of the block from the target, for OpBlockNew -1 is used to enforce that the BlockIndex
	// is not important in this case
	BlockIndex int
	// additional literal data, if the block was modified, or a new block if the Block was not matched (BlockIndex == 0)
	Data []byte
}

Operation represents an instruction given by the source to the target, in order to allow the target to update its content.