kdiff

README

kdiff

Simple binary diff library implemented using suffixarray index. Creates diff between two files and can be used to patch base file into new file. In my case I needed to hold base file (and its index) in memory to speed up processing of all subsequent new files. This is the reason for not using already existing solutions.

This project was inspired by go-bsdiff.

Features

• Contains siple CLI tools (kdiff, kpatch).
• Patch file can be serialized into protobuf.
• Patched output is validated by using sha256 hash sum.
• Base file can be stored in prepared form in memory. Recycling it can save most of processing.

Limitations

• High memory usage, therefore not suitable for bigger files (>1 GiB).
• In some instances for small files (<100B) can produce diffs that are bigger.
• Currently single-threaded.

How to use as tools

Installation

go install github.com/MartinKuzma/kdiff/cmd/kdiff@latest
go install github.com/MartinKuzma/kdiff/cmd/kpatch@latest

Usage

# create patch
kdiff file-old.exe file-new.exe file.patch
# apply patch
kpatch file-old.exe file.patch reconstructed-file.exe

How to use as a library

import "github.com/MartinKuzma/kdiff/pkg/kdiff"

// Process base file into indexed form.
base := kdiff.NewBase(baseFile)
// Create diff with 64KiB buffer for reading new file.
df, err := kdiff.DiffReader(base, newFile, 64*1024)
if err != nil {
  panic(err)
}

outputWriter, err := os.Create("outputfile")
if err != nil {
  panic(err)
}
// Create original file using base file from diff.
err = df.Patch(baseFile, outputWriter)
if err != nil {
  panic(err)
}