delta

README

go-delta-sib - A Go package and Custom library for generating delta using snappy compression

Suggestions:

• Works best on text files, database dumps and any other files with lots of repeating patterns and few changes between updates.

• Don't compress bytes returned by Delta.Bytes() because they are already compressed using Snappy compression.

Demonstration:

package main

import (
    "fmt"
    "github.com/DTSL/go-delta-sib"
)

func main() {
    fmt.Print("Binary delta update demo:\n\n")

    // The original data (20 bytes):
    var source = []byte("quick brown fox, lazy dog, and five boxing wizards")
    fmt.Print("The original is:", "\n", string(source), "\n\n")

    // The updated data containing the original and new content (82 bytes):
    var target = []byte(
        "The quick brown fox jumps over the lazy dog. " +
        "The five boxing wizards jump quickly.",
    )
    fmt.Print("The update is:", "\n", string(target), "\n\n")

    var dbytes []byte
    {
    	// Use Make() to generate a compressed patch from source and target
    	var d = delta.Make(source, target)
    	
    	// Convert the delta to a slice of bytes (e.g. for writing to a file)
    	dbytes = d.Bytes()
    }

    // Create a Delta from the byte slice
    var d = delta.Load(dbytes)

    // Apply the patch to source to get the target
    // The size of the patch is much shorter than target.
    var target2, err = d.Apply(source)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Print("Patched:", "\n", string(target2), "\n\n")
} //                                                                        main

Documentation

Index

• Constants
• Variables
• func GetBufPool() *bytes.Buffer
• func PutBufPool(buf *bytes.Buffer)
• func SetErrorFunc(fn func(args ...interface{}) error)
• type Delta
  • func Load(data []byte, compressionAlgo string) (Delta, error)
  • func Make(a, b []byte, compressionAlgo string) Delta
  • func (ob *Delta) Apply(source []byte) ([]byte, error)
  • func (ob *Delta) Bytes() []byte
  • func (ob *Delta) Dump()
  • func (ob Delta) GoString() string
  • func (ob *Delta) NewCount() int
  • func (ob *Delta) OldCount() int
  • func (ob *Delta) SourceSize() int
  • func (ob *Delta) TargetSize() int
• type Pool
  • func NewBufPool() *Pool
  • func (p *Pool) Get() *bytes.Buffer
  • func (p *Pool) Put(buf *bytes.Buffer)

Constants

const (
	// MatchLimit specifies the maximum number of positions tracked
	// for each unique key in the map of source data. See makeMap().
	MatchLimit = 50

	// MatchSize specifies the size of unique
	// chunks being searched for, in bytes.
	MatchSize = 9
)

Variables

var (
	// DebugInfo when set, causes printing of messages helpful for debugging.
	DebugInfo = false

	// DebugTiming controls timing (benchmarking) of time spent in each function.
	DebugTiming = false

	// DebugWriteArgs when set, prints the arguments passed to write()
	DebugWriteArgs = false
)

var TempBufferSize = 16 * 1024 * 1024 // 32 MB

Functions

func GetBufPool

func GetBufPool() *bytes.Buffer

GetBufPool Get calls Get on the default Pool.

func PutBufPool

func PutBufPool(buf *bytes.Buffer)

PutBufPool Put calls Put on the default Pool.

func SetErrorFunc

func SetErrorFunc(fn func(args ...interface{}) error)

SetErrorFunc changes the error-handling function, so that all errors in this package will be sent to this handler, which is useful for custom logging and mocking during unit tests. To restore the default error handler use SetErrorFunc(nil).

Types

type Delta

type Delta struct {
	// contains filtered or unexported fields

} //                                                                       Delta

Delta stores the binary delta difference between two byte arrays

func Load

func Load(data []byte, compressionAlgo string) (Delta, error)

Load fills a new Delta structure from a byte array previously returned by Delta.Bytes().

func Make

func Make(a, b []byte, compressionAlgo string) Delta

func (*Delta) Apply

func (ob *Delta) Apply(source []byte) ([]byte, error)

Apply uses the 'source' byte array, applies this Delta to it and returns the updated byte array. If this delta was not generated from source, returns an error.

func (*Delta) Bytes

func (ob *Delta) Bytes() []byte

Bytes converts the Delta structure to a byte array (for serializing to a file, etc.)

func (*Delta) Dump

func (ob *Delta) Dump()

Dump prints this object to the console in a human-friendly format.

func (Delta) GoString

func (ob Delta) GoString() string

GoString returns a Go-syntax representation of the Delta structure. It implements the fmt.GoStringer interface.

func (*Delta) NewCount

func (ob *Delta) NewCount() int

NewCount returns the number of chunks not matched in source array.

func (*Delta) OldCount

func (ob *Delta) OldCount() int

OldCount returns the number of matched chunks in source array.

func (*Delta) SourceSize

func (ob *Delta) SourceSize() int

SourceSize returns the size of the source byte array, in bytes.

func (*Delta) TargetSize

func (ob *Delta) TargetSize() int

TargetSize returns the size of the target byte array, in bytes.

type Pool

type Pool struct {

	// MaxCap defines the maximum capacity accepted for recycled buffer.
	// If Put() is called with a buffer larger than this value, it's discarded.
	// See https://github.com/golang/go/issues/23199 .
	// 0 means there is no maximum capacity.
	MaxCap int
	// contains filtered or unexported fields
}

Pool is a pool of *bytes.Buffer.

func NewBufPool

func NewBufPool() *Pool

NewBufPool New returns a new Pool.

func (*Pool) Get

func (p *Pool) Get() *bytes.Buffer

Get gets a buffer from the Pool, resets it and returns it.

func (*Pool) Put

func (p *Pool) Put(buf *bytes.Buffer)

Put puts the buffer to the Pool. WARNING: the call MUST NOT reuse the buffer's content after this call.