rollsum

Documentation

Overview

rollsum provides an implementation of a rolling checksum - a checksum that's efficient to advance a byte or more at a time. It is inspired by the rollsum in rsync, but differs in that the internal values used are 32bit integers - to make a conformant implementation, a find a replace on "32" should be almost sufficient (although it would be highly recommended to test against known values from the original implementation).

Rollsum32 supports the hash.Hash implementation, but is not used much in go-sync, mostly in order to share and access the underlying circular buffer storage, and use the implementation as efficiently as possible.

Index

• Constants
• type Rollsum32
  • func NewRollsum32(blocksize uint) *Rollsum32
  • func (r *Rollsum32) BlockSize() int
  • func (r *Rollsum32) GetLastBlock() []byte
  • func (r *Rollsum32) Reset()
  • func (r *Rollsum32) Size() int
  • func (r *Rollsum32) Sum(b []byte) []byte
  • func (r *Rollsum32) Write(p []byte) (n int, err error)
• type Rollsum32Base
  • func NewRollsum32Base(blockSize uint) *Rollsum32Base
  • func (r *Rollsum32Base) AddAndRemoveBytes(add []byte, remove []byte, length int)
  • func (r *Rollsum32Base) AddByte(b byte)
  • func (r *Rollsum32Base) AddBytes(bs []byte)
  • func (r *Rollsum32Base) GetSum(b []byte)
  • func (r *Rollsum32Base) RemoveByte(b byte, length int)
  • func (r *Rollsum32Base) RemoveBytes(bs []byte, length int)
  • func (r *Rollsum32Base) Reset()
  • func (r *Rollsum32Base) SetBlock(block []byte)
  • func (r *Rollsum32Base) Size() int

Constants

const FULL_BYTES_16 = (1 << 16) - 1

Types

type Rollsum32

type Rollsum32 struct {
	Rollsum32Base
	// contains filtered or unexported fields
}

Rollsum32 is a rolling checksum implemenation inspired by rsync, but with 32bit internal values Create one using NewRollsum32

func NewRollsum32

func NewRollsum32(blocksize uint) *Rollsum32

func (*Rollsum32) BlockSize

func (r *Rollsum32) BlockSize() int

The most efficient byte length to call Write with

func (*Rollsum32) GetLastBlock

func (r *Rollsum32) GetLastBlock() []byte

func (*Rollsum32) Reset

func (r *Rollsum32) Reset()

func (*Rollsum32) Size

func (r *Rollsum32) Size() int

the number of bytes

func (*Rollsum32) Sum

func (r *Rollsum32) Sum(b []byte) []byte

Sum appends the current hash to b and returns the resulting slice. It does not change the underlying hash state. Note that this is to allow Sum() to reuse a preallocated buffer

func (*Rollsum32) Write

func (r *Rollsum32) Write(p []byte) (n int, err error)

cannot be called concurrently

type Rollsum32Base

type Rollsum32Base struct {
	// contains filtered or unexported fields
}

The specification of hash.Hash is such that it cannot be implemented without implementing storage but the most optimal storage scheme depends on usage of the circular buffer & hash

func NewRollsum32Base

func NewRollsum32Base(blockSize uint) *Rollsum32Base

Rollsum32Base decouples the rollsum algorithm from the implementation of hash.Hash and the storage the rolling checksum window this allows us to write different versions of the storage for the distinctly different use-cases and optimize the storage with the usage pattern.

func (*Rollsum32Base) AddAndRemoveBytes

func (r *Rollsum32Base) AddAndRemoveBytes(add []byte, remove []byte, length int)

func (*Rollsum32Base) AddByte

func (r *Rollsum32Base) AddByte(b byte)

Add a single byte into the rollsum

func (*Rollsum32Base) AddBytes

func (r *Rollsum32Base) AddBytes(bs []byte)

func (*Rollsum32Base) GetSum

func (r *Rollsum32Base) GetSum(b []byte)

Puts the sum into b. Avoids allocation. b must have length >= 4

func (*Rollsum32Base) RemoveByte

func (r *Rollsum32Base) RemoveByte(b byte, length int)

Remove a byte from the end of the rollsum Use the previous length (before removal)

func (*Rollsum32Base) RemoveBytes

func (r *Rollsum32Base) RemoveBytes(bs []byte, length int)

func (*Rollsum32Base) Reset

func (r *Rollsum32Base) Reset()

Reset the hash to the initial state

func (*Rollsum32Base) SetBlock

func (r *Rollsum32Base) SetBlock(block []byte)

Set a whole block of blockSize

func (*Rollsum32Base) Size

func (r *Rollsum32Base) Size() int

size of the hash in bytes