xxhash

README

xxhash

xxhash is a Go implementation of the 64-bit xxHash algorithm, XXH64. This is a high-quality hashing algorithm that is much faster than anything in the Go standard library.

This package provides a straightforward API:

func Sum64(b []byte) uint64
func Sum64String(s string) uint64
type Digest struct{ ... }
    func New() *Digest

The Digest type implements hash.Hash64. Its key methods are:

func (*Digest) Write([]byte) (int, error)
func (*Digest) WriteString(string) (int, error)
func (*Digest) Sum64() uint64

The package is written with optimized pure Go and also contains even faster assembly implementations for amd64 and arm64. If desired, the purego build tag opts into using the Go code even on those architectures.

Compatibility

This package is in a module and the latest code is in version 2 of the module. You need a version of Go with at least "minimal module compatibility" to use github.com/cespare/xxhash/v2:

• 1.9.7+ for Go 1.9
• 1.10.3+ for Go 1.10
• Go 1.11 or later

I recommend using the latest release of Go.

Benchmarks

Here are some quick benchmarks comparing the pure-Go and assembly implementations of Sum64.

input size	purego	asm
4 B	1.3 GB/s	1.2 GB/s
16 B	2.9 GB/s	3.5 GB/s
100 B	6.9 GB/s	8.1 GB/s
4 KB	11.7 GB/s	16.7 GB/s
10 MB	12.0 GB/s	17.3 GB/s

These numbers were generated on Ubuntu 20.04 with an Intel Xeon Platinum 8252C CPU using the following commands under Go 1.19.2:

benchstat <(go test -tags purego -benchtime 500ms -count 15 -bench 'Sum64$')
benchstat <(go test -benchtime 500ms -count 15 -bench 'Sum64$')

Projects using this package

• InfluxDB
• Prometheus
• VictoriaMetrics
• FreeCache
• FastCache
• Ristretto
• Badger

Documentation

Overview

Package xxhash implements the 64-bit variant of xxHash (XXH64) as described at http://cyan4973.github.io/xxHash/.

Index

• func Sum64(b []byte) uint64
• func Sum64String(s string) uint64
• type Digest
  • func New() *Digest
  • func NewWithSeed(seed uint64) *Digest
  • func (d *Digest) BlockSize() int
  • func (d *Digest) MarshalBinary() ([]byte, error)
  • func (d *Digest) Reset()
  • func (d *Digest) ResetWithSeed(seed uint64)
  • func (d *Digest) Size() int
  • func (d *Digest) Sum(b []byte) []byte
  • func (d *Digest) Sum64() uint64
  • func (d *Digest) UnmarshalBinary(b []byte) error
  • func (d *Digest) Write(b []byte) (n int, err error)
  • func (d *Digest) WriteString(s string) (n int, err error)

Functions

func Sum64

func Sum64(b []byte) uint64

Sum64 computes the 64-bit xxHash digest of b with a zero seed.

func Sum64String

func Sum64String(s string) uint64

Sum64String computes the 64-bit xxHash digest of s with a zero seed. It may be faster than Sum64([]byte(s)) by avoiding a copy.

Types

type Digest

type Digest struct {
	// contains filtered or unexported fields
}

Digest implements hash.Hash64.

Note that a zero-valued Digest is not ready to receive writes. Call Reset or create a Digest using New before calling other methods.

func New

func New() *Digest

New creates a new Digest with a zero seed.

func NewWithSeed

func NewWithSeed(seed uint64) *Digest

NewWithSeed creates a new Digest with the given seed.

func (*Digest) BlockSize

func (d *Digest) BlockSize() int

BlockSize always returns 32 bytes.

func (*Digest) MarshalBinary

func (d *Digest) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface.

func (*Digest) Reset

func (d *Digest) Reset()

Reset clears the Digest's state so that it can be reused. It uses a seed value of zero.

func (*Digest) ResetWithSeed

func (d *Digest) ResetWithSeed(seed uint64)

ResetWithSeed clears the Digest's state so that it can be reused. It uses the given seed to initialize the state.

func (*Digest) Size

func (d *Digest) Size() int

Size always returns 8 bytes.

func (*Digest) Sum

func (d *Digest) Sum(b []byte) []byte

Sum appends the current hash to b and returns the resulting slice.

func (*Digest) Sum64

func (d *Digest) Sum64() uint64

Sum64 returns the current hash.

func (*Digest) UnmarshalBinary

func (d *Digest) UnmarshalBinary(b []byte) error

UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.

func (*Digest) Write

func (d *Digest) Write(b []byte) (n int, err error)

Write adds more data to d. It always returns len(b), nil.

func (*Digest) WriteString

func (d *Digest) WriteString(s string) (n int, err error)

WriteString adds more data to d. It always returns len(s), nil. It may be faster than Write([]byte(s)) by avoiding a copy.