xxhash

README

xxhash

xxhash is a Go implementation of the 64-bit xxHash algorithm, XXH64. This is a high-quality hashing algorithm that is much faster than anything in the Go standard library.

This package provides a straightforward API:

func Sum64(b []byte) uint64
func Sum64String(s string) uint64
type Digest struct{ ... }
    func New() *Digest

The Digest type implements hash.Hash64. Its key methods are:

func (*Digest) Write([]byte) (int, error)
func (*Digest) WriteString(string) (int, error)
func (*Digest) Sum64() uint64

This implementation provides a fast pure-Go implementation and an even faster assembly implementation for amd64.

Compatibility

This package is in a module and the latest code is in version 2 of the module. You need a version of Go with at least "minimal module compatibility" to use github.com/cespare/xxhash/v2:

• 1.9.7+ for Go 1.9
• 1.10.3+ for Go 1.10
• Go 1.11 or later

I recommend using the latest release of Go.

Benchmarks

Here are some quick benchmarks comparing the pure-Go and assembly implementations of Sum64.

input size	purego	asm
5 B	979.66 MB/s	1291.17 MB/s
100 B	7475.26 MB/s	7973.40 MB/s
4 KB	17573.46 MB/s	17602.65 MB/s
10 MB	17131.46 MB/s	17142.16 MB/s

These numbers were generated on Ubuntu 18.04 with an Intel i7-8700K CPU using the following commands under Go 1.11.2:

$ go test -tags purego -benchtime 10s -bench '/xxhash,direct,bytes'
$ go test -benchtime 10s -bench '/xxhash,direct,bytes'

Projects using this package

• InfluxDB
• Prometheus
• VictoriaMetrics
• FreeCache
• FastCache

Documentation

Overview

Package xxhash implements the 64-bit variant of xxHash (XXH64) as described at http://cyan4973.github.io/xxHash/.

Index

• func Sum64(b []byte) uint64
• func Sum64String(s string) uint64
• type Digest
  • func New() *Digest
  • func (d *Digest) BlockSize() int
  • func (d *Digest) MarshalBinary() ([]byte, error)
  • func (d *Digest) Reset()
  • func (d *Digest) Size() int
  • func (d *Digest) Sum(b []byte) []byte
  • func (d *Digest) Sum64() uint64
  • func (d *Digest) UnmarshalBinary(b []byte) error
  • func (d *Digest) Write(b []byte) (n int, err error)
  • func (d *Digest) WriteString(s string) (n int, err error)

Functions

func Sum64

func Sum64(b []byte) uint64

Sum64 computes the 64-bit xxHash digest of b.

func Sum64String

func Sum64String(s string) uint64

Sum64String computes the 64-bit xxHash digest of s. It may be faster than Sum64([]byte(s)) by avoiding a copy.

Types

type Digest

type Digest struct {
	// contains filtered or unexported fields
}

Digest implements hash.Hash64.

func New

func New() *Digest

New creates a new Digest that computes the 64-bit xxHash algorithm.

func (*Digest) BlockSize

func (d *Digest) BlockSize() int

BlockSize always returns 32 bytes.

func (*Digest) MarshalBinary

func (d *Digest) MarshalBinary() ([]byte, error)

MarshalBinary implements the encoding.BinaryMarshaler interface.

func (*Digest) Reset

func (d *Digest) Reset()

Reset clears the Digest's state so that it can be reused.

func (*Digest) Size

func (d *Digest) Size() int

Size always returns 8 bytes.

func (*Digest) Sum

func (d *Digest) Sum(b []byte) []byte

Sum appends the current hash to b and returns the resulting slice.

func (*Digest) Sum64

func (d *Digest) Sum64() uint64

Sum64 returns the current hash.

func (*Digest) UnmarshalBinary

func (d *Digest) UnmarshalBinary(b []byte) error

UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.

func (*Digest) Write

func (d *Digest) Write(b []byte) (n int, err error)

Write adds more data to d. It always returns len(b), nil.

func (*Digest) WriteString

func (d *Digest) WriteString(s string) (n int, err error)

WriteString adds more data to d. It always returns len(s), nil. It may be faster than Write([]byte(s)) by avoiding a copy.