README
¶
xxhash 
This is a native Go implementation of the excellent xxhash* algorithm, an extremely fast non-cryptographic Hash algorithm, working at speeds close to RAM limits.
- The C implementation is (Copyright (c) 2012-2014, Yann Collet)
Install
go get github.com/OneOfOne/xxhash
Features
- On Go 1.7+ the pure go version is faster than CGO for all inputs.
- Supports ChecksumString{32,64} xxhash{32,64}.WriteString, which uses no copies when it can, falls back to copy on appengine.
- The native version falls back to a less optimized version on appengine due to the lack of unsafe.
- Almost as fast as the mostly pure assembly version written by the brilliant cespare, while also supporting seeds.
- To manually toggle the appengine version build with
-tags safe.
Benchmark
Core i7-4790 @ 3.60GHz, Linux 4.12.6-1-ARCH (64bit), Go tip (+ff90f4af66 2017-08-19)
➤ go test -bench '64' -count 5 -tags cespare | benchstat /dev/stdin
name time/op
# https://github.com/cespare/xxhash
XXSum64Cespare/Func-8 160ns ± 2%
XXSum64Cespare/Struct-8 173ns ± 1%
XXSum64ShortCespare/Func-8 6.78ns ± 1%
XXSum64ShortCespare/Struct-8 19.6ns ± 2%
# this package (default mode, using unsafe)
XXSum64/Func-8 170ns ± 1%
XXSum64/Struct-8 182ns ± 1%
XXSum64Short/Func-8 13.5ns ± 3%
XXSum64Short/Struct-8 20.4ns ± 0%
# this package (appengine, *not* using unsafe)
XXSum64/Func-8 241ns ± 5%
XXSum64/Struct-8 243ns ± 6%
XXSum64Short/Func-8 15.2ns ± 2%
XXSum64Short/Struct-8 23.7ns ± 5%
CRC64ISO-8 1.23µs ± 1%
CRC64ISOString-8 2.71µs ± 4%
CRC64ISOShort-8 22.2ns ± 3%
Fnv64-8 2.34µs ± 1%
Fnv64Short-8 74.7ns ± 8%
Usage
h := xxhash.New64()
// r, err := os.Open("......")
// defer f.Close()
r := strings.NewReader(F)
io.Copy(h, r)
fmt.Println("xxhash.Backend:", xxhash.Backend)
fmt.Println("File checksum:", h.Sum64())
TODO
- Rewrite the 32bit version to be more optimized.
- General cleanup as the Go inliner gets smarter.
License
This project is released under the Apache v2. license. See LICENSE for more details.
Documentation
¶
Index ¶
- Constants
- func Checksum32(in []byte) uint32
- func Checksum32S(in []byte, seed uint32) (h uint32)
- func Checksum64(in []byte) uint64
- func Checksum64S(in []byte, seed uint64) uint64
- func ChecksumString32(s string) uint32
- func ChecksumString32S(s string, seed uint32) uint32
- func ChecksumString64(s string) uint64
- func ChecksumString64S(s string, seed uint64) uint64
- func NewHash32() hash.Hash
- func NewHash64() hash.Hash
- type XXHash32
- func (xx *XXHash32) BlockSize() int
- func (xx *XXHash32) MarshalBinary() ([]byte, error)
- func (xx *XXHash32) Reset()
- func (xx *XXHash32) Size() int
- func (xx *XXHash32) Sum(in []byte) []byte
- func (xx *XXHash32) Sum32() (h uint32)
- func (xx *XXHash32) UnmarshalBinary(b []byte) error
- func (xx *XXHash32) Write(in []byte) (n int, err error)
- func (xx *XXHash32) WriteString(s string) (int, error)
- type XXHash64
- func (xx *XXHash64) BlockSize() int
- func (xx *XXHash64) MarshalBinary() ([]byte, error)
- func (xx *XXHash64) Reset()
- func (xx *XXHash64) Size() int
- func (xx *XXHash64) Sum(in []byte) []byte
- func (xx *XXHash64) Sum64() (h uint64)
- func (xx *XXHash64) UnmarshalBinary(b []byte) error
- func (xx *XXHash64) Write(in []byte) (n int, err error)
- func (xx *XXHash64) WriteString(s string) (int, error)
Constants ¶
const Backend = "GoUnsafe"
Backend returns the current version of xxhash being used.
Variables ¶
This section is empty.
Functions ¶
func Checksum32 ¶
Checksum32 returns the checksum of the input data with the seed set to 0.
func Checksum32S ¶
Checksum32S returns the checksum of the input bytes with the specific seed.
func Checksum64S ¶
Checksum64S returns the 64bit xxhash checksum for a single input
func ChecksumString32 ¶
ChecksumString32 returns the checksum of the input data, without creating a copy, with the seed set to 0.
func ChecksumString32S ¶
ChecksumString32S returns the checksum of the input data, without creating a copy, with the specific seed.
func ChecksumString64 ¶
ChecksumString64 returns the checksum of the input data, without creating a copy, with the seed set to 0.
func ChecksumString64S ¶
ChecksumString64S returns the checksum of the input data, without creating a copy, with the specific seed.
Types ¶
type XXHash32 ¶
type XXHash32 struct {
// contains filtered or unexported fields
}
func New32 ¶
func New32() *XXHash32
New32 creates a new hash.Hash32 computing the 32bit xxHash checksum starting with the seed set to 0.
func NewS32 ¶
NewS32 creates a new hash.Hash32 computing the 32bit xxHash checksum starting with the specific seed.
func (*XXHash32) BlockSize ¶
BlockSize returns the hash's underlying block size. The Write method must be able to accept any amount of data, but it may operate more efficiently if all writes are a multiple of the block size.
func (*XXHash32) MarshalBinary ¶ added in v1.2.8
MarshalBinary implements the encoding.BinaryMarshaler interface.
func (*XXHash32) Sum ¶
Sum appends the current hash to b and returns the resulting slice. It does not change the underlying hash state.
func (*XXHash32) UnmarshalBinary ¶ added in v1.2.8
UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.
type XXHash64 ¶
type XXHash64 struct {
// contains filtered or unexported fields
}
func New64 ¶
func New64() *XXHash64
New64 creates a new hash.Hash64 computing the 64bit xxHash checksum starting with the seed set to 0x0.
func NewS64 ¶
NewS64 creates a new hash.Hash64 computing the 64bit xxHash checksum starting with the specific seed.
func (*XXHash64) BlockSize ¶
BlockSize returns the hash's underlying block size. The Write method must be able to accept any amount of data, but it may operate more efficiently if all writes are a multiple of the block size.
func (*XXHash64) MarshalBinary ¶ added in v1.2.8
MarshalBinary implements the encoding.BinaryMarshaler interface.
func (*XXHash64) Sum ¶
Sum appends the current hash to b and returns the resulting slice. It does not change the underlying hash state.
func (*XXHash64) UnmarshalBinary ¶ added in v1.2.8
UnmarshalBinary implements the encoding.BinaryUnmarshaler interface.
Source Files
Directories