blake3

package module
v0.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 14, 2024 License: CC0-1.0 Imports: 8 Imported by: 214

README

BLAKE3

go.dev Go Report Card SourceGraph

Pure Go implementation of BLAKE3 with AVX2 and SSE4.1 acceleration.

Special thanks to the excellent avo making writing vectorized version much easier.

Benchmarks

Caveats

This library makes some different design decisions than the upstream Rust crate around internal buffering. Specifically, because it does not target the embedded system space, nor does it support multithreading, it elects to do its own internal buffering. This means that a user does not have to worry about providing large enough buffers to get the best possible performance, but it does worse on smaller input sizes. So some notes:

  • The Rust benchmarks below are all single-threaded to match this Go implementation.
  • I make no attempt to get precise measurements (cpu throttling, noisy environment, etc.) so please benchmark on your own systems.
  • These benchmarks are run on an i7-6700K which does not support AVX-512, so Rust is limited to use AVX2 at sizes above 8 kib.
  • I tried my best to make them benchmark the same thing, but who knows? 😄

Charts

In this case, both libraries are able to avoid a lot of data copying and will use vectorized instructions to hash as fast as possible, and perform similarly.

Large Full Buffer

For incremental writes, you must provide the Rust version large enough buffers so that it can use vectorized instructions. This Go library performs consistently regardless of the size being sent into the update function.

Incremental

The downside of internal buffering is most apparent with small sizes as most time is spent initializing the hasher state. In terms of hashing rate, the difference is 3-4x, but in an absolute sense it's ~100ns (see tables below). If you wish to hash a large number of very small strings and you care about those nanoseconds, be sure to use the Reset method to avoid re-initializing the state.

Small Full Buffer

Timing Tables

Small
Size Full Buffer Reset Full Buffer Rate Reset Rate
64 b 205ns 86.5ns 312MB/s 740MB/s
256 b 364ns 250ns 703MB/s 1.03GB/s
512 b 575ns 468ns 892MB/s 1.10GB/s
768 b 795ns 682ns 967MB/s 1.13GB/s
Large
Size Incremental Full Buffer Reset Incremental Rate Full Buffer Rate Reset Rate
1 kib 1.02µs 1.01µs 891ns 1.00GB/s 1.01GB/s 1.15GB/s
2 kib 2.11µs 2.07µs 1.95µs 968MB/s 990MB/s 1.05GB/s
4 kib 2.28µs 2.15µs 2.05µs 1.80GB/s 1.90GB/s 2.00GB/s
8 kib 2.64µs 2.52µs 2.44µs 3.11GB/s 3.25GB/s 3.36GB/s
16 kib 4.93µs 4.54µs 4.48µs 3.33GB/s 3.61GB/s 3.66GB/s
32 kib 9.41µs 8.62µs 8.54µs 3.48GB/s 3.80GB/s 3.84GB/s
64 kib 18.2µs 16.7µs 16.6µs 3.59GB/s 3.91GB/s 3.94GB/s
128 kib 36.3µs 32.9µs 33.1µs 3.61GB/s 3.99GB/s 3.96GB/s
256 kib 72.5µs 65.7µs 66.0µs 3.62GB/s 3.99GB/s 3.97GB/s
512 kib 145µs 131µs 132µs 3.60GB/s 4.00GB/s 3.97GB/s
1024 kib 290µs 262µs 262µs 3.62GB/s 4.00GB/s 4.00GB/s
No ASM
Size Incremental Full Buffer Reset Incremental Rate Full Buffer Rate Reset Rate
64 b 253ns 254ns 134ns 253MB/s 252MB/s 478MB/s
256 b 553ns 557ns 441ns 463MB/s 459MB/s 580MB/s
512 b 948ns 953ns 841ns 540MB/s 538MB/s 609MB/s
768 b 1.38µs 1.40µs 1.35µs 558MB/s 547MB/s 570MB/s
1 kib 1.77µs 1.77µs 1.70µs 577MB/s 580MB/s 602MB/s
1024 kib 880µs 883µs 878µs 596MB/s 595MB/s 598MB/s

The speed caps out at around 1 kib, so most rows have been elided from the presentation.

Documentation

Overview

Package blake3 provides an SSE4.1/AVX2 accelerated BLAKE3 implementation.

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func DeriveKey added in v0.0.3 

func DeriveKey(context string, material []byte, out []byte)

DeriveKey derives a key based on reusable key material of any length, in the given context. The key will be stored in out, using all of its current length.

Context strings must be hardcoded constants, and the recommended format is "[application] [commit timestamp] [purpose]", e.g., "example.com 2019-12-25 16:18:03 session tokens v1".

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	out := make([]byte, 32)

	// See the documentation for good practices on what the context should be.
	blake3.DeriveKey(
		"my-application v0.1.1 session tokens v1",  // context
		[]byte("some material to derive key from"), // material
		out,
	)

	fmt.Printf("%x\n", out)
}

Output:

98a3333af735f89eb301b56eaf6a77713aa03cdb0057e5b04352a63ea9204add

func Sum256 added in v0.1.1 

func Sum256(data []byte) (sum [32]byte)

Sum256 returns the first 256 bits of the unkeyed digest of the data.

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	digest := blake3.Sum256([]byte("some data"))

	fmt.Printf("%x\n", digest[:])
}

Output:

b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2

func Sum512 added in v0.1.1 

func Sum512(data []byte) (sum [64]byte)

Sum512 returns the first 512 bits of the unkeyed digest of the data.

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	digest := blake3.Sum512([]byte("some data"))

	fmt.Printf("%x\n", digest[0:32])
	fmt.Printf("%x\n", digest[32:64])
}

Output:

b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2
1b55688951738e3a7155d6398eb56c6bc35d5bca5f139d98eb7409be51d1be32

Types

type Digest added in v0.0.3 

type Digest struct {
	// contains filtered or unexported fields
}

Digest captures the state of a Hasher allowing reading and seeking through the output stream.

func (*Digest) Read added in v0.0.3 

func (d *Digest) Read(p []byte) (n int, err error)

Read reads data from the hasher into out. It always fills the entire buffer and never errors. The stream will wrap around when reading past 2^64 bytes.

func (*Digest) Seek added in v0.0.3 

func (d *Digest) Seek(offset int64, whence int) (int64, error)

Seek sets the position to the provided location. Only SeekStart and SeekCurrent are allowed.

Example 
package main

import (
	"fmt"
	"io"

	"github.com/zeebo/blake3"
)

func main() {
	h := blake3.New()
	h.Write([]byte("some data"))
	d := h.Digest()

	out := make([]byte, 32)
	d.Seek(32, io.SeekStart)
	d.Read(out)

	fmt.Printf("%x\n", out)
}

Output:

1b55688951738e3a7155d6398eb56c6bc35d5bca5f139d98eb7409be51d1be32

type Hasher 

type Hasher struct {
	// contains filtered or unexported fields
}

Hasher is a hash.Hash for BLAKE3.

func New 

func New() *Hasher

New returns a new Hasher that has a digest size of 32 bytes.

If you need more or less output bytes than that, use Digest method.

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	h := blake3.New()

	h.Write([]byte("some data"))

	fmt.Printf("%x\n", h.Sum(nil))
}

Output:

b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2

func NewDeriveKey added in v0.0.3 

func NewDeriveKey(context string) *Hasher

NewDeriveKey returns a Hasher that is initialized with the context string. See DeriveKey for details. It has a digest size of 32 bytes.

If you need more or less output bytes than that, use the Digest method.

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	// See the documentation for good practices on what the context should be.
	h := blake3.NewDeriveKey("my-application v0.1.1 session tokens v1")

	h.Write([]byte("some material to derive key from"))

	fmt.Printf("%x\n", h.Sum(nil))
}

Output:

98a3333af735f89eb301b56eaf6a77713aa03cdb0057e5b04352a63ea9204add

func NewKeyed added in v0.0.3 

func NewKeyed(key []byte) (*Hasher, error)

NewKeyed returns a new Hasher that uses the 32 byte input key and has a digest size of 32 bytes.

If you need more or less output bytes than that, use the Digest method.

Example 
package main

import (
	"bytes"
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	h1, err := blake3.NewKeyed(bytes.Repeat([]byte("1"), 32))
	if err != nil {
		panic(err)
	}

	h2, err := blake3.NewKeyed(bytes.Repeat([]byte("2"), 32))
	if err != nil {
		panic(err)
	}

	h1.Write([]byte("some data"))
	h2.Write([]byte("some data"))

	fmt.Printf("%x\n", h1.Sum(nil))
	fmt.Printf("%x\n", h2.Sum(nil))
}

Output:

107c6f88638356d73cdb80f4d56ffe50abcbd9664a80c8ab2b83b1f946ebaba1
b4be81075bef5a2448158ee5eeddaed897fe44a564c2cb088facbe7824a25073

func (*Hasher) BlockSize 

func (h *Hasher) BlockSize() int

BlockSize implements part of the hash.Hash interface. It returns the most natural size to write to the Hasher.

func (*Hasher) Clone added in v0.2.0 

func (h *Hasher) Clone() *Hasher

Clone returns a new Hasher with the same internal state.

Modifying the resulting Hasher will not modify the original Hasher, and vice versa.

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	h1 := blake3.New()
	h1.WriteString("some")

	h2 := h1.Clone()
	fmt.Println("before:")
	fmt.Printf("h1: %x\n", h1.Sum(nil))
	fmt.Printf("h2: %x\n\n", h2.Sum(nil))

	h2.WriteString(" data")

	fmt.Println("h2 modified:")
	fmt.Printf("h1: %x\n", h1.Sum(nil))
	fmt.Printf("h2: %x\n\n", h2.Sum(nil))

	h1.WriteString(" data")

	fmt.Println("h1 converged:")
	fmt.Printf("h1: %x\n", h1.Sum(nil))
	fmt.Printf("h2: %x\n", h2.Sum(nil))

}

Output:

before:
h1: 2f610cf2e7e0dc09384cbaa75b2ae5d9704ac9a5ac7f28684342856e2867c707
h2: 2f610cf2e7e0dc09384cbaa75b2ae5d9704ac9a5ac7f28684342856e2867c707

h2 modified:
h1: 2f610cf2e7e0dc09384cbaa75b2ae5d9704ac9a5ac7f28684342856e2867c707
h2: b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2

h1 converged:
h1: b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2
h2: b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2

func (*Hasher) Digest added in v0.0.3 

func (h *Hasher) Digest() *Digest

Digest takes a snapshot of the hash state and returns an object that can be used to read and seek through 2^64 bytes of digest output.

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	h := blake3.New()
	h.Write([]byte("some data"))
	d := h.Digest()

	out := make([]byte, 64)
	d.Read(out)

	fmt.Printf("%x\n", out[0:32])
	fmt.Printf("%x\n", out[32:64])
}

Output:

b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2
1b55688951738e3a7155d6398eb56c6bc35d5bca5f139d98eb7409be51d1be32

func (*Hasher) Reset 

func (h *Hasher) Reset()

Reset implements part of the hash.Hash interface. It causes the Hasher to act as if it was newly created.

Example 
package main

import (
	"fmt"

	"github.com/zeebo/blake3"
)

func main() {
	h := blake3.New()

	h.Write([]byte("some data"))
	fmt.Printf("%x\n", h.Sum(nil))

	h.Reset()

	h.Write([]byte("some data"))
	fmt.Printf("%x\n", h.Sum(nil))
}

Output:

b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2
b224a1da2bf5e72b337dc6dde457a05265a06dec8875be379e2ad2be5edb3bf2

func (*Hasher) Size 

func (h *Hasher) Size() int

Size implements part of the hash.Hash interface. It returns the number of bytes the hash will output in Sum.

func (*Hasher) Sum 

func (h *Hasher) Sum(b []byte) []byte

Sum implements part of the hash.Hash interface. It appends the digest of the Hasher to the provided buffer and returns it.

func (*Hasher) Write 

func (h *Hasher) Write(p []byte) (int, error)

Write implements part of the hash.Hash interface. It never returns an error.

func (*Hasher) WriteString added in v0.0.3 

func (h *Hasher) WriteString(p string) (int, error)

WriteString is like Write but specialized to strings to avoid allocations.

Source Files

View all Source files

Directories

Path Synopsis
avo module
internal
alg
alg/compress
alg/compress/compress_pure
alg/compress/compress_sse41
alg/hash
alg/hash/hash_avx2
alg/hash/hash_pure
consts
utils

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.