ssz

package module
v0.1.2-patch Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 3, 2023 License: MIT Imports: 17 Imported by: 3

README

FastSSZ

The FastSSZ project in this reposity is a combination of two things: a high performant low level library to work with SSZ encodings (root of this project) and the (sszgen) code generator that generates the SSZ encodings for Go structs using the SSZ library. By combining both, this library achieves peak Go native performance and zero memory allocation. The repository uses as test the official Ethereum SSZ tests (spectests) for the Consensus Spec data structures.

If you are only looking for the Consensus data structures and types with the SSZ support, it is recommended to use go-eth-consensus instead, since it is already integrated with other parts of the Consensus stack, like the Beacon http API.

Clone:

$ git clone git@github.com:ferranbt/fastssz.git

Download the eth2.0 spec tests

$ make get-spec-tests

Regenerate the test spec encodings:

$ make build-spec-tests

Generate encodings for a specific package:

$ go run sszgen/*.go --path ./ethereumapis/eth/v1alpha1 [--objs BeaconBlock,Eth1Data]

Optionally, you can specify the objs you want to generate. Otherwise, it will generate encodings for all structs in the package. Note that if a struct does not have 'ssz' tags when required (i.e size of arrays), the generator will fail.

By default, it generates a file with the prefix '_encoding.go' for each file that contains a generated struct. Optionally, you can combine all the outputs in a single file with the 'output' flag.

$ go run sszgen/*.go --path ./ethereumapis/eth/v1alpha1 --output ./ethereumapis/eth/v1alpha1/encoding.go

Test the spectests:

$ go test -v ./spectests/... -run TestSpec

Run the fuzzer:

$ FUZZ_TESTS=True go test -v ./spectests/... -run TestFuzz

To install the generator run:

$ go get github.com/ferranbt/fastssz/sszgen

Benchmark (BeaconBlock):

$ go test -v ./spectests/... -run=XXX -bench=.
goos: linux
goarch: amd64
pkg: github.com/ferranbt/fastssz/spectests
cpu: AMD Ryzen 5 2400G with Radeon Vega Graphics
BenchmarkMarshal_Fast
BenchmarkMarshal_Fast-8             	  291054	      4088 ns/op	    8192 B/op	       1 allocs/op
BenchmarkMarshal_SuperFast
BenchmarkMarshal_SuperFast-8        	  798883	      1354 ns/op	       0 B/op	       0 allocs/op
BenchmarkUnMarshal_Fast
BenchmarkUnMarshal_Fast-8           	   64065	     17614 ns/op	   11900 B/op	     210 allocs/op
BenchmarkHashTreeRoot_Fast
BenchmarkHashTreeRoot_Fast-8        	   25863	     45932 ns/op	       0 B/op	       0 allocs/op
BenchmarkHashTreeRoot_SuperFast
BenchmarkHashTreeRoot_SuperFast-8   	   54078	     21999 ns/op	       0 B/op	       0 allocs/op
PASS
ok  	github.com/ferranbt/fastssz/spectests	5.501s

Package reference

To reference a struct from another package use the '--include' flag to point to that package.

Example:

$ go run sszgen/*.go --path ./example2
$ go run sszgen/*.go --path ./example
[ERR]: could not find struct with name 'Checkpoint'
$ go run sszgen/*.go --path ./example --include ./example2

There are some caveats required to use this functionality.

  • If multiple input paths import the same package, all of them need to import it with the same alias if any.
  • If the folder of the package is not the same as the name of the package, any input file that imports this package needs to do it with an alias.

Fast HashTreeRoot

Fastssz integrates with Prysm gohashtree library to do high performance and concurrent Sha256 hashing. It achieves a 2x performance improvement with respect to the normal sequential hashing. As of now, this feature is not yet enabled by default since it does not use the gohashtree main branch. You can track the updates on this issue.

In order to use this feature, enable manually the hash function in the Hasher like in the benchmark example.

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	ErrOffset                = fmt.Errorf("incorrect offset")
	ErrSize                  = fmt.Errorf("incorrect size")
	ErrBytesLength           = fmt.Errorf("bytes array does not have the correct length")
	ErrVectorLength          = fmt.Errorf("vector does not have the correct length")
	ErrListTooBig            = fmt.Errorf("list length is higher than max value")
	ErrEmptyBitlist          = fmt.Errorf("bitlist is empty")
	ErrInvalidVariableOffset = fmt.Errorf("invalid ssz encoding. first variable element offset indexes into fixed value data")
)
View Source 
var (
	// ErrIncorrectByteSize means that the byte size is incorrect
	ErrIncorrectByteSize = fmt.Errorf("incorrect byte size")

	// ErrIncorrectListSize means that the size of the list is incorrect
	ErrIncorrectListSize = fmt.Errorf("incorrect list size")
)

Functions

func CalculateLimit 

func CalculateLimit(maxCapacity, numItems, size uint64) uint64

func DecodeDynamicLength 

func DecodeDynamicLength(buf []byte, maxSize int) (int, error)

DecodeDynamicLength decodes the length from the dynamic input

func DivideInt 

func DivideInt(a, b int) (int, bool)

DivideInt divides the int fully

func DivideInt2 

func DivideInt2(a, b, max int) (int, error)

func ErrBytesLengthFn 

func ErrBytesLengthFn(name string, found, expected int) error

func ErrListTooBigFn 

func ErrListTooBigFn(name string, found, max int) error

func ErrVectorLengthFn 

func ErrVectorLengthFn(name string, found, expected int) error

func ExtendUint16 

func ExtendUint16(b []uint16, needLen int) []uint16

ExtendUint16 extends a uint16 buffer to a given size

func ExtendUint64 

func ExtendUint64(b []uint64, needLen int) []uint64

ExtendUint64 extends a uint64 buffer to a given size

func ExtendUint8 

func ExtendUint8(b []uint8, needLen int) []uint8

ExtendUint16 extends a uint16 buffer to a given size

func HashWithDefaultHasher 

func HashWithDefaultHasher(v HashRoot) ([32]byte, error)

HashWithDefaultHasher hashes a HashRoot object with a Hasher from the default HasherPool

func MarshalBool 

func MarshalBool(dst []byte, b bool) []byte

MarshalBool marshals a boolean to dst

func MarshalSSZ 

func MarshalSSZ(m Marshaler) ([]byte, error)

MarshalSSZ marshals an object

func MarshalTime 

func MarshalTime(dst []byte, t time.Time) []byte

MarshalTime marshals a time to dst

func MarshalUint16 

func MarshalUint16(dst []byte, i uint16) []byte

MarshalUint16 marshals a little endian uint16 to dst

func MarshalUint32 

func MarshalUint32(dst []byte, i uint32) []byte

MarshalUint32 marshals a little endian uint32 to dst

func MarshalUint64 

func MarshalUint64(dst []byte, i uint64) []byte

MarshalUint64 marshals a little endian uint64 to dst

func MarshalUint8 

func MarshalUint8(dst []byte, i uint8) []byte

MarshalUint8 marshals a little endian uint8 to dst

func ReadOffset 

func ReadOffset(buf []byte) uint64

ReadOffset reads an offset from buf

func UnmarshalBool 

func UnmarshalBool(src []byte) bool

UnmarshalBool unmarshals a boolean from the src input

func UnmarshalDynamic 

func UnmarshalDynamic(src []byte, length int, f func(indx int, b []byte) error) error

UnmarshalDynamic unmarshals the dynamic items from the input

func UnmarshalSSZTest 

func UnmarshalSSZTest(content []byte, result interface{}) error

func UnmarshalTime 

func UnmarshalTime(src []byte) time.Time

UnmarshalTime unmarshals a time.Time from the src input

func UnmarshallUint16 

func UnmarshallUint16(src []byte) uint16

UnmarshallUint16 unmarshals a little endian uint16 from the src input

func UnmarshallUint32 

func UnmarshallUint32(src []byte) uint32

UnmarshallUint32 unmarshals a little endian uint32 from the src input

func UnmarshallUint64 

func UnmarshallUint64(src []byte) uint64

UnmarshallUint64 unmarshals a little endian uint64 from the src input

func UnmarshallUint8 

func UnmarshallUint8(src []byte) uint8

UnmarshallUint8 unmarshals a little endian uint8 from the src input

func ValidateBitlist 

func ValidateBitlist(buf []byte, bitLimit uint64) error

ValidateBitlist validates that the bitlist is correct

func VerifyMultiproof 

func VerifyMultiproof(root []byte, proof [][]byte, leaves [][]byte, indices []int) (bool, error)

VerifyMultiproof verifies a proof for multiple leaves against the given root.

func VerifyProof 

func VerifyProof(root []byte, proof *Proof) (bool, error)

VerifyProof verifies a single merkle branch. It's more efficient than VerifyMultiproof for proving one leaf.

func WriteOffset 

func WriteOffset(dst []byte, i int) []byte

WriteOffset writes an offset to dst

Types

type CompressedMultiproof 

type CompressedMultiproof struct {
	Indices    []int
	Leaves     [][]byte
	Hashes     [][]byte
	ZeroLevels []int // Stores the level for every omitted zero hash in the proof
}

CompressedMultiproof represents a compressed merkle proof of several leaves. Compression is achieved by omitting zero hashes (and their hashes). `ZeroLevels` contains information which helps the verifier fill in those hashes.

func (*CompressedMultiproof) Decompress 

func (c *CompressedMultiproof) Decompress() *Multiproof

Decompress returns a new multiproof, filling in the omitted zero hashes. See `CompressedMultiProof` for more info.

type HashFn 

type HashFn func(dst []byte, input []byte) error

func NativeHashWrapper 

func NativeHashWrapper(hashFn hash.Hash) HashFn

type HashRoot 

type HashRoot interface {
	GetTree() (*Node, error)
	HashTreeRoot() ([32]byte, error)
	HashTreeRootWith(hh HashWalker) error
}

type HashWalker 

type HashWalker interface {
	// Intended for testing purposes to know the latest hash generated during merkleize
	Hash() []byte
	AppendUint8(i uint8)
	AppendUint64(i uint64)
	AppendBytes32(b []byte)
	PutUint64(i uint64)
	PutUint32(i uint32)
	PutUint16(i uint16)
	PutUint8(i uint8)
	FillUpTo32()
	Append(i []byte)
	PutBitlist(bb []byte, maxSize uint64)
	PutBool(b bool)
	PutBytes(b []byte)
	Index() int
	Merkleize(indx int)
	MerkleizeWithMixin(indx int, num, limit uint64)
}

type Hasher 

type Hasher struct {
	// contains filtered or unexported fields
}

Hasher is a utility tool to hash SSZ structs

func NewHasher 

func NewHasher() *Hasher

NewHasher creates a new Hasher object with sha256 hash

func NewHasherWithHash 

func NewHasherWithHash(hh hash.Hash) *Hasher

NewHasherWithHash creates a new Hasher object with a custom hash.Hash function

func NewHasherWithHashFn 

func NewHasherWithHashFn(hh HashFn) *Hasher

NewHasherWithHashFn creates a new Hasher object with a custom HashFn function

func (*Hasher) Append 

func (h *Hasher) Append(i []byte)

func (*Hasher) AppendBytes32 

func (h *Hasher) AppendBytes32(b []byte)

func (*Hasher) AppendUint64 

func (h *Hasher) AppendUint64(i uint64)

func (*Hasher) AppendUint8 

func (h *Hasher) AppendUint8(i uint8)

func (*Hasher) FillUpTo32 

func (h *Hasher) FillUpTo32()

func (*Hasher) Hash 

func (h *Hasher) Hash() []byte

func (*Hasher) HashRoot 

func (h *Hasher) HashRoot() (res [32]byte, err error)

HashRoot creates the hash final hash root

func (*Hasher) Index 

func (h *Hasher) Index() int

Index marks the current buffer index

func (*Hasher) Merkleize 

func (h *Hasher) Merkleize(indx int)

Merkleize is used to merkleize the last group of the hasher

func (*Hasher) MerkleizeWithMixin 

func (h *Hasher) MerkleizeWithMixin(indx int, num, limit uint64)

MerkleizeWithMixin is used to merkleize the last group of the hasher

func (*Hasher) PutBitlist 

func (h *Hasher) PutBitlist(bb []byte, maxSize uint64)

PutBitlist appends a ssz bitlist

func (*Hasher) PutBool 

func (h *Hasher) PutBool(b bool)

PutBool appends a boolean

func (*Hasher) PutBytes 

func (h *Hasher) PutBytes(b []byte)

PutBytes appends bytes

func (*Hasher) PutRootVector 

func (h *Hasher) PutRootVector(b [][]byte, maxCapacity ...uint64) error

PutRootVector appends an array of roots

func (*Hasher) PutUint16 

func (h *Hasher) PutUint16(i uint16)

PutUint16 appends a uint16 in 32 bytes

func (*Hasher) PutUint32 

func (h *Hasher) PutUint32(i uint32)

PutUint32 appends a uint32 in 32 bytes

func (*Hasher) PutUint64 

func (h *Hasher) PutUint64(i uint64)

PutUint64 appends a uint64 in 32 bytes

func (*Hasher) PutUint64Array 

func (h *Hasher) PutUint64Array(b []uint64, maxCapacity ...uint64)

PutUint64Array appends an array of uint64

func (*Hasher) PutUint8 

func (h *Hasher) PutUint8(i uint8)

PutUint16 appends a uint16 in 32 bytes

func (*Hasher) Reset 

func (h *Hasher) Reset()

Reset resets the Hasher obj

type HasherPool 

type HasherPool struct {
	// contains filtered or unexported fields
}

HasherPool may be used for pooling Hashers for similarly typed SSZs. 

var DefaultHasherPool HasherPool

DefaultHasherPool is a default hasher pool

func (*HasherPool) Get 

func (hh *HasherPool) Get() *Hasher

Get acquires a Hasher from the pool.

func (*HasherPool) Put 

func (hh *HasherPool) Put(h *Hasher)

Put releases the Hasher to the pool.

type Marshaler 

type Marshaler interface {
	MarshalSSZTo(dst []byte) ([]byte, error)
	MarshalSSZ() ([]byte, error)
	SizeSSZ() int
}

Marshaler is the interface implemented by types that can marshal themselves into valid SZZ.

type Multiproof 

type Multiproof struct {
	Indices []int
	Leaves  [][]byte
	Hashes  [][]byte
}

Multiproof represents a merkle proof of several leaves.

func (*Multiproof) Compress 

func (p *Multiproof) Compress() *CompressedMultiproof

Compress returns a new proof with zero hashes omitted. See `CompressedMultiproof` for more info.

type Node 

type Node struct {
	// contains filtered or unexported fields
}

Node represents a node in the tree backing of a SSZ object.

func EmptyLeaf 

func EmptyLeaf() *Node

func LeafFromBool 

func LeafFromBool(b bool) *Node

func LeafFromBytes 

func LeafFromBytes(b []byte) *Node

func LeafFromUint16 

func LeafFromUint16(i uint16) *Node

func LeafFromUint32 

func LeafFromUint32(i uint32) *Node

func LeafFromUint64 

func LeafFromUint64(i uint64) *Node

func LeafFromUint8 

func LeafFromUint8(i uint8) *Node

func LeavesFromUint64 

func LeavesFromUint64(items []uint64) []*Node

func NewNodeWithLR 

func NewNodeWithLR(left, right *Node) *Node

NewNodeWithLR initializes a branch node.

func NewNodeWithValue 

func NewNodeWithValue(value []byte) *Node

NewNodeWithValue initializes a leaf node.

func ProofTree 

func ProofTree(v HashRoot) (*Node, error)

ProofTree hashes a HashRoot object with a Hasher from the default HasherPool

func TreeFromChunks 

func TreeFromChunks(chunks [][]byte) (*Node, error)

TreeFromChunks constructs a tree from leaf values. The number of leaves should be a power of 2.

func TreeFromNodes 

func TreeFromNodes(leaves []*Node) (*Node, error)

TreeFromNodes constructs a tree from leaf nodes. This is useful for merging subtrees. The number of leaves should be a power of 2.

func TreeFromNodesWithMixin 

func TreeFromNodesWithMixin(leaves []*Node, num, limit int) (*Node, error)

func (*Node) Get 

func (n *Node) Get(index int) (*Node, error)

Get fetches a node with the given general index.

func (*Node) Hash 

func (n *Node) Hash() []byte

Hash returns the hash of the subtree with the given Node as its root. If root has no children, it returns root's value (not its hash).

func (*Node) Prove 

func (n *Node) Prove(index int) (*Proof, error)

Prove returns a list of sibling values and hashes needed to compute the root hash for a given general index.

func (*Node) ProveMulti 

func (n *Node) ProveMulti(indices []int) (*Multiproof, error)

func (*Node) Show 

func (n *Node) Show(maxDepth int)

type Proof 

type Proof struct {
	Index  int
	Leaf   []byte
	Hashes [][]byte
}

Proof represents a merkle proof against a general index.

type Unmarshaler 

type Unmarshaler interface {
	UnmarshalSSZ(buf []byte) error
}

Unmarshaler is the interface implemented by types that can unmarshal a SSZ description of themselves

type Wrapper 

type Wrapper struct {
	// contains filtered or unexported fields
}

func (*Wrapper) AddBytes 

func (w *Wrapper) AddBytes(b []byte)

func (*Wrapper) AddEmpty 

func (w *Wrapper) AddEmpty()

func (*Wrapper) AddNode 

func (w *Wrapper) AddNode(n *Node)

func (*Wrapper) AddUint16 

func (w *Wrapper) AddUint16(i uint16)

func (*Wrapper) AddUint32 

func (w *Wrapper) AddUint32(i uint32)

func (*Wrapper) AddUint64 

func (w *Wrapper) AddUint64(i uint64)

func (*Wrapper) AddUint8 

func (w *Wrapper) AddUint8(i uint8)

func (*Wrapper) Append 

func (w *Wrapper) Append(i []byte)

func (*Wrapper) AppendBytes32 

func (w *Wrapper) AppendBytes32(b []byte)

func (*Wrapper) AppendUint64 

func (w *Wrapper) AppendUint64(i uint64)

func (*Wrapper) AppendUint8 

func (w *Wrapper) AppendUint8(i uint8)

func (*Wrapper) Commit 

func (w *Wrapper) Commit(i int)

func (*Wrapper) CommitWithMixin 

func (w *Wrapper) CommitWithMixin(i, num, limit int)

func (*Wrapper) FillUpTo32 

func (w *Wrapper) FillUpTo32()

func (*Wrapper) Hash 

func (w *Wrapper) Hash() []byte

func (*Wrapper) Index 

func (w *Wrapper) Index() int

func (*Wrapper) Merkleize 

func (w *Wrapper) Merkleize(indx int)

func (*Wrapper) MerkleizeWithMixin 

func (w *Wrapper) MerkleizeWithMixin(indx int, num, limit uint64)

func (*Wrapper) Node 

func (w *Wrapper) Node() *Node

func (*Wrapper) PutBitlist 

func (w *Wrapper) PutBitlist(bb []byte, maxSize uint64)

func (*Wrapper) PutBool 

func (w *Wrapper) PutBool(b bool)

func (*Wrapper) PutBytes 

func (w *Wrapper) PutBytes(b []byte)

func (*Wrapper) PutUint16 

func (w *Wrapper) PutUint16(i uint16)

func (*Wrapper) PutUint32 

func (w *Wrapper) PutUint32(i uint32)

func (*Wrapper) PutUint64 

func (w *Wrapper) PutUint64(i uint64)

func (*Wrapper) PutUint8 

func (w *Wrapper) PutUint8(i uint8)

Source Files

View all Source files

Directories

Path Synopsis
Code generated by fastssz.
 Code generated by fastssz.
generator
testcases
Code generated by fastssz.
 Code generated by fastssz.
testcases/other
Code generated by fastssz.
 Code generated by fastssz.
testcases/other2
version
Code generated by fastssz.
 Code generated by fastssz.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.