HashStablePack

module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 25, 2019 License: MIT

README

Hash Stable Pack

This is a code generation tool for QUICK struct content compare or hash computation.

For
  • Quick compare nested struct without reflection (10~20 times faster)

    BenchmarkCompare/benchmark_reflect-8         	  100000	      20074 ns/op //reflect.DeepEqual
    BenchmarkCompare/benchmark_hsp-8             	  500000	       2322 ns/op
    BenchmarkCompare/benchmark_hsp_1_cached-8    	 1000000	       1101 ns/op
    BenchmarkCompare/benchmark_hsp_both_cached-8   100000000	       11.2 ns/op
    

    bench cases see here

  • Quick calculation of struct hash or signature without reflection. used in SQLess for block hash.

How

Basically it will generate an MarshalHash method which follow the MessagePack Spec but :

  1. Without the struct key.
  2. Stable output of map.
  3. Can be used to compare different type with same hsp tag.

That is the following 2 structs with different member name

For more: see test cases

//go:generate hsp

type Person1 struct {
	Name       string
	Age        int
	Address    string
	Map        map[string]int
	unexported bool             // this field is ignored
	Unexported string `hsp:"-"` // this field is ignored
}

type Person2 struct {
	Name       string
	Address    string
	Age        int
	Map222     map[string]int `hspack:"Map"`
	unexported bool             // this field is ignored
	Unexported string `hsp:"-"` // this field is ignored
}

But with the same name and content of exported member, MarshalHash will produce the same bytes array:

package person

import (
	"bytes"
	"testing"
)

func TestMarshalHashAccountStable3(t *testing.T) {
	p1 := Person1{
		Name:       "Auxten",
		Address:    "@SQLess.io",
		Age:        70,
		Map:         map[string]int{"ss": 2, "s": 1, "sss": 3},
		unexported: false,
	}
	p2 := Person2{
		Name:       "Auxten",
		Address:    "@SQLess.io",
		Age:        70,
		Map222:      map[string]int{"ss": 2, "s": 1, "sss": 3},
		unexported: true,
	}
	bts1, err := p1.MarshalHash()
	if err != nil {
		t.Fatal(err)
	}
	bts2, err := p2.MarshalHash()
	if err != nil {
		t.Fatal(err)
	}
	if !bytes.Equal(bts1, bts2) {
		t.Fatal("hash not stable")
	}
}

the order of struct member is sorted by struct tag (if not, use name)

You can read more about MessagePack in the wiki, or at msgpack.org.

Why?
  • Use Go as your schema language
  • Performance
Why not?
  • MessagePack: member name is unnecessary, different implementation may add some fields which made result undetermined. And also golang's map...
  • Prorobuf: struct must defined in proto language, and other limitations discussed here
Quickstart
  1. Quick Install
go get -u github.com/SQLess/HashStablePack/hsp
  1. Add tag for source In a source file, include the following directive:
//go:generate hsp
  1. Run go generate
go generate ./...

The hsp command will generate serialization methods for all exported type declarations in the file.

By default, the code generator will only generate MarshalHash and Msgsize method

func (z *Test) MarshalHash() (o []byte, err error)
func (z *Test) Msgsize() (s int)
Features
  • Extremely fast generated code
  • Test and benchmark generation
  • Support for complex type declarations
  • Native support for Go's time.Time, complex64, and complex128 types
  • Support for arbitrary type system extensions
  • File-based dependency model means fast codegen regardless of source tree size.
License

This lib is inspired by https://github.com/tinylib/msgp Most Code is diverted from https://github.com/tinylib/msgp, but It's an total different lib for usage. So I created a new project instead of forking it.

Directories

Path Synopsis
This package is the support library for the hsp code generator (http://github.com/SQLess/HashStablePack).
This package is the support library for the hsp code generator (http://github.com/SQLess/HashStablePack).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL