hashstructure

README

hashstructure

hashstructure is a Go library for creating a unique hash value for arbitrary values in Go.

This can be used to key values in a hash (for use in a map, set, etc.) that are complex. The most common use case is comparing two values without sending data across the network, caching values locally (de-dup), and so on.

Features

• Hash any arbitrary Go value, including complex types.

• Tag a struct field to ignore it and not affect the hash value.

• Tag a slice type struct field to treat it as a set where ordering doesn't affect the hash code but the field itself is still taken into account to create the hash value.

• Optionally, specify a custom hash function to optimize for speed, collision avoidance for your data set, etc.

• Optionally, hash the output of .String() on structs that implement fmt.Stringer, allowing effective hashing of time.Time

• Optionally, override the hashing process by implementing Hashable.

Installation

Standard go get:

$ go get github.com/cubiest/hashstructure/v2

Note on v2: It is highly recommended you use the "v2" release since this fixes some significant hash collisions issues from v1. In practice, we used v1 for many years in real projects at HashiCorp and never had issues, but it is highly dependent on the shape of the data you're hashing and how you use those hashes.

When using v2+, you can still generate weaker v1 hashes by using the FormatV1 format when calling Hash.

Usage & Example

For usage and examples see the Godoc.

A quick code example is shown below:

type ComplexStruct struct {
    Name     string                 `hash:"name"`
    Age      uint                   `hash:"age"`
    Metadata map[string]interface{} `hash:"meta"`

    Aux *ComplexStruct `hash:"-"` // Ingore for hashing.
}

v := ComplexStruct{
    Name: "Lucy-May",
    Age:  7,
    Metadata: map[string]interface{}{
        "pets":      true,
        "location": "Australia",
        "siblings": []string{"Kate", "Clara", "Tob", "Ben"},
    },
}

// By default, "hash" is expected for the tag name,
// but can be overridden using hashstructure.HashOptions.TagName.
hash, err := hashstructure.Hash(v, hashstructure.FormatV2, nil)
if err != nil {
    panic(err)
}

fmt.Printf("%d", hash)
// Output:
// 1065311810974221966

Documentation

Index

• func Hash(v interface{}, format Format, opts *HashOptions) (uint64, error)
• type ErrFormat
  • func (*ErrFormat) Error() string
• type ErrNotStringer
  • func (ens *ErrNotStringer) Error() string
• type Format
• type HashOptions
• type Hashable
• type Includable
• type IncludableMap

Examples

• Hash
• Hash (V1)

Functions

func Hash

func Hash(v interface{}, format Format, opts *HashOptions) (uint64, error)

Hash returns the hash value of an arbitrary value.

If opts is nil, then default options will be used. See HashOptions for the default values. The same *HashOptions value cannot be used concurrently. None of the values within a *HashOptions struct are safe to read/write while hashing is being done.

The "format" is required and must be one of the format values defined by this library. You should probably just use "FormatV2". This allows generated hashes uses alternate logic to maintain compatibility with older versions.

Notes on the value:

• Unexported fields on structs are ignored and do not affect the hash value.

• Adding an exported field to a struct with the zero value will change the hash value.

For structs, the hashing can be controlled using tags. For example:

struct {
    Name string
    UUID string `hash:"ignore"`
}

The available tag values are:

• "ignore" or "-" - The field will be ignored and not affect the hash code.

• "set" - The field will be treated as a set, where ordering doesn't affect the hash code. This only works for slices.

• "string" - The field will be hashed as a string, only works when the field implements fmt.Stringer

Example

type ComplexStruct struct {
	Name     string
	Age      uint
	Metadata map[string]interface{}
}

v := ComplexStruct{
	Name: "mitchellh",
	Age:  64,
	Metadata: map[string]interface{}{
		"car":      true,
		"location": "California",
		"siblings": []string{"Bob", "John"},
	},
}

hash, err := Hash(v, FormatV2, nil)
if err != nil {
	panic(err)
}

fmt.Printf("%d", hash)

Output:

1839806922502695369

Example (V1)

type ComplexStruct struct {
	Name     string
	Age      uint
	Metadata map[string]interface{}
}

v := ComplexStruct{
	Name: "mitchellh",
	Age:  64,
	Metadata: map[string]interface{}{
		"car":      true,
		"location": "California",
		"siblings": []string{"Bob", "John"},
	},
}

hash, err := Hash(v, FormatV1, nil)
if err != nil {
	panic(err)
}

fmt.Printf("%d", hash)

Output:

6691276962590150517

Types

type ErrFormat

type ErrFormat struct{}

ErrFormat is returned when an invalid format is given to the Hash function.

func (*ErrFormat) Error

func (*ErrFormat) Error() string

type ErrNotStringer

type ErrNotStringer struct {
	Field string
}

ErrNotStringer is returned when there's an error with hash:"string"

func (*ErrNotStringer) Error

func (ens *ErrNotStringer) Error() string

Error implements error for ErrNotStringer

type Format

type Format uint

Format specifies the hashing process used. Different formats typically generate different hashes for the same value and have different properties.

const (

	// FormatV1 is the format used in v1.x of this library. This has the
	// downsides noted in issue #18 but allows simultaneous v1/v2 usage.
	FormatV1 Format

	// FormatV2 is the current recommended format and fixes the issues
	// noted in FormatV1.
	FormatV2
)

type HashOptions

type HashOptions struct {
	// Hasher is the hash function to use. If this isn't set, it will
	// default to FNV.
	Hasher hash.Hash64

	// TagName is the struct tag to look at when hashing the structure.
	// By default this is "hash".
	TagName string

	// ZeroNil is flag determining if nil pointer should be treated equal
	// to a zero value of pointed type. By default this is false.
	ZeroNil bool

	// IgnoreZeroValue is determining if zero value fields should be
	// ignored for hash calculation.
	IgnoreZeroValue bool

	// SlicesAsSets assumes that a `set` tag is always present for slices.
	// Default is false (in which case the tag is used instead)
	SlicesAsSets bool

	// UseStringer will attempt to use fmt.Stringer always. If the struct
	// doesn't implement fmt.Stringer, it'll fall back to trying usual tricks.
	// If this is true, and the "string" tag is also set, the tag takes
	// precedence (meaning that if the type doesn't implement fmt.Stringer, we
	// panic)
	UseStringer bool
}

HashOptions are options that are available for hashing.

type Hashable

type Hashable interface {
	Hash() (uint64, error)
}

Hashable is an interface that can optionally be implemented by a struct to override the hash value. This value will override the hash value for the entire struct. Entries in the struct will not be hashed.

type Includable

type Includable interface {
	HashInclude(field string, v interface{}) (bool, error)
}

Includable is an interface that can optionally be implemented by a struct. It will be called for each field in the struct to check whether it should be included in the hash.

type IncludableMap

type IncludableMap interface {
	HashIncludeMap(field string, k, v interface{}) (bool, error)
}

IncludableMap is an interface that can optionally be implemented by a struct. It will be called when a map-type field is found to ask the struct if the map item should be included in the hash.