maps

README

vespucci

vespucci implements utility functions for transforming values into a representation using only the simple types used in golang's mapping to JSON:

• map[string]interface{}
• []interface{}
• float64
• string
• bool
• nil

This process is referred to as "normalizing" the value. The package also offers many useful utility functions for working with normalized values:

• Contains
• Equivalent
• Conflicts
• Keys
• Get
• Merge
• Empty
• Transform

These functions are useful when dealing with business values which may be represented as structs, maps, or JSON, depending on the context.

Normalization will convert maps, slices, and primitives directly to one of the types above. For other values, it will fall back on marshaling the value to JSON, then unmarshaling it into interface{}. Raw JSON can be passed as a value by wrapping it in json.RawMessage:

v, err := Normalize(json.RawMessage(b))

The mapstest package provides useful testing assertions, built on top of Contains and Equivalent. These are useful for asserting whether a value is approximately equal to an expected value. For example:

jsonResp := httpget()
mapstest.AssertContains(t, json.RawMessage(jsonResp), map[string]interface{}{
  "color":"red",
  "size":1,
})

Because both values are normalized before comparison, either value can be raw JSON, a struct, a map, a slice, or a primitive value. Normalization is recursive, so any of these types can be nested within each other. And there are useful assertion options controlling how loose the match can be.
For example:

v1 := map[string]interface{}{
  "color":"bigred",
  "size":0,
  "createdAt":time.Now().String,
}

v2 := map[string]interface{}{
  "color":"red",
  "size":1,
  "createdAt": time.Now,
}

mapstest.AssertContains(t, v1, v2, 
  maps.StringContains(),       // allows "bigred" to match "red"
  maps.EmptyValuesMatchAny(),  // allows size to match.  The presence and 
                               // and type v2.size is checked
  maps.AllowTimeDelta(time.Second),  // allows v1.createdAt to be parsed into
                                     // a time.Time, and allows some skew between
                                     // v1.createdAt and v2.createdAt
)

Documentation

Overview

Package maps is a set of utility functions for working with maps. Generally, maps and slices of any kind will work, but performance is optimized for maps returned by json.Unmarshal(b, &interface{}). If all the maps are map[string]interface{}, and all the slices are []interface{}, and all the rest of the values are primitives, then reflection is avoided.

Index

• Variables
• func Conflicts(v1, v2 interface{}) bool
• func Contains(v1, v2 interface{}, options ...ContainsOption) bool
• func Empty(v interface{}) bool
• func Equivalent(v1, v2 interface{}, options ...ContainsOption) bool
• func Get(v interface{}, path string, opts ...NormalizeOption) (interface{}, error)
• func Keys(m map[string]interface{}) (keys []string)
• func Merge(v1, v2 interface{}, opts ...NormalizeOption) interface{}
• func Normalize(v1 interface{}, opts ...NormalizeOption) (interface{}, error)
• func NormalizeWithOptions(v interface{}, opt NormalizeOptions) (interface{}, error)
• func Transform(v interface{}, transformer func(in interface{}) (interface{}, error), ...) (interface{}, error)
• type ContainsOption
  • func AllowTimeDelta(d time.Duration) ContainsOption
  • func EmptyValuesMatchAny() ContainsOption
  • func IgnoreTimeZones(b bool) ContainsOption
  • func ParseTimes() ContainsOption
  • func RoundTimes(d time.Duration) ContainsOption
  • func StringContains() ContainsOption
  • func Trace(s *string) ContainsOption
  • func TruncateTimes(d time.Duration) ContainsOption
• type Match
  • func ContainsMatch(v1, v2 interface{}, options ...ContainsOption) Match
  • func EquivalentMatch(v1, v2 interface{}, options ...ContainsOption) Match
• type NormalizeOption
  • func Copy(b bool) NormalizeOption
  • func Deep(b bool) NormalizeOption
  • func Marshal(b bool) NormalizeOption
  • func NormalizeTime(b bool) NormalizeOption
• type NormalizeOptionFunc
  • func (f NormalizeOptionFunc) Apply(options *NormalizeOptions)
• type NormalizeOptions
• type Path
  • func ParsePath(path string) (Path, error)
  • func (p Path) String() string

Variables

var EmptyMapValuesMatchAny = EmptyValuesMatchAny

EmptyMapValuesMatchAny is an alias for EmptyValuesMatchAny.

var ErrStop = errors.New("stop")

ErrStop can be returned by transform functions to end recursion early. The Transform function will not return an error.

var IndexOutOfBoundsError = merry.New("Index out of bounds")

IndexOutOfBoundsError indicates the index doesn't exist in the slice.

var PathNotFoundError = merry.New("Path not found")

PathNotFoundError indicates the requested path was not present in the value.

var PathNotMapError = merry.New("Path not map")

PathNotMapError indicates the value at the path is not a map.

var PathNotSliceError = merry.New("Path not slice")

PathNotSliceError indicates the value at the path is not a slice.

Functions

func Conflicts

func Conflicts(v1, v2 interface{}) bool

Conflicts returns true if trees share common key paths, but the values at those paths are not equal. i.e. if the two maps were merged, no values would be overwritten conflicts == !contains(v1, v2) && !excludes(v1, v2) conflicts == !contains(merge(v1, v2), v1)

func Contains

func Contains(v1, v2 interface{}, options ...ContainsOption) bool

Contains tests whether v1 "contains" v2. The notion of containment is based on postgres' JSONB containment operators.

A map v1 "contains" another map v2 if v1 has contains all the keys in v2, and if the values in v2 are contained by the corresponding values in v1.

{"color":"red"} contains {}
{"color":"red"} contains {"color":"red"}
{"color":"red","flavor":"beef"} contains {"color":"red"}
{"labels":{"color":"red","flavor":"beef"}} contains {"labels":{"flavor":"beef"}}
{"tags":["red","green","blue"]} contains {"tags":["red","green"]}

A scalar value v1 contains value v2 if they are equal.

5 contains 5
"red" contains "red"

A slice v1 contains a slice v2 if all the values in v2 are contained by at least one value in v1:

["red","green"] contains ["red"]
["red"] contains ["red","red","red"]
// In this case, the single value in v1 contains each of the values
// in v2, so v1 contains v2
[{"type":"car","color":"red","wheels":4}] contains [{"type":"car"},{"color","red"},{"wheels":4}]

A slice v1 also can contain a *scalar* value v2:

["red"] contains "red"

A struct v1 contains a struct v2 if they are deeply equal (using reflect.DeepEquals)

func Empty

func Empty(v interface{}) bool

Empty returns true if v is nil, empty, or a zero value.

If v is a pointer, it is empty if the pointer is nil or invalid, but not empty if it points to a value, even if that value is zero. For example:

Empty(0)  // true
i := 0
Empty(&i) // false
Empty(Widget{}) // true, zero value
Empty(&Widget{}) // false, non-nil pointer

Maps, slices, arrays, and channels are considered empty if their length is zero.

Strings are empty if they contain nothing but whitespace.

func Equivalent

func Equivalent(v1, v2 interface{}, options ...ContainsOption) bool

Equivalent checks if v1 and v2 are approximately deeply equal to each other. It takes the same comparison options as Contains. It is equivalent to:

Equivalent(v1, v2) == Contains(v1, v2) && Contains(v2, v1)

ContainsOptions which only work in one direction, like StringContains, will always treat v2 as a pattern or rule to match v1 against. For example:

b := Equivalent("thefox", "fox", StringContains())

b is true because "thefox" contains "fox", even though the inverse is not true

func Get

func Get(v interface{}, path string, opts ...NormalizeOption) (interface{}, error)

Get extracts the value at path from v. Path is in the form:

response.things[2].color.red

You can use `merry` to test the types of return errors:

_, err := maps.Get("","")
if merry.Is(err, maps.PathNotFoundError) {
  ...

Returns PathNotFoundError if the next key in the path is not found.

Returns PathNotMapError if evaluating a key against a value which is not a map (e.g. a slice or a primitive value, against which we can't evaluate a key name).

Returns IndexOutOfBoundsError if evaluating a slice index against a slice value, and the index is out of bounds.

Returns PathNotSliceError if evaluating a slice index against a value which isn't a slice.

func Keys

func Keys(m map[string]interface{}) (keys []string)

Keys returns a slice of the keys in the map

func Merge

func Merge(v1, v2 interface{}, opts ...NormalizeOption) interface{}

Merge returns a new map, which is the deep merge of the normalized values of v1 and v2.

Values in v2 override values in v1.

Slices are merged simply by adding any v2 values which aren't already in v1's slice. This won't do anything fancy with slices that have duplicate values. Order is ignored. E.g.:

[5, 6, 7] + [5, 5, 5, 4] = [5, 6, 7, 4]

The return value is a copy. v1 and v2 are not modified.

func Normalize

func Normalize(v1 interface{}, opts ...NormalizeOption) (interface{}, error)

Normalize recursively converts v1 into a tree of maps, slices, and primitives. The types in the result will be the types the json package uses for unmarshalling into interface{}. The rules are:

1. All maps with string keys will be converted into map[string]interface{} 2. All slices will be converted to []interface{} 3. All primitive numeric types will be converted into float64 4. string, bool, and nil are unmodified 5. All other values will be converted into the above types by doing a json.Marshal and Unmarshal

Values in v1 will be modified in place if possible

func NormalizeWithOptions

func NormalizeWithOptions(v interface{}, opt NormalizeOptions) (interface{}, error)

NormalizeWithOptions does the same as Normalize, but with options.

func Transform

func Transform(v interface{}, transformer func(in interface{}) (interface{}, error), opts ...NormalizeOption) (interface{}, error)

Transform applies a transformation function to each value in tree. Values are normalized before being passed to the transformer function. Any maps and slices are passed to the transform function as the whole value first, then each child value of the map/slice is passed to the transform function.

The value returned by the transformer will replace the original value.

If the transform function returns a non-primitive value, it will recurse into the new value.

If the transformer function returns the error ErrStop, the process will abort with no error.

Types

type ContainsOption

type ContainsOption func(ctx *containsCtx)

ContainsOption is an option which modifies the behavior of the Contains() function

func AllowTimeDelta

func AllowTimeDelta(d time.Duration) ContainsOption

AllowTimeDelta configures the precision of time comparison. Time values will be considered equal if the difference between the two values is less than d.

Implies ParseTimes

func EmptyValuesMatchAny

func EmptyValuesMatchAny() ContainsOption

EmptyValuesMatchAny is a ContainsOption which allows looser matching of empty values. If set, a value in v1 will match a value in v2 if:

- v1 contains v2 - OR v2 is nil - OR v2 is the zero value of the type of v1's value

This is convenient when testing whether a struct contains another struct. Structs are normalized by marshalling them to JSON. Fields which don't have the `omitempty` option will appear in the normalized v2 value as map keys with zero values. Using this option will allow that to match.

This option can also be used to test for the presence of keys in v1 without needing to test the value:

v1 := map[string]interface{}{"color":"blue"}
v2 := map[string]interface{}{"color":nil}
Contains(v1, v2)  // false
Contains(v1, v2, EmptyMapValuesMatchAny()) // true
v1 := map[string]interface{}{}
Contains(v1, v2, EmptyMapValuesMatchAny()) // false, because v1 doesn't have "color" key

Another use is testing the general type of the value:

v1 := map[string]interface{}{"size":5}
v2 := map[string]interface{}{"size":0}
Contains(v1, v2)  // false
Contains(v1, v2, EmptyMapValuesMatchAny()) // true
v2 := map[string]interface{}{"size":""}
Contains(v1, v2, EmptyMapValuesMatchAny()) // false, because type of value doesn't match (v1: number, v2: string)

func IgnoreTimeZones

func IgnoreTimeZones(b bool) ContainsOption

IgnoreTimeZones will ignore the time zones of time values (otherwise the time zones must match).

Implies ParseTimes

func ParseTimes

func ParseTimes() ContainsOption

ParseTimes enables special processing for date values. Contains typically marshals time.Time values to a string before comparison. This means the EmptyValuesMatchAny() option will not work as expected for time values.

When ParseTimes is specified, after the values are normalized to strings, the code will attempt to parse any string values back into time.Time values. This allows correct processing of the time.Time zero values.

func RoundTimes

func RoundTimes(d time.Duration) ContainsOption

RoundTimes will round time values (see time.Time#Round)

Implies ParseTimes

func StringContains

func StringContains() ContainsOption

StringContains is a ContainsOption which uses strings.Contains(v1, v2) to test for string containment.

Without this option, strings (like other primitive values) must match exactly.

Contains("brown fox", "fox") // false
Contains("brown fox", "fox", StringContains()) // true

func Trace

func Trace(s *string) ContainsOption

Trace sets `s` to a string describing the path to the values where containment was false. Helps debugging why one value doesn't contain another. Sample output:

-> v1: map[time:2017-03-03T14:08:30.097698864-05:00]
-> v2: map[time:0001-01-01T00:00:00Z]
-> "time"
--> v1: 2017-03-03T14:08:30.097698864-05:00
--> v2: 0001-01-01T00:00:00Z

If `s` is nil, it does nothing.

func TruncateTimes

func TruncateTimes(d time.Duration) ContainsOption

TruncateTimes will truncate time values (see time.Time#Truncate)

Implies ParseTimes

type Match

type Match struct {
	Matches bool
	Path    string
	V1      interface{}
	V2      interface{}
	Error   error
	Message string
}

Match is the result of ContainsMatch or EquivalentMatch. It reports whether the match succeeded, and if not, where and why it failed.

If the match succeed, Matches will be true and the rest of the fields will be empty. Otherwise, V1, V2, and Path will be set to the values and location where the match failed. Message will be set to an explanation of the failure. And if the failure was due to an error, Error will be set.

func ContainsMatch

func ContainsMatch(v1, v2 interface{}, options ...ContainsOption) Match

ContainsMatch is the same as Contains, but returns the normalized versions of v1 and v2 used in the comparison.

func EquivalentMatch

func EquivalentMatch(v1, v2 interface{}, options ...ContainsOption) Match

EquivalentMatch is the same as Equivalent, but returns the normalized versions of v1 and v2 used in the comparison.

type NormalizeOption

type NormalizeOption interface {
	Apply(*NormalizeOptions)
}

NormalizeOption is an option function for the Normalize operation.

func Copy

func Copy(b bool) NormalizeOption

Copy causes Normalize to return a copy of the original value.

func Deep

func Deep(b bool) NormalizeOption

Deep causes normalization to recurse.

func Marshal

func Marshal(b bool) NormalizeOption

Marshal allows normalization to resort to JSON marshaling if the value can't be directly coerced into one of the standard types.

func NormalizeTime

func NormalizeTime(b bool) NormalizeOption

NormalizeTime cause normalization to preserve time.Time values instead of converting them to strings.

type NormalizeOptionFunc

type NormalizeOptionFunc func(*NormalizeOptions)

NormalizeOptionFunc is a function which implements NormalizeOption.

func (NormalizeOptionFunc) Apply

func (f NormalizeOptionFunc) Apply(options *NormalizeOptions)

Apply implements NormalizeOption.

type NormalizeOptions

type NormalizeOptions struct {
	// Make copies of all maps and slices.  The result will not share
	// any maps or slices with input value.
	Copy bool

	// if values are encountered which are not primitives, maps, or slices, attempt to
	// turn them into primitives, maps, and slices by running through json.Marshal and json.Unmarshal
	Marshal bool

	// Perform the operation recursively.  If false, only v is normalized, but nested values are not
	Deep bool

	// Treat time.Time values as an additional normalized type.  If false, time values are converted
	// to json's standard string formatted time.  If true, time values are preserved as time.Time, and
	// string values are coerced to time if they are in the JSON RFC3339 format.
	NormalizeTime bool
}

NormalizeOptions are options for the Normalize function.

type Path

type Path []interface{}

Path is a slice of either strings or slice indexes (ints).

func ParsePath

func ParsePath(path string) (Path, error)

ParsePath parses a string path into a Path slice. String paths look like:

user.name.first
user.addresses[3].street

func (Path) String

func (p Path) String() string

String implements the Stringer interface. It returns the string representation of a Path. Path.String() and ParsePath() are inversions of each other.