conjungo

README

conjungo

A merge utility designed for flexibility and customizability. The library has a single simple point of entry that works out of the box for most basic use cases. From there, customizations can be made to the way two items are merged to fit your specific needs.

Merge any two things of the same type, including maps, slices, structs, and even basic types like string and int. By default, the target value will be overwritten by the source. If the overwrite option is turned off, only new values in source that do not already exist in target will be added.

If you would like to change the way two items of a particular type get merged, custom merge functions can be defined for any type or kind (see below).

Why Conjungo?

The definition of Conjungo:

I.v. a., to bind together, connect, join, unite (very freq. in all perr. and species of composition); constr. with cum, inter se, the dat., or the acc. only; trop. also with ad.

Reference: Latin Dictionary...

There are other merge libraries written in go, but none of them have the flexibility of this one. If you simply need to merge two things, a default set of merge functions are defined for merging maps, slices, and structs like most other libraries. But, if the way these default functions are defined does not meet your needs, Conjungo provides the ability to define your own merge functions. For instance, the default behavior when merging two integers is to replace the target with the source, but if you'd like to redefine that, you can write a custom merge function that is used when assessing integers. A custom function could add the two integers and return the result, or return the larger of the two integers. You could define a custom merge function for a specific struct type in your code, and define how that gets merged. The customizability of how things get merged is the focus of Conjungo.

The need for this library arose when we were merging large custom structs. We found that there was no single library that merged all the parts of the struct in the way that we needed. We had struct fields that were pointers to sub structs and maps that needed to be followed instead of simply replaced. We had slices that needed to be appended but also deduped. Conjungo solves these types of problems by allowing custom functions to be defined to handle each type.

Setup

To get conjungo:

go get github.com/InVisionApp/conjungo

We recommend that you vendor it within your project. We chose to use govendor.

govendor fetch github.com/InVisionApp/conjungo

Usage

Simple Merge

Merge two structs together:

type Foo struct {
	Name    string
	Size    int
	Special bool
	SubMap  map[string]string
}

targetStruct := Foo{
	Name:    "target",
	Size:    2,
	Special: false,
	SubMap:  map[string]string{"foo": "unchanged", "bar": "orig"},
}

sourceStruct := Foo{
	Name:    "source",
	Size:    4,
	Special: true,
	SubMap:  map[string]string{"bar": "newVal", "safe": "added"},
}

err := conjungo.Merge(&targetStruct, sourceStruct, nil)
if err != nil {
	log.Error(err)
}

results in:

{
  "Name": "source",
  "Size": 4,
  "Special": true,
  "SubMap": {
    "bar": "newVal",
    "foo": "unchanged",
    "safe": "added"
  }
}

Options

Overwrite bool
If true, overwrite a target value with source value even if it already exists

ErrorOnUnexported bool
Unexported fields on a struct can not be set. When a struct contains an unexported field, the default behavior is to treat the entire struct as a single entity and replace according to Overwrite settings.
If this is enabled, an error will be thrown instead.

Custom Merge Functions

Define a custom merge function for a type:

opts := conjungo.NewOptions()
opts.MergeFuncs.SetTypeMergeFunc(
	reflect.TypeOf(0),
	// merge two 'int' types by adding them together
	func(t, s reflect.Value, o *conjungo.Options) (reflect.Value, error) {
		iT, _ := t.Interface().(int)
		iS, _ := s.Interface().(int)
		return reflect.ValueOf(iT + iS), nil
	},
)

x := 1
y := 2

err := conjungo.Merge(&x, y, opts)
if err != nil {
	log.Error(err)
}

// x == 3

Define a custom merge function for a kind:

opts := conjungo.NewOptions()
opts.MergeFuncs.SetKindMergeFunc(
	reflect.TypeOf(struct{}{}).Kind(),
	// merge two 'struct' kinds by replacing the target with the source
	// provides a mechanism to set override = true for just structs
	func(t, s reflect.Value, o *conjungo.Options) (reflect.Value, error) {
		return s, nil
	},
)

Define a custom merge function for a struct type:

	type Foo struct {
		Name string
		Size int
	}

	target := Foo{
		Name: "bar",
		Size: 25,
	}

	source := Foo{
		Name: "baz",
		Size: 35,
	}

	opts := conjungo.NewOptions()
	opts.MergeFuncs.SetTypeMergeFunc(
		reflect.TypeOf(Foo{}),
		// merge two 'int' types by adding them together
		func(t, s reflect.Value, o *conjungo.Options) (reflect.Value, error) {
			tFoo := t.Interface().(Foo)
			sFoo := s.Interface().(Foo)

			// names are merged by concatenating them
			tFoo.Name = tFoo.Name + "." + sFoo.Name
			// sizes are merged by averaging them
			tFoo.Size = (tFoo.Size + sFoo.Size) / 2

			return reflect.ValueOf(tFoo), nil
		},
	)

Define a custom type and a function to merge it:

type jsonString string

var targetJSON jsonString = `
{
  "a": "wrong",
  "b": 1,
  "c": {"bar": "orig", "foo": "unchanged"},
}`

var sourceJSON jsonString = `
{
  "a": "correct",
  "b": 2,
  "c": {"bar": "newVal", "safe": "added"},
}`

opts := conjungo.NewOptions()
opts.MergeFuncs.SetTypeMergeFunc(
	reflect.TypeOf(jsonString("")),
	// merge two json strings by unmarshalling them to maps
	func(t, s reflect.Value, o *conjungo.Options) (reflect.Value, error) {
		targetStr, _ := t.Interface().(jsonString)
		sourceStr, _ := s.Interface().(jsonString)

		targetMap := map[string]interface{}{}
		if err := json.Unmarshal([]byte(targetStr), &targetMap); err != nil {
			return reflect.Value{}, err
		}

		sourceMap := map[string]interface{}{}
		if err := json.Unmarshal([]byte(sourceStr), &sourceMap); err != nil {
			return reflect.Value{}, err
		}

		err := conjungo.Merge(&targetMap, sourceMap, o)
		if err != nil {
			return reflect.Value{}, err
		}

		mergedJSON, err := json.Marshal(targetMap)
		if err != nil {
			return reflect.Value{}, err
		}

		return reflect.ValueOf(jsonString(mergedJSON)), nil
	},
)

See working examples for more details.

Documentation

Index

• func Merge(target, source interface{}, opt *Options) error
• type MergeFunc
• type Options
  • func NewOptions() *Options
  • func (o *Options) SetDefaultMergeFunc(mf MergeFunc)
  • func (o *Options) SetKindMergeFunc(k reflect.Kind, mf MergeFunc)
  • func (o *Options) SetTypeMergeFunc(t reflect.Type, mf MergeFunc)

Functions

func Merge

func Merge(target, source interface{}, opt *Options) error

Merge the given source onto the given target following the options given. The target value must be a pointer. If opt is nil, defaults will be used. If an error occurs during the merge process the target will be unmodified. Merge will accept any two entities, as long as their types are the same. See Options and MergeFunc for further customization possibilities.

Types

type MergeFunc

type MergeFunc func(target, source reflect.Value, o *Options) (reflect.Value, error)

A MergeFunc defines how two items are merged together. It should accept a reflect.Value representation of a target and source, and return the final merged product. The value returned from the function will be written directly to the parent value, as long as there is no error. Options are also passed in, and it is the responsibility of the function to honor these options and handle any variations in behavior that should occur.

type Options

type Options struct {
	// Overwrite a target value with source value even if it already exists
	Overwrite bool

	// Unexported fields on a struct can not be set. When a struct contains an unexported
	// field, the default behavior is to treat the entire struct as a single entity and
	// replace according to Overwrite settings. If this is enabled, an error will be thrown instead.
	//
	// Note: this is used by the default mergeStruct function, and may not apply if that is
	// overwritten with a custom function. Custom struct merge functions should consider
	// using this value as well.
	ErrorOnUnexported bool

	// To be used by merge functions to pass values down into recursive calls freely
	Context context.Context
	// contains filtered or unexported fields
}

Options is used to determine the behavior of a merge. It also holds the collection of functions used to determine merge behavior of various types. Always use NewOptions() to generate options and then modify as needed.

func NewOptions

func NewOptions() *Options

NewOptions generates default Options. Overwrite is set to true, and a set of default merge function definitions are added.

func (*Options) SetDefaultMergeFunc

func (o *Options) SetDefaultMergeFunc(mf MergeFunc)

SetDefaultMergeFunc is used to define a default merge func that will be used as a fallback when there is no specific merge behavior defined for a given item. If using NewOptions(), a very basic default merge function is predefined which will return the source in overwrite mode and the target otherwise. Use this to define custom default behavior when the simple case is not sufficient.

func (*Options) SetKindMergeFunc

func (o *Options) SetKindMergeFunc(k reflect.Kind, mf MergeFunc)

SetKindMergeFunc is used to define a custom merge func that will be used to merge two items of a particular kind. Accepts reflect.Kind and the MergeFunc to merge it. This is useful for defining more general merge behavior, for instance merge all maps or structs in a particular way. A default merge behavior is predefined for map, slice and struct when using NewOptions()

func (*Options) SetTypeMergeFunc

func (o *Options) SetTypeMergeFunc(t reflect.Type, mf MergeFunc)

SetTypeMergeFunc is used to define a custom merge func that will be used to merge two items of a particular type. Accepts the reflect.Type representation of the type and the MergeFunc to merge it. This is useful for defining specific merge behavior of things such as specific struct types