tmc

package module
v0.5.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 2, 2019 License: MIT Imports: 9 Imported by: 7

README

Codec for a Typed Map

Provides round-trip serialization of typed Go maps.

How to Use

See the GoDoc for some basic examples and how to configure custom codec, adapters etc.

Why?

Text based serialization formats like JSON and YAML are convenient, but when used with Go maps, most type information gets lost in translation.

Listed below is a round-trip example in JSON (see https://play.golang.org/p/zxt-wi4Ljz3 for a runnable version):

package main

import (
	"encoding/json"
	"log"
	"math/big"
	"time"

	"github.com/kr/pretty"
)

func main() {
	mi := map[string]interface{}{
		"vstring":   "Hello",
		"vint":      32,
		"vrat":      big.NewRat(1, 2),
		"vtime":     time.Now(),
		"vduration": 3 * time.Second,
		"vsliceint": []int{1, 3, 4},
		"nested": map[string]interface{}{
			"vint":      55,
			"vduration": 5 * time.Second,
		},
		"nested-typed-int": map[string]int{
			"vint": 42,
		},
		"nested-typed-duration": map[string]time.Duration{
			"v1": 5 * time.Second,
			"v2": 10 * time.Second,
		},
	}

	data, err := json.Marshal(mi)
	if err != nil {
		log.Fatal(err)
	}
	m := make(map[string]interface{})
	if err := json.Unmarshal(data, &m); err != nil {
		log.Fatal(err)
	}

	pretty.Print(m)

}

This prints:

map[string]interface {}{
    "vint":      float64(32),
    "vrat":      "1/2",
    "vtime":     "2009-11-10T23:00:00Z",
    "vduration": float64(3e+09),
    "vsliceint": []interface {}{
        float64(1),
        float64(3),
        float64(4),
    },
    "vstring": "Hello",
    "nested":  map[string]interface {}{
        "vduration": float64(5e+09),
        "vint":      float64(55),
    },
    "nested-typed-duration": map[string]interface {}{
        "v2": float64(1e+10),
        "v1": float64(5e+09),
    },
    "nested-typed-int": map[string]interface {}{
        "vint": float64(42),
    },
}

And that is very different from the origin:

  • All numbers are now float64
  • time.Duration is also float64
  • time.Now and *big.Rat are strings
  • Slices are []interface {}, maps map[string]interface {}

So, for structs, you can work around some of the limitations above with custom MarshalJSON, UnmarshalJSON, MarshalText and UnmarshalText.

For the commonly used flexible and schema-lessmap[string]interface {} this is, as I'm aware of, not an option.

Using this library, the above can be written to (see https://play.golang.org/p/PlDetQP5aWd for a runnable example):

package main

import (
	"log"
	"math/big"
	"time"

	"github.com/bep/tmc"

	"github.com/kr/pretty"
)

func main() {
	mi := map[string]interface{}{
		"vstring":   "Hello",
		"vint":      32,
		"vrat":      big.NewRat(1, 2),
		"vtime":     time.Now(),
		"vduration": 3 * time.Second,
		"vsliceint": []int{1, 3, 4},
		"nested": map[string]interface{}{
			"vint":      55,
			"vduration": 5 * time.Second,
		},
		"nested-typed-int": map[string]int{
			"vint": 42,
		},
		"nested-typed-duration": map[string]time.Duration{
			"v1": 5 * time.Second,
			"v2": 10 * time.Second,
		},
	}

	c, err := tmc.New()
	if err != nil {
		log.Fatal(err)
	}

	data, err := c.Marshal(mi)
	if err != nil {
		log.Fatal(err)
	}
	m := make(map[string]interface{})
	if err := c.Unmarshal(data, &m); err != nil {
		log.Fatal(err)
	}

	pretty.Print(m)

}

This prints:

map[string]interface {}{
    "vduration":        time.Duration(3000000000),
    "vint":             int(32),
    "nested-typed-int": map[string]int{"vint":42},
    "vsliceint":        []int{1, 3, 4},
    "vstring":          "Hello",
    "vtime":            time.Time{
        wall: 0x0,
        ext:  63393490800,
        loc:  (*time.Location)(nil),
    },
    "nested": map[string]interface {}{
        "vduration": time.Duration(5000000000),
        "vint":      int(55),
    },
    "nested-typed-duration": map[string]time.Duration{"v1":5000000000, "v2":10000000000},
    "vrat":                  &big.Rat{
        a:  big.Int{
            neg: false,
            abs: {0x1},
        },
        b:  big.Int{
            neg: false,
            abs: {0x2},
        },
    },
}

Performance

The implementation is easy to reason aobut (it uses reflection), but It's not particulary fast and probably not suited for big data. A simple benchmark with a roundtrip marshal/unmarshal is included. On my MacBook it shows:

BenchmarkCodec/JSON_regular-4         	   50000	     27523 ns/op	    6742 B/op	     171 allocs/op
BenchmarkCodec/JSON_typed-4           	   20000	     66644 ns/op	   16234 B/op	     411 allocs/op

Documentation

Overview

Example 
package main

import (
	"fmt"
	"log"

	"github.com/bep/tmc"
)

func main() {
	m1 := map[string]interface{}{"num": 42}
	c, err := tmc.New()
	if err != nil {
		log.Fatal(err)
	}

	data, err := c.Marshal(m1)
	if err != nil {
		log.Fatal(err)
	}

	m2 := make(map[string]interface{})
	err = c.Unmarshal(data, &m2)
	if err != nil {
		log.Fatal(err)
	}
	num := m2["num"]

	fmt.Printf("%v (%T)", num, num)
}

Output:

42 (int)

Index

Examples

Constants

This section is empty.

Variables

View Source 
var (
	// DefaultTypeAdapters contains the default set of type adapters.
	DefaultTypeAdapters = []Adapter{

		NewAdapter(time.Now(), nil, nil),
		NewAdapter(
			3*time.Hour,
			func(s string) (interface{}, error) { return time.ParseDuration(s) },
			func(v interface{}) (string, error) { return v.(time.Duration).String(), nil },
		),

		NewAdapter(big.NewRat(1, 2), nil, nil),
		NewAdapter(
			int(32),
			func(s string) (interface{}, error) {
				return strconv.Atoi(s)
			},
			func(v interface{}) (string, error) {
				return strconv.Itoa(v.(int)), nil
			},
		),
	}
)
View Source 
var JSONMarshaler = new(jsonMarshaler)

JSONMarshaler encodes and decodes JSON and is the default used in this codec.

Functions

func WithMarshalUnmarshaler 

func WithMarshalUnmarshaler(marshaler MarshalUnmarshaler) func(c *Codec) error

WithMarshalUnmarshaler sets the MarshalUnmarshaler to use. Default is JSONMarshaler.

Example 
package main

import (
	"fmt"
	"log"

	"github.com/bep/tmc"
	yaml "gopkg.in/yaml.v2"
)

func main() {
	m1 := map[string]interface{}{"num": 42}
	c, err := tmc.New(tmc.WithMarshalUnmarshaler(new(yamlMarshaler)))
	if err != nil {
		log.Fatal(err)
	}

	data, err := c.Marshal(m1)
	if err != nil {
		log.Fatal(err)
	}

	m2 := make(map[string]interface{})
	err = c.Unmarshal(data, &m2)
	if err != nil {
		log.Fatal(err)
	}
	num := m2["num"]

	fmt.Printf("%v (%T)", num, num)
}

type yamlMarshaler int

func (yamlMarshaler) Marshal(v interface{}) ([]byte, error) {
	return yaml.Marshal(v)

}

func (yamlMarshaler) Unmarshal(b []byte, v interface{}) error {
	return yaml.Unmarshal(b, v)
}

Output:

42 (int)

func WithTypeAdapters 

func WithTypeAdapters(typeAdapters []Adapter) func(c *Codec) error

WithTypeAdapters sets the type adapters to use. Note that if more than one adapter exists for the same type, the last one will win. This means that if you want to use the default adapters, but override some of them, you can do: 

adapters := append(typedmapcodec.DefaultTypeAdapters, mycustomAdapters ...)
codec := typedmapcodec.New(WithTypeAdapters(adapters))

func WithTypeSep 

func WithTypeSep(sep string) func(c *Codec) error

WithTypeSep sets the separator to use before the type information encoded in the key field. Default is "|".

Types

type Adapter 

type Adapter interface {
	FromString(s string) (interface{}, error)
	MarshalText() (text []byte, err error)
	Type() reflect.Type
	Wrap(v interface{}) Adapter
}

Adapter wraps a type to preserve type information when encoding and decoding a map.

The simples way to create new adapters is via the NewAdapter function.

func NewAdapter 

func NewAdapter(
	target interface{},
	fromString func(s string) (interface{}, error),
	toString func(v interface{}) (string, error)) Adapter

NewAdapter creates a new adapter that wraps the target type.

fromString can be omitted if target implements encoding.TextUnmarshaler. toString can be omitted if target implements encoding.TextMarshaler.

It will panic if it can not be created.

type Codec 

type Codec struct {
	// contains filtered or unexported fields
}

Codec provides methods to marshal and unmarshal a Go map while preserving type information.

func New 

func New(opts ...Option) (*Codec, error)

New creates a new Coded with some optional options.

func (*Codec) Marshal 

func (c *Codec) Marshal(v interface{}) ([]byte, error)

Marshal accepts a Go map and marshals it to the configured marshaler anntated with type information.

func (*Codec) Unmarshal 

func (c *Codec) Unmarshal(data []byte, v interface{}) error

Unmarshal unmarshals the given data to the given Go map, using any annotated type information found to preserve the type information stored in Marshal.

type MarshalUnmarshaler 

type MarshalUnmarshaler interface {
	Marshal(v interface{}) ([]byte, error)
	Unmarshal(b []byte, v interface{}) error
}

MarshalUnmarshaler is the interface that must be implemented if you want to add support for more than JSON to this codec.

type Option 

type Option func(c *Codec) error

Option configures the Codec.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.