gson

package module
v0.0.0-...-2e73bae Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 9, 2021 License: MIT Imports: 17 Imported by: 8

README

Object formats and notations

Build Status Coverage Status GoDoc Go Report Card GitPitch

  • High performance algorithms for data transformation, serialization and manipulation.
  • Based on well established standards.
  • ZERO allocation when transforming from one format to another, except for APIs creating golang values from encoded data.
  • JSON for web.
  • CBOR for machine.
  • Binary-Collation for crazy fast comparison/sorting.

This package is under continuous development, but the APIs are fairly stable.

What is what

JSON

  • Java Script Object Notation, also called JSON, RFC-7159
  • Fast becoming the internet standard for data exchange.
  • Human readable format, not so friendly for machine representation.

Value (aka gson)

  • Golang object parsed from JSON, CBOR or collate representation.
  • JSON arrays are represented in golang as []interface{}.
  • JSON objects, aka properties, are presented in golang as map[string]interface{}.
  • Following golang-types can be transformed to JSON, CBOR, or, Binary-collation - nil, bool, byte, int8, int16, uint16, int32, uint32, int, uint, int64, uint64, float32, float64, string, []interface{}, map[string]interface{}, [][2]interface{}.
  • For type [][2]interface{}, first item is treated as key (string) and second item is treated as value, hence equivalent to map[string]interface{}.
  • Gson objects support operations like, Get(), Set(), and Delete() on individual fields located by the json-pointer.

CBOR

  • Concise Binary Object Representation, also called CBOR, RFC-7049link.
  • Machine friendly, designed for IoT, inter-networking of light weight devices, and easy to implement in many languages.
  • Can be used for more than data exchange, left to user imagination :) ...

Binary-Collation

  • A custom encoding based on a paper and improvised to handle JSON specification.
  • Binary representation preserving the sort order.
  • Transform back to original JSON from binary representation.
  • Numbers can be treated as floating-point, for better performance or either as floating-point or integer, for flexibility.
  • More details can be found here

JSON-Pointer

  • URL like field locator within a JSON object, RFC-6901.
  • For navigating through JSON arrays and objects, but to any level of nesting.
  • JSON-pointers shall be unquoted before they are used as path into JSON document.
  • Documents encoded in CBOR format using LengthPrefix are not supported by lookup APIs.

Performance and memory pressure

Following Benchmark is made on a map data which has a shape similar to:

{"key1": nil, "key2": true, "key3": false,
"key4": "hello world", "key5": 10.23122312}

or,

{"a":null,"b":true,"c":false,"d\"":10,"e":"tru\"e", "f":[1,2]}

BenchmarkVal2JsonMap5-8  3000000   461 ns/op    0 B/op  0 allocs/op
BenchmarkVal2CborMap5-8  5000000   262 ns/op    0 B/op  0 allocs/op
BenchmarkVal2CollMap-8   1000000  1321 ns/op  128 B/op  2 allocs/op
BenchmarkJson2CborMap-8  2000000   838 ns/op    0 B/op  0 allocs/op
BenchmarkCbor2JsonMap-8  2000000  1010 ns/op    0 B/op  0 allocs/op
BenchmarkJson2CollMap-8  1000000  1825 ns/op  202 B/op  2 allocs/op
BenchmarkColl2JsonMap-8  1000000  2028 ns/op  434 B/op  6 allocs/op
BenchmarkCbor2CollMap-8  1000000  1692 ns/op  131 B/op  2 allocs/op
BenchmarkColl2CborMap-8  1000000  1769 ns/op  440 B/op  6 allocs/op

Though converting to golang value incurs cost.

BenchmarkJson2ValMap5    1000000  1621 ns/op   699 B/op  14 allocs/op
BenchmarkCbor2ValMap5    1000000  1711 ns/op   496 B/op  18 allocs/op
BenchmarkColl2ValMap     1000000  2235 ns/op  1440 B/op  33 allocs/op

Configuration

Configuration APIs are not re-entrant. For concurrent use of Gson, please create a gson.Config{} per routine, or protect them with a mutex.

NumberKind

There are two ways to treat numbers in Gson, as integers (upto 64-bit width) or floating-point (float64).

  • FloatNumber configuration can be used to tell Gson to treat all numbers as floating point. This has the convenience of having precision between discrete values, but suffers round of errors and inability to represent integer values greater than 2^53. DEFAULT choice.

  • SmartNumber will use int64, uint64 for representing integer values and use float64 when decimal precision is required. Choosing this option, Gson might incur a slight performance penalty.

Can be configured per configuration instance via SetNumberKind().

MaxKeys

Maximum number of keys allowed in a property object. This can be configured globally via gson.MaxKeys or per configuration via SetMaxkeys().

Memory-pools

Gson uses memory pools:

  • Pool of Byte blocks for listing json-pointers.
  • Pool of Byte blocks for for encoding / decoding string types, and property keys.
  • Pool of set of strings for sorting keys within property.

Memory foot print of gson depends on the pools size, maximum length of input string, number keys in input property-map.

Mempools can be configured globally via gson.MaxStringLen, gson.MaxKeys, gson.MaxCollateLen, gson.MaxJsonpointerLen package variables or per configuration instance via ResetPools().

CBOR ContainerEncoding

In CBOR both map and array (called container types) can be encoded as length followed by items, or items followed by end-marker.

  • LengthPrefix to encode length of container type first, followed by each item in the container.
  • Stream to encode each item in the container as it appears in the input stream, finally ending it with a End-Stream-Marker. DEFAULT choice.

Can be configured per configuration instance via SetContainerEncoding

JSON Strict

  • If configured as true, encode / decode JSON strings operations will use Golang's encoding / JSON package.

Can be configured per configuration instance via SetStrict.

JSON SpaceKind

How to interpret space characters ? There are two options:

  • AnsiSpace will be faster but does not support unicode.
  • UnicodeSpace will be slower but supports unicode. DEFAULT choice.

Can be configured per configuration instance via SetSpaceKind.

Collate ArrayLenPrefix

While sorting array, which is a container type, should collation algorithm consider the arity of the array ? If ArrayLenPrefix prefix is configured as true, arrays with more number of items will sort after arrays with lesser number of items.

Can be configured per configuration instance via SortbyArrayLen.

Collate PropertyLenPrefix

While sorting property-map, which is a container type, should collation algorithm consider number of entries in the map ? If PropertyLenPrefix is configured as true, maps with more number of items will sort after maps with lesser number of items.

Can be configured per configuration instance via SortbyPropertyLen.

JSON-Pointer JsonpointerLength

Maximum length a JSON-pointer string can take. Can be configured globally via MaxJsonpointerLen or per configuration instance via SetJptrlen.

NOTE: JSON pointers are composed of path segments, there is an upper limit to the number of path-segments a JSON pointer can have. If your configuration exceeds that limit, try increasing the JsonpointerLength.

Transforms

transforms

Value to CBOR

  • Golang types nil, true, false are encodable into CBOR format.
  • All Golang number types, including signed, unsigned, and floating-point variants, are encodable into CBOR format.
  • Type []byte is encoded as CBOR byte-string.
  • Type string is encoded as CBOR text.
  • Generic array is interpreted as Golang []interface{} and encoded as CBOR array.
    • With LengthPrefix option for ContainerEncoding, arrays and maps are encoded with its length.
    • With Stream option, arrays and maps are encoded using Indefinite and Breakstop encoding.
  • Generic property is interpreted as golang [][2]interface{} and encoded as CBOR array of 2-element array, where the first item is key represented as string and second item is any valid JSON value.
  • Before encoding map[string]interface{} type, use GolangMap2cborMap() function to transform them to [][2]interface{}.
  • Following golang data types are encoded using CBOR-tags,
    • Type time.Time encoded with tag-0.
    • Type Epoch type supplied by CBOR package, encoded with tag-1.
    • Type EpochMicro type supplied by CBOR package, encoded with tag-1.
    • Type math/big.Int positive numbers are encoded with tag-2, and negative numbers are encoded with tag-3.
    • Type DecimalFraction type supplied by CBOR package, encoded with tag-4.
    • Type BigFloat type supplied by CBOR package, encoded with tag-5.
    • Type CborTagBytes type supplied by CBOR package, encoded with tag-24.
    • Type regexp.Regexp encoded with tag-35.
    • Type CborTagPrefix type supplied by CBOR package, encoded with tag-55799.
  • All other types shall cause a panic.

Value to collate

  • Types nil, true, false, float64, int64, int, string, []byte, []interface{}, map[string]interface{} are supported for collation.
  • All JSON numbers are collated as arbitrary sized floating point numbers.
  • Array-length (if configured) and property-length (if configured) are collated as integer.

JSON to Value

  • Gson uses custom parser that must be faster than encoding/JSON.
  • Numbers can be interpreted as integer, or float64,
    • FloatNumber to interpret JSON number as 64-bit floating point.
    • SmartNumber to interpret JSON number as int64, or uint64, or float64.
  • Whitespace can be interpreted, based on configuration type SpaceKind. SpaceKind can be one of the following AnsiSpace or UnicodeSpace.
    • AnsiSpace that should be faster
    • UnicodeSpace supports unicode white-spaces as well.

JSON to collate

  • All number are collated as float.
  • If config.nk is FloatNumber, all numbers are interpreted as float64 and collated as float64.
  • If config.nk is SmartNumber, all JSON numbers are collated as arbitrary sized floating point numbers.
  • Array-length (if configured) and property-length (if configured) are collated as integer.

JSON to CBOR

  • JSON Types null, true, false are encodable into CBOR format.
  • Types number are encoded based on configuration type NumberKind, which can be one of the following.
    • If config.nk is FloatNumber, all numbers are encoded as CBOR-float64.
    • If config.nk is SmartNumber, all JSON float64 numbers are encoded as CBOR-float64, and, all JSON positive integers are encoded as CBOR-uint64, and, all JSON negative integers are encoded as CBOR-int64.
  • Type string will be parsed and translated into UTF-8, and subsequently encoded as CBOR-text.
  • Type arrays can be encoded in Stream mode, using CBOR's indefinite-length scheme, or in LengthPrefix mode.
  • Type properties can be encoded either using CBOR's indefinite-length scheme (Stream), or using CBOR's LengthPrefix.
  • Property-keys are always interpreted as string and encoded as UTF-8 CBOR-text.

CBOR to Value

  • Reverse of all value to CBOR encoding, described above, are supported.
  • Cannot decode float16 type and int64 > 9223372036854775807.
  • Indefinite byte-string chunks, text chunks shall be decoded outside this package using IsIndefinite*() and IsBreakstop() APIs.

CBOR to JSON

  • CBOR types nil, true, false are transformed back to equivalent JSON types.
  • Types float32 and float64 are transformed back to 32 bit JSON-float and 64 bit JSON-float respectively, in non-exponent format.
  • Type integer is transformed back to JSON-integer representation, and integers exceeding 9223372036854775807 are not supported.
  • Type array either with length prefix or with indefinite encoding are converted back to JSON array.
  • Type map either with length prefix or with indefinite encoding are converted back to JSON property.
  • Type bytes-strings are not supported or transformed to JSON.
  • Type CBOR-text with indefinite encoding are not supported.
  • Type Simple type float16 are not supported.

For transforming to and from binary-collation refer here

CBOR to Collate

  • CBOR Types null, true, false, float32, float64, integer, string, []byte (aka binary), array, object can be collated.
  • All number are collated as float.
  • If config.nk is FloatNumber, all numbers are interpreted as float64 and collated as float64.
  • If config.nk is SmartNumber, all JSON numbers are collated as arbitrary sized floating point numbers.
  • Array-length (if configured) and property-length (if configured) are collated as integer.
  • Indefinite-length encoding for text and binary are not supported.
  • LengthPrefix and Stream encoding for array and maps are supported.

Collate to CBOR

  • Missing, null, true, false, floating-point, small-decimal, integer, string, []byte (aka binary), array, object types from its collated from can be converted back to CBOR.
  • Since all numbers are collated as float, it is converted back to text representation of float, in format: [+-]x.e[+-].
  • If config.nk is FloatNumber, all number are encoded as CBOR-float64.
  • If config.nk is SmartNumber, all numbers whose exponent is >= 15 is encoded as uint64 (if number is positive), or int64 (if number is negative). Others are encoded as CBOR-float64.

Collate to JSON

  • Since all numbers are collated as float, it is converted back to text representation of float, in format: [+-]x.e[+-].
  • If config.nk is FloatNumber, all number are encoded as JSON-float64.
  • If config.nk is SmartNumber, all numers whose exponent is >= 15 is encoded as uint64 (if number is positive), or int64 (if number is negative). Others are encoded as JSON-float64.

Collate to Value

  • Since all numbers are collated as float, it is converted back to text representation of float, in format: [+-]x.e[+-].
  • If config.nk is FloatNumber, all number are encoded as JSON-float64.
  • If config.nk is SmartNumber, all numers whose exponent is >= 15 is encoded as uint64 (if number is positive), or int64 (if number is negative). Others are treated as float64.

Articles

How to contribute

Issue Stats Issue Stats

  • Pick an issue, or create an new issue. Provide adequate documentation for the issue.
  • Assign the issue or get it assigned.
  • Work on the code, once finished, raise a pull request.
  • Gson is written in golang, hence expected to follow the global guidelines for writing go programs.
  • If the changeset is more than few lines, please generate a report card.
  • As of now, branch master is the development branch.

Task list

  • Binary collation: transparently handle int64, uint64 and float64.
  • Support for json.Number.
  • UTF-8 collation of strings.
  • JSON-pointer.
    • JSON pointer for looking up within CBOR map.
    • JSON pointer for looking up within value-map.

Notes

  • Don't change the tag number.
  • All supplied APIs will panic in case of error, applications can recover from panic, dump a stack trace along with input passed on to the API, and subsequently handle all such panics as a single valued error.
  • For now, maximum integer range shall be within int64.
  • Config instances, and its APIs, are neither re-entrant nor thread safe.

list of changes from github.com/prataprc/collatejson

  • Codec type is renamed to Config.
  • Caller should make sure that the o/p buffer passed to encoding and decoding APIs are adequately sized.
  • Name and signature of NewCodec() (now, NewDefaultConfig) has changed.
  • Configuration APIs,SortbyArrayLen, SortbyPropertyLen, UseMissing, NumberType all now return the config object back the caller - helps in call-chaining.
  • All APIs panic instead of returning an error.
  • Output buffer should have its len() == cap(), so that encoder and decoder can avoid append and instead use buffer index.

Documentation

Overview

Package gson provide a toolkit for JSON representation, collation and transformation.

Package provides APIs to convert data representation from one format to another. Supported formats are:

  • JSON
  • Golang value
  • CBOR - Concise Binary Object Representation
  • Binary-collation

CBOR:

Concise Binary Object Representation, CBOR, is based on RFC-7049 specification to encode golang data into machine friendly format. Following golang native types are supported:

  • nil, true, false.
  • native integer types, and its alias, of all width.
  • float32, float64.
  • slice of bytes.
  • native string.
  • slice of interface - []interface{}.
  • map of string to interface{} - map[string]interface{}.

Types from golang's standard library and custom types provided by this package that can be encoded using CBOR:

  • CborTagBytes: a cbor encoded []bytes treated as value.
  • CborUndefined: encode a data-item as undefined.
  • CborIndefinite: encode bytes, string, array and map of unspecified length.
  • CborBreakStop: to encode end of CborIndefinite length item.
  • CborTagEpoch: in seconds since epoch.
  • CborTagEpochMicro: in micro-seconds epoch.
  • CborTagFraction: m*(10**e)
  • CborTagFloat: m*(2**e)
  • CborTagPrefix: to self identify a binary blog as CBOR.

Package also provides an implementation for encoding JSON to CBOR and vice-versa:

  • Number can be encoded as integer or float.
  • Arrays and maps are encoded using indefinite encoding.
  • Byte-string encoding is not used.

Json-Pointer:

Package also provides a RFC-6901 (JSON-pointers) implementation.

NOTE: Buffer supplied to APIs NewJson(), NewCbor(), NewCollate() should atleast be 128 bytes in size.

Index

Examples

Constants

View Source 
const CborMaxSmallInt = 23

CborMaxSmallInt maximum integer value that can be stored as associative value for cborType0 or cborType1.

View Source 
const MissingLiteral = Missing("~[]{}falsenilNA~")

MissingLiteral is undocumented, for now.

Variables

View Source 
var (
	Terminator  byte = 0
	TypeMissing byte = 49
	TypeNull    byte = 50
	TypeFalse   byte = 60
	TypeTrue    byte = 70
	TypeNumber  byte = 80
	TypeString  byte = 90
	TypeLength  byte = 100
	TypeArray   byte = 110
	TypeObj     byte = 120
	TypeBinary  byte = 130
)

Collation order for supported types. Applications desiring different ordering between types can initialize these byte values before instantiating a config object.

View Source 
var MaxCollateLen = 1024 * 1024

MaxCollateLen maximum length of collated value. Affects memory pool. Changing this value will affect all new configuration instances.

View Source 
var MaxJsonpointerLen = 2048

MaxJsonpointerLen size of json-pointer path. Affects memory pool. Changing this value will affect all new configuration instances.

View Source 
var MaxKeys = 1024

MaxKeys maximum number of keys allowed in a property object. Affects memory pool. Changing this value will affect all new configuration instances.

View Source 
var MaxStringLen = 1024 * 1024

MaxStringLen maximum length of string value inside json document. Affects memory pool. Changing this value will affect all new configuration instances.

Functions

func CborMap2golangMap 

func CborMap2golangMap(value interface{}) interface{}

CborMap2golangMap used by validation tools. Transforms [][2]interface{} to map[string]interface{} that is required for converting golang to cbor and vice-versa.

func Fixtojson 

func Fixtojson(config *Config, val interface{}) interface{}

Fixtojson used by validation tools.

func GolangMap2cborMap 

func GolangMap2cborMap(value interface{}) interface{}

GolangMap2cborMap used by validation tools. Transforms map[string]interface{} to [][2]interface{} that is required for converting golang to cbor and vice-versa.

Types

type Cbor 

type Cbor struct {
	// contains filtered or unexported fields
}

Cbor encapsulates configuration and a cbor buffer. Use config object's NewCbor() method to Create new instance of Cbor. Map element in cbor encoding should have its keys sorted.

Example 
config := NewDefaultConfig()
cbr := config.NewCbor(make([]byte, 0, 128))

val1 := [][]byte{[]byte("hello"), []byte("world")}
cbr.EncodeBytechunks(val1)
fmt.Printf("value: %v cbor: %q\n", val1, cbr.Bytes())
cbr.Reset(nil)

val2 := [][2]interface{}{{"first", true}}
cbr.EncodeMapslice(val2)
fmt.Printf("value: %v cbor: %q\n", val2, cbr.Bytes())
cbr.Reset(nil)

val3, val4 := byte(128), int8(-10)
cbr.EncodeSimpletype(val3).EncodeSmallint(val4)
fmt.Printf("value: {%v,%v} cbor: %q\n", val3, val4, cbr.Bytes())
cbr.Reset(nil)

val5 := []string{"sound", "ok", "horn"}
cbr.EncodeTextchunks(val5)
fmt.Printf("value: %v cbor: %q\n", val5, cbr.Bytes())
cbr.Reset(nil)

config = NewDefaultConfig()
cbr = config.NewCbor(make([]byte, 0, 1024))
jsn := config.NewJson(make([]byte, 0, 1024))
clt := config.NewCollate(make([]byte, 0, 1024))

config.NewValue([]interface{}{10, 20, 100}).Tocbor(cbr)
fmt.Printf("to json   : %q\n", cbr.Tojson(jsn).Bytes())
fmt.Printf("to collate: %q\n", cbr.Tocollate(clt).Bytes())
fmt.Printf("to value  : %v\n", cbr.Tovalue())

Output:

value: [[104 101 108 108 111] [119 111 114 108 100]] cbor: "_EhelloEworld\xff"
value: [[first true]] cbor: "efirst\xf5"
value: {128,-10} cbor: "\xf8\x80)"
value: [sound ok horn] cbor: "\u007fesoundbokdhorn\xff"
to json   : "[10,20,100]"
to collate: "nP>>21-\x00P>>22-\x00P>>31-\x00\x00"
to value  : [10 20 100]

func (*Cbor) Append 

func (cbr *Cbor) Append(jptr *Jsonpointer, item, newdoc *Cbor) *Cbor

Append item at the end of an array field specified by json-pointer.

Example 
config := NewDefaultConfig()
cbr1 := config.NewCbor(make([]byte, 0, 1024))
cbr2 := config.NewCbor(make([]byte, 0, 1024))
cbritem := config.NewCbor(make([]byte, 0, 1024))
olditem := config.NewCbor(make([]byte, 0, 1024))

config.NewJson([]byte(`[]`)).Tocbor(cbr1)
fmt.Println("start with  `[]`")

ptr := config.NewJsonpointer("")
config.NewValue(10.0).Tocbor(cbritem.Reset(nil))
cbr1.Append(ptr, cbritem, cbr2)
fmt.Printf("after appending 10 %v\n", cbr2.Tovalue())
cbr1.Reset(nil)

ptr = config.NewJsonpointer("")
config.NewValue(20.0).Tocbor(cbritem.Reset(nil))
cbr2.Prepend(ptr, cbritem, cbr1)
fmt.Printf("after prepending 20 %v\n", cbr1.Tovalue())
cbr2.Reset(nil)

ptr = config.NewJsonpointer("/1")
config.NewValue(30.0).Tocbor(cbritem.Reset(nil))
cbr1.Set(ptr, cbritem, cbr2, olditem)
fmsg := "after setting 30 to second item %v old value %v\n"
fmt.Printf(fmsg, cbr2.Tovalue(), olditem.Tovalue())
cbr1.Reset(nil)

ptr = config.NewJsonpointer("/1")
cbr2.Get(ptr, cbritem.Reset(nil))
fmt.Printf("get second item %v\n", cbritem.Tovalue())

ptr = config.NewJsonpointer("/0")
cbr2.Delete(ptr, cbr1, cbritem.Reset(nil))
fmsg = "after deleting first item %v, deleted value %v\n"
fmt.Printf(fmsg, cbr1.Tovalue(), cbritem.Tovalue())
cbr2.Reset(nil)

Output:

start with  `[]`
after appending 10 [10]
after prepending 20 [20 10]
after setting 30 to second item [20 30] old value 10
get second item 30
after deleting first item [30], deleted value 20

func (*Cbor) Bytes 

func (cbr *Cbor) Bytes() []byte

Bytes return the byte-slice holding the CBOR data.

func (*Cbor) Delete 

func (cbr *Cbor) Delete(jptr *Jsonpointer, newdoc, deleted *Cbor) *Cbor

Delete field or nested field specified by json-pointer.

func (*Cbor) EncodeBytechunks 

func (cbr *Cbor) EncodeBytechunks(chunks [][]byte) *Cbor

EncodeBytechunks to encode several chunks of bytes as an indefinite-sequence of byte-blocks.

func (*Cbor) EncodeMapslice 

func (cbr *Cbor) EncodeMapslice(items [][2]interface{}) *Cbor

EncodeMapslice to encode key,value pairs into cbor buffer. Whether to encode them as indefinite-sequence of pairs, or as length prefixed pairs is decided by config.ContainerEncoding.

func (*Cbor) EncodeSimpletype 

func (cbr *Cbor) EncodeSimpletype(typcode byte) *Cbor

EncodeSimpletype to encode simple type into cbor buffer. Code points 0..19 and 32..255 are un-assigned.

func (*Cbor) EncodeSmallint 

func (cbr *Cbor) EncodeSmallint(item int8) *Cbor

EncodeSmallint to encode tiny integers between -23..+23 into cbor buffer.

func (*Cbor) EncodeTextchunks 

func (cbr *Cbor) EncodeTextchunks(chunks []string) *Cbor

EncodeTextchunks to encode several chunks of text as an indefinite-sequence of byte-blocks.

func (*Cbor) Get 

func (cbr *Cbor) Get(jptr *Jsonpointer, item *Cbor) *Cbor

Get field or nested field specified by json-pointer.

func (*Cbor) Prepend 

func (cbr *Cbor) Prepend(jptr *Jsonpointer, item, newdoc *Cbor) *Cbor

Prepend item to the beginning of an array field specified by json-pointer.

func (*Cbor) Reset 

func (cbr *Cbor) Reset(data []byte) *Cbor

Reset overwrite buffer with data, or if data is nil, reset buffer to zero-length.

func (*Cbor) Set 

func (cbr *Cbor) Set(jptr *Jsonpointer, item, newdoc, old *Cbor) *Cbor

Set field or nested field specified by json-pointer.

func (*Cbor) Tocollate 

func (cbr *Cbor) Tocollate(clt *Collate) *Collate

Tocollate convert to binary-collation.

func (*Cbor) Tojson 

func (cbr *Cbor) Tojson(jsn *Json) *Json

Tojson convert to json encoded value.

func (*Cbor) Tovalue 

func (cbr *Cbor) Tovalue() interface{}

Tovalue convert to golang native value.

type CborBreakStop 

type CborBreakStop byte

CborBreakStop code, cborType7/cborItemBreak

type CborIndefinite 

type CborIndefinite byte

CborIndefinite code, {cborType2,Type3,Type4,Type5}/cborIndefiniteLength

type CborTagBytes 

type CborTagBytes []byte

CborTagBytes codepoint-24, bytes in cbor format.

type CborTagEpoch 

type CborTagEpoch int64

CborTagEpoch codepoint-1, followed by int64 of seconds since 1970-01-01T00:00Z in UTC time.

type CborTagEpochMicro 

type CborTagEpochMicro float64

CborTagEpochMicro codepoint-1, followed by float64 of seconds/us since 1970-01-01T00:00Z in UTC time.

type CborTagFloat 

type CborTagFloat [2]int64

CborTagFloat codepoint-5, followed by [2]int64{e,m} => m*(2**e).

type CborTagFraction 

type CborTagFraction [2]int64

CborTagFraction codepoint-4, followed by [2]int64{e,m} => m*(10**e).

type CborTagPrefix 

type CborTagPrefix []byte

CborTagPrefix codepoint-5579, followed by byte-string.

type CborUndefined 

type CborUndefined byte

CborUndefined simple type, cborType7/cborSimpleUndefined

type Collate 

type Collate struct {
	// contains filtered or unexported fields
}

Collate abstraction for value encoded into binary-collation.

Example 
config := NewDefaultConfig()
cbr := config.NewCbor(make([]byte, 0, 1024))
jsn := config.NewJson(make([]byte, 0, 1024))
clt := config.NewCollate(make([]byte, 0, 1024))

config.NewValue([]interface{}{10, 20, 100}).Tocollate(clt)
fmt.Printf("json : %q\n", clt.Tojson(jsn).Bytes())
fmt.Printf("cbor : %q\n", clt.Tocbor(cbr).Bytes())
fmt.Printf("value: %v\n", clt.Tovalue())

Output:

json : "[1e+01,2e+01,1e+02]"
cbor : "\x9f\xfb@$\x00\x00\x00\x00\x00\x00\xfb@4\x00\x00\x00\x00\x00\x00\xfb@Y\x00\x00\x00\x00\x00\x00\xff"
value: [10 20 100]

func (*Collate) Bytes 

func (clt *Collate) Bytes() []byte

Bytes return the byte-slice holding the collated data.

func (*Collate) Reset 

func (clt *Collate) Reset(data []byte) *Collate

Reset overwrite buffer with data, or if data is nil, reset buffer to zero-length.

func (*Collate) Tocbor 

func (clt *Collate) Tocbor(cbr *Cbor) *Cbor

Tocbor convert to cbor encoded value.

func (*Collate) Tojson 

func (clt *Collate) Tojson(jsn *Json) *Json

Tojson convert to json encoded text.

func (*Collate) Tovalue 

func (clt *Collate) Tovalue() interface{}

Tovalue convert to golang native value.

type Config 

type Config struct {
	// contains filtered or unexported fields
}

Config is the root object to access all transformations and APIs exported by this package. Before calling any of the config-methods, make sure to initialize them with desired settings.

NOTE: Config objects are immutable.

Example 
config := NewDefaultConfig()
// override default configuration options.
// IMPORTANT: config objects are immutable, so assign back
// the new object returned by each of the settings method.
config = config.SetContainerEncoding(LengthPrefix)
config = config.SetJptrlen(1024).SetMaxkeys(10000)
config = config.SetNumberKind(FloatNumber).SetSpaceKind(AnsiSpace)
config = config.SortbyArrayLen(true).SortbyPropertyLen(true)
config = config.UseMissing(false)
fmt.Println(config)

Output:

nk:FloatNumber, ws:AnsiSpace, ct:LengthPrefix, arrayLenPrefix:true, propertyLenPrefix:true, doMissing:false

func NewDefaultConfig 

func NewDefaultConfig() *Config

NewDefaultConfig return a new configuration with default settings: 

+FloatNumber        +Stream
+UnicodeSpace       -strict
+doMissing          -arrayLenPrefix     +propertyLenPrefix
MaxJsonpointerLen   MaxKeys
MaxStringLen
MaxCollateLen

Several methods are available to change configuration parameters.

func (*Config) NewCbor 

func (config *Config) NewCbor(buffer []byte) *Cbor

NewCbor factory to create a new Cbor instance. If buffer is nil, a new buffer of 128 byte capacity will be allocated. Cbor object can be re-used after a Reset() call.

func (*Config) NewCollate 

func (config *Config) NewCollate(buffer []byte) *Collate

NewCollate factor to create a new Collate instance. If buffer is nil, a new buffer of 128 byte capacity will be allocated. Collate object can be re-used after a Reset() call.

func (*Config) NewJson 

func (config *Config) NewJson(buffer []byte) *Json

NewJson factory to create a new Json instance. If buffer is nil, a new buffer of 128 byte capacity will be allocated. Json object can be re-used after a Reset() call.

func (*Config) NewJsonpointer 

func (config *Config) NewJsonpointer(path string) *Jsonpointer

NewJsonpointer create a instance of Jsonpointer.

func (*Config) NewValue 

func (config *Config) NewValue(value interface{}) *Value

NewValue factory to create a new Value instance. Value instances are immutable, and can be used and re-used any number of times.

func (Config) ResetPools 

func (config Config) ResetPools(strlen, numkeys, itemlen, ptrlen int) *Config

ResetPools configure a new set of pools with specified size, instead of using the default size: MaxStringLen, MaxKeys, MaxCollateLen, and, MaxJsonpointerLen. 

strlen  - maximum length of string value inside JSON document
numkeys - maximum number of keys that a property object can have
itemlen - maximum length of collated value.
ptrlen  - maximum possible length of json-pointer.

func (Config) SetContainerEncoding 

func (config Config) SetContainerEncoding(ct ContainerEncoding) *Config

SetContainerEncoding configure to encode / decode cbor arrays and maps.

func (Config) SetJptrlen 

func (config Config) SetJptrlen(n int) *Config

SetJptrlen will set the maximum size for jsonpointer path.

func (Config) SetMaxkeys 

func (config Config) SetMaxkeys(n int) *Config

SetMaxkeys configure to set the maximum number of keys allowed in property item.

func (Config) SetNumberKind 

func (config Config) SetNumberKind(nk NumberKind) *Config

SetNumberKind configure to interpret number values.

func (Config) SetSpaceKind 

func (config Config) SetSpaceKind(ws SpaceKind) *Config

SetSpaceKind setting to interpret whitespaces in json text.

func (Config) SetStrict 

func (config Config) SetStrict(what bool) *Config

SetStrict setting to enforce strict transforms to and from JSON. If set to true, 

a. IntNumber configuration float numbers in JSON text still are parsed.
b. Use golang stdlib encoding/json for transforming strings to JSON.

func (Config) SetTextCollator 

func (config Config) SetTextCollator(collator *collate.Collator) *Config

SetTextCollator for string type. If collator is not set, strings will be treated as byte-array and compared as such.

func (Config) SortbyArrayLen 

func (config Config) SortbyArrayLen(what bool) *Config

SortbyArrayLen setting to sort array of smaller-size before larger ones.

func (Config) SortbyPropertyLen 

func (config Config) SortbyPropertyLen(what bool) *Config

SortbyPropertyLen setting to sort properties of smaller size before larger ones.

func (*Config) String 

func (config *Config) String() string

func (Config) UseMissing 

func (config Config) UseMissing(what bool) *Config

UseMissing setting to use TypeMissing collation.

type ContainerEncoding 

type ContainerEncoding byte

ContainerEncoding method to encode arrays and maps into cbor. 

const (
	// LengthPrefix to encode number of items in the collection type.
	LengthPrefix ContainerEncoding = iota + 1

	// Stream to encode collection types as indefinite sequence of items.
	Stream
)

func (ContainerEncoding) String 

func (ct ContainerEncoding) String() string

type Json 

type Json struct {
	// contains filtered or unexported fields
}

Json abstraction for value encoded as json text.

Example 
config := NewDefaultConfig()
cbr := config.NewCbor(make([]byte, 0, 1024))
jsn := config.NewJson(make([]byte, 0, 1024))
clt := config.NewCollate(make([]byte, 0, 1024))

config.NewValue([]interface{}{10, 20, 100}).Tojson(jsn)
fmt.Printf("collate : %q\n", jsn.Tocollate(clt).Bytes())
fmt.Printf("cbor : %q\n", jsn.Tocbor(cbr).Bytes())
_, value := jsn.Tovalue()
fmt.Printf("value: %v\n", value)

Output:

collate : "nP>>21-\x00P>>22-\x00P>>31-\x00\x00"
cbor : "\x9f\xfb@$\x00\x00\x00\x00\x00\x00\xfb@4\x00\x00\x00\x00\x00\x00\xfb@Y\x00\x00\x00\x00\x00\x00\xff"
value: [10 20 100]

func (*Json) Bytes 

func (jsn *Json) Bytes() []byte

Bytes return reference to byte-slice of valid json-buffer.

func (*Json) Reset 

func (jsn *Json) Reset(data []byte) *Json

Reset overwrite buffer with data, or if data is nil, reset buffer to zero-length.

func (*Json) Tocbor 

func (jsn *Json) Tocbor(cbr *Cbor) *Cbor

Tocbor convert json encoded value into cbor encoded binary string.

func (*Json) Tocollate 

func (jsn *Json) Tocollate(clt *Collate) *Collate

Tocollate convert json encoded value into binary-collation.

func (*Json) Tovalue 

func (jsn *Json) Tovalue() (*Json, interface{})

Tovalue parse json text to golang native value. Return remaining text.

func (*Json) Tovalues 

func (jsn *Json) Tovalues() []interface{}

Tovalues parse json text to one or more go native values.

type Jsonpointer 

type Jsonpointer struct {
	// contains filtered or unexported fields
}

Jsonpointer abstracts rfc-6901 into a type. allows ~0 and ~1 escapes, property lookup by specifying the key, and array lookup by specifying the index. Also allows empty "" pointer and empty key "/".

func (*Jsonpointer) Path 

func (jptr *Jsonpointer) Path() []byte

Path return the path value.

func (*Jsonpointer) ResetPath 

func (jptr *Jsonpointer) ResetPath(path string) *Jsonpointer

ResetPath to reuse the Jsonpointer object for a new path.

func (*Jsonpointer) ResetSegments 

func (jptr *Jsonpointer) ResetSegments(segments []string) *Jsonpointer

ResetSegments variant of ResetPath to reconstruct the path from segments.

func (*Jsonpointer) Segments 

func (jptr *Jsonpointer) Segments() [][]byte

Segments return path segments, segments in a path are separated by "/"

type Missing 

type Missing string

Missing denotes a special type for an item that evaluates to _nothing_.

func (Missing) Equal 

func (m Missing) Equal(n string) bool

Equal checks wether n is MissingLiteral

type NumberKind 

type NumberKind byte

NumberKind how to treat numbers. 

const (
	// FloatNumber to treat number as float64.
	FloatNumber NumberKind = iota + 1

	// SmartNumber to treat number as either integer or fall back to float64.
	SmartNumber
)

func (NumberKind) String 

func (nk NumberKind) String() string

type SpaceKind 

type SpaceKind byte

SpaceKind to skip white-spaces in JSON text. 

const (
	// AnsiSpace will skip white space characters defined by ANSI spec.
	AnsiSpace SpaceKind = iota + 1

	// UnicodeSpace will skip white space characters defined by Unicode spec.
	UnicodeSpace
)

func (SpaceKind) String 

func (ws SpaceKind) String() string

type Value 

type Value struct {
	// contains filtered or unexported fields
}

Value abstractions for golang-native value.

func (*Value) Append 

func (val *Value) Append(jptr *Jsonpointer, item interface{}) interface{}

Append item to end of an array pointed by json-pointer. returns `newval`, is guaranteed to be updated, 

val := NewValue([]interface{}{"hello", "world"})
newval, _ = val.Append("", "welcome")

func (*Value) Compare 

func (val *Value) Compare(other *Value) int

Compare to value object.

func (*Value) Data 

func (val *Value) Data() interface{}

Data returns the golang data.

func (*Value) Delete 

func (val *Value) Delete(jptr *Jsonpointer) (newval, deleted interface{})

Delete field or nested field specified by json pointer. While `newval` is guaranteed to be updated, `val` _may_ not be. Suggested usage, 

val := NewValue([]interface{}{"hello", "world"})
newval, _ = val.Delete("/1")

func (*Value) Get 

func (val *Value) Get(jptr *Jsonpointer) (item interface{})

Get field or nested field specified by json pointer.

func (*Value) ListPointers 

func (val *Value) ListPointers(ptrs []string) []string

ListPointers all possible pointers in value.

func (*Value) Prepend 

func (val *Value) Prepend(jptr *Jsonpointer, item interface{}) interface{}

Prepend an item to the beginning of an array. returns `newval`, is guaranteed to be updated, 

val := NewValue([]interface{}{"hello", "world"})
newval, _ = val.Append("", "welcome")

func (*Value) Set 

func (val *Value) Set(jptr *Jsonpointer, item interface{}) (newval, oldval interface{})

Set field or nested field specified by json pointer. While `newval` is guaranteed to contain the `item`, `val` _may_ not be. Suggested usage, 

val := config.NewValue([]interface{}{"hello"})
newval, _ = val.Set("/-", "world")

func (*Value) Tocbor 

func (val *Value) Tocbor(cbr *Cbor) *Cbor

Tocbor encode golang native into cbor binary.

func (*Value) Tocollate 

func (val *Value) Tocollate(clt *Collate) *Collate

Tocollate encode golang native into binary-collation.

func (*Value) Tojson 

func (val *Value) Tojson(jsn *Json) *Json

Tojson encode golang native value to json text.

Source Files

View all Source files

Directories

Path Synopsis
template
src/go

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.