cork

package module
v0.0.0-...-6b3677f Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 16, 2020 License: Apache-2.0 Imports: 12 Imported by: 0

README

cork

An efficient binary serialisation format for Go (Golang).

Features
  • Simple and efficient encoding
  • Based on MsgPack serialization algorithm
  • Stores go type information inside encoded data
  • Faster serialization than gob binary encoding
  • More efficient output data size than gob binary encoding
  • Serializes native go types, and arbritary structs or interfaces
  • Enables predetermined encoding for structs without run-time reflection
  • Allows serialization to and from maps, structs, slices, and nil interfaces
Installation
go get github.com/abcum/cork

Documentation

Overview

CORK is a dynamic and static binary object serialization specification similar to GOB and MessagePack. CORK manages encoding and decoding binary data streams, for use in transmission of, or storage of data.

Basics

A CORK data stream is self-describing. Each data item in the stream is preceded by a description of its type, expressed in terms of a small set of predefined types. Pointers to values are not transmitted, but the contents are flattened and transmitted.

To use cork, create an Encoder and present it with a series of data items as values or addresses that can be dereferenced to values. The Encoder makes sure all type information is sent before it is needed. At the receive side, a Decoder retrieves values from the encoded stream and unpacks them into local variables.

Types

CORK has built in support for the built-in Golang types 

nil
bool
string
[]byte
int
int8
int16
int32
int64
uint
uint8
uint16
uint32
uint64
float32
float64
complex64
complex128
time.Time
interface{}
[]<T>
map[<T>]<T>

Structs

When a struct is encountered whilst encoding (and that struct does not satisfy the Corker interface) then the struct will be encoded into the stream as a map with the keys encoded as strings, and the values as the relevant type. Any struct tags describing how the struct should be encoded will be used.

Corkers

CORK allows applications to define application-specific types to be added to the encoding format. Each extended type must be assigned a unique byte from 0x00 upto 0xFF. Application-specific types (otherwise known as Corkers) are able to encode themselves into a binary data value, and are able to decode themselves from that same binary data value.

To define a custom type, an application must ensure that the type satisfies the Corker interface, and must then register the type using the Register method.

If a custom type is found in the stream when decoding, but no type with the specified unique byte is registered, then the binary data value will be decoded as a raw binary data value.

Types and Values

The source and destination values/types need not correspond exactly. For structs, fields (identified by name) that are in the source but absent from the receiving variable will be ignored. Fields that are in the receiving variable but missing from the transmitted type or value will be ignored in the destination. If a field with the same name is present in both, their types must be compatible. Both the receiver and transmitter will do all necessary indirection and dereferencing to convert between cork encoded data and actual Go values. For instance, a cork type that is schematically, 

struct { A, B int }

can be sent from or received into any of these Go types: 

struct { A, B int }	// the same
*struct { A, B int }	// extra indirection of the struct
struct { *A, **B int }	// extra indirection of the fields
struct { A, B int64 }	// different concrete value type; see below

It may also be received into any of these: 

struct { A, B int }	// the same
struct { B, A int }	// ordering doesn't matter; matching is by name
struct { A, B, C int }	// extra field (C) ignored
struct { B int }	// missing field (A) ignored; data will be dropped
struct { B, C int }	// missing field (A) ignored; extra field (C) ignored.

Attempting to receive into these types will draw a decode error: 

struct { A int; B uint }	// change of signedness for B
struct { A int; B float }	// change of type for B
struct { }			// no field names in common
struct { C, D int }		// no field names in common

Integers can be serialized using two different methods: arbritrary precision or full precision. When using arbritrary precision, integers (int, int8, int16, int32, int64) and unsinged integers (uint, uint8, uint16, uint32, uint64) are encoded with a variable-length encoding using as few bytes as possible, and are decoded into the destination variable, or an integer with the necessary capacity (when decoding into a nil interface). When using full precision, all integers (int8, int16, int32, int64) and unsinged integers (uint8, uint16, uint32, uint64) are encoded with a fixed-length encoding format, and are able to be decoded into the corresponding variable type when decoding into a nil interface.

Signed integers may be received into any signed integer variable: int, int16, etc.; unsigned integers may be received into any unsigned integer variable; and floating point values may be received into any floating point variable. However, the destination variable must be able to represent the value or the decode operation will fail.

Structs, arrays and slices are also supported. Structs encode and decode only exported fields. Struct tags (using the 'cork' descriptor) can specify custom key names to be used when serailizing, and can be omitted entirely, or when empty using the 'omitempty' tag keyword. Strings and arrays of bytes are supported with a special, efficient representation. When a slice is decoded, if the existing slice has capacity the slice will be extended in place; if not, a new array is allocated. Regardless, the length of the resulting slice reports the number of elements decoded.

In general, if allocation is required, the decoder will allocate memory. If not, it will update the destination variables with values read from the stream. It does not initialize them first, so if the destination is a compound value such as a map, struct, or slice, the decoded values will be merged elementwise into the existing variables.

Functions and channels will not be encoded into a CORK. Attempting to encode such a value at the top level will fail. A struct field of chan or func type is treated exactly like an unexported field and is ignored.

Specification

The full specification can be found at http://github.com/abcum/cork/SPEC.md

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Decode 

func Decode(src []byte) (dst interface{})

Decode decodes binary data.

func DecodeInto 

func DecodeInto(src []byte, dst interface{})

DecodeInto decodes a byte slice into a Go object.

func Encode 

func Encode(src interface{}) (dst []byte)

Encode encodes a Go object.

func EncodeInto 

func EncodeInto(src interface{}, dst *[]byte)

EncodeInto encodes a Go object into a byte slice.

func Register 

func Register(value interface{})

Register adds a Corker type to the registry, enabling the object type to be encoded and decoded using the Corker methods.

Types

type Corker 

type Corker interface {
	ExtendCORK() byte
	MarshalCORK() ([]byte, error)
	UnmarshalCORK([]byte) error
}

Corker represents an object which can encode and decode itself.

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder represents a CORK decoder.

func NewDecoder 

func NewDecoder(r io.Reader) *Decoder

NewDecoder returns a Decoder for decoding from an io.Reader.

func NewDecoderBytes 

func NewDecoderBytes(b []byte) *Decoder

NewDecoderBytes returns a Decoder for decoding directly from a byte slice.

func NewDecoderBytesFromPool 

func NewDecoderBytesFromPool(b []byte) *Decoder

NewDecoderBytesFromPool returns a Decoder for decoding directly from a byte slice. The Decoder is taken from a pool of decoders, and must be put back when finished, using d.Reset().

func NewDecoderFromPool 

func NewDecoderFromPool(r io.Reader) *Decoder

NewDecoderFromPool returns a Decoder for decoding into an io.Reader. The Decoder is taken from a pool of decoders, and must be put back when finished, using d.Reset().

func (*Decoder) Decode 

func (d *Decoder) Decode(dst interface{}) (err error)

Decode decodes the stream into the 'dst' object.

The decoder can not decode into a nil pointer, but can decode into a nil interface. If you do not know what type of stream it is, pass in a pointer to a nil interface. We will decode and store a value in that nil interface.

When decoding into a nil interface{}, we will decode into an appropriate value based on the contents of the stream. When decoding into a non-nil interface{} value, the mode of encoding is based on the type of the value.

Example: 

// Decoding into a non-nil typed value
var s string
buf := bytes.NewBuffer(src)
err = cork.NewDecoder(buf).Decode(&s)

// Decoding into nil interface
var v interface{}
buf := bytes.NewBuffer(src)
err := cork.NewDecoder(buf).Decode(&v)

func (*Decoder) Options 

func (d *Decoder) Options(h *Handle) *Decoder

Options sets the configuration options that the Decoder should use.

func (*Decoder) Reset 

func (d *Decoder) Reset()

Reset flushes adds the Decoder back into the sync pool. If the Decoder was not originally from the sync pool, then the Decoder is discarded.

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder represents a CORK encoder.

func NewEncoder 

func NewEncoder(w io.Writer) *Encoder

NewEncoder returns an Encoder for encoding into an io.Writer.

func NewEncoderBytes 

func NewEncoderBytes(b *[]byte) *Encoder

NewEncoderBytes returns an Encoder for encoding directly into a byte slice.

func NewEncoderBytesFromPool 

func NewEncoderBytesFromPool(b *[]byte) *Encoder

NewEncoderBytesFromPool returns an Encoder for encoding directly into a byte slice. The Encoder is taken from a pool of encoders, and must be put back when finished, using e.Reset().

func NewEncoderFromPool 

func NewEncoderFromPool(w io.Writer) *Encoder

NewEncoderFromPool returns an Encoder for encoding into an io.Writer. The Encoder is taken from a pool of encoders, and must be put back when finished, using e.Reset().

func (*Encoder) Encode 

func (e *Encoder) Encode(src interface{}) (err error)

Encode encodes the 'src' object into the stream.

The decoder can be configured using struct tags. The 'cork' key if found will be analysed for any configuration options when encoding.

Each exported struct field is encoded unless:

  • the field's tag is "-"
  • the field is empty and its tag specifies the "omitempty" option.

When encoding a struct as a map, the first string in the tag (before the comma) will be used for the map key, and if not specified will default to the struct key name.

The empty values (for omitempty option) are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero. 

type Tester struct {
	Test bool   `cork:"-"`              // Skip this field
	Name string `cork:"name"`           // Use key "name" in encode stream
	Size int32  `cork:"size"`           // Use key "size" in encode stream
	Data []byte `cork:"data,omitempty"` // Use key data in encode stream, and omit if empty
}

Example: 

// Encoding a typed value
var s string = "Hello"
buf := bytes.NewBuffer(nil)
err = cork.NewEncoder(buf).Encode(s)

// Encoding a struct
var t &Tester{Name: "Temp", Size: 0}
buf := bytes.NewBuffer(nil)
err = cork.NewEncoder(buf).Encode(t)

func (*Encoder) Options 

func (e *Encoder) Options(h *Handle) *Encoder

Options sets the configuration options that the Encoder should use.

func (*Encoder) Reset 

func (e *Encoder) Reset()

Reset flushes any remaing data and adds the Encoder back into the sync pool. If the Encoder was not originally from the sync pool, then the Encoder is discarded.

type Handle 

type Handle struct {
	// SortMaps specifies whether maps should be sorted before
	// being encoded into CORK. This guarantees that the same
	// input data is always encoded into the same binary data.
	SortMaps bool
	// ArrType specifies the type of slice to use when decoding
	// into a nil interface during schema-less decoding of a
	// slice in the stream.
	//
	// If not specified, we use []interface{}
	ArrType interface{}

	// MapType specifies the type of map to use when decoding
	// into a nil interface during schema-less decoding of a
	// map in the stream.
	//
	// If not specified, we use map[interface{}]interface{}
	MapType interface{}
}

type Reader 

type Reader struct {
	// contains filtered or unexported fields
}

Reader is used when self-decoding a cork.Selfer item from binary form.

func (*Reader) DecodeAny 

func (r *Reader) DecodeAny(v interface{})

DecodeAny decodes a value from the Reader.

func (*Reader) DecodeArr 

func (r *Reader) DecodeArr(v interface{})

DecodeArr decodes an array from the Reader.

func (*Reader) DecodeBool 

func (r *Reader) DecodeBool(v *bool)

DecodeBool decodes a boolean value from the Reader.

func (*Reader) DecodeByte 

func (r *Reader) DecodeByte(v *byte)

DecodeByte decodes a byte value from the Reader.

func (*Reader) DecodeBytes 

func (r *Reader) DecodeBytes(v *[]byte)

DecodeBytes decodes a byte slice value from the Reader.

func (*Reader) DecodeComplex128 

func (r *Reader) DecodeComplex128(v *complex128)

DecodeComplex128 decodes a complex128 value from the Reader.

func (*Reader) DecodeComplex64 

func (r *Reader) DecodeComplex64(v *complex64)

DecodeComplex64 decodes a complex64 value from the Reader.

func (*Reader) DecodeCorker 

func (r *Reader) DecodeCorker(v Corker)

DecodeCorker decodes a cork.Corker value from the Reader.

func (*Reader) DecodeFloat32 

func (r *Reader) DecodeFloat32(v *float32)

DecodeFloat32 decodes a float32 value from the Reader.

func (*Reader) DecodeFloat64 

func (r *Reader) DecodeFloat64(v *float64)

DecodeFloat64 decodes a float64 value from the Reader.

func (*Reader) DecodeInt 

func (r *Reader) DecodeInt(v *int)

DecodeInt decodes an int value from the Reader.

func (*Reader) DecodeInt16 

func (r *Reader) DecodeInt16(v *int16)

DecodeInt16 decodes an int16 value from the Reader.

func (*Reader) DecodeInt32 

func (r *Reader) DecodeInt32(v *int32)

DecodeInt32 decodes an int32 value from the Reader.

func (*Reader) DecodeInt64 

func (r *Reader) DecodeInt64(v *int64)

DecodeInt64 decodes an int64 value from the Reader.

func (*Reader) DecodeInt8 

func (r *Reader) DecodeInt8(v *int8)

DecodeInt8 decodes an int8 value from the Reader.

func (*Reader) DecodeInterface 

func (r *Reader) DecodeInterface(v *interface{})

DecodeInterface decodes any value from the Reader, with whatever type is next in the stream.

func (*Reader) DecodeMap 

func (r *Reader) DecodeMap(v interface{})

DecodeMap decodes a map from the Reader.

func (*Reader) DecodeReflect 

func (r *Reader) DecodeReflect(v reflect.Value)

DecodeReflect decodes a reflect.Value value from the Reader.

func (*Reader) DecodeSelfer 

func (r *Reader) DecodeSelfer(v Selfer)

DecodeSelfer decodes a cork.Selfer value from the Reader.

func (*Reader) DecodeString 

func (r *Reader) DecodeString(v *string)

DecodeString decodes a string value from the Reader.

func (*Reader) DecodeTime 

func (r *Reader) DecodeTime(v *time.Time)

DecodeTime decodes a time.Time value from the Reader.

func (*Reader) DecodeUint 

func (r *Reader) DecodeUint(v *uint)

DecodeUint decodes a uint value from the Reader.

func (*Reader) DecodeUint16 

func (r *Reader) DecodeUint16(v *uint16)

DecodeUint16 decodes a uint16 value from the Reader.

func (*Reader) DecodeUint32 

func (r *Reader) DecodeUint32(v *uint32)

DecodeUint32 decodes a uint32 value from the Reader.

func (*Reader) DecodeUint64 

func (r *Reader) DecodeUint64(v *uint64)

DecodeUint64 decodes a uint64 value from the Reader.

func (*Reader) DecodeUint8 

func (r *Reader) DecodeUint8(v *uint8)

DecodeUint8 decodes a uint8 value from the Reader.

type Selfer 

type Selfer interface {
	ExtendCORK() byte
	MarshalCORK(*Writer) error
	UnmarshalCORK(*Reader) error
}

Selfer represents an object which can encode and decode itself.

type Writer 

type Writer struct {
	// contains filtered or unexported fields
}

Writer is used when self-encoding a cork.Selfer item into binary form.

func (*Writer) EncodeAny 

func (w *Writer) EncodeAny(v interface{})

func (*Writer) EncodeArr 

func (w *Writer) EncodeArr(v interface{})

EncodeArr encodes an array to the Writer.

func (*Writer) EncodeBool 

func (w *Writer) EncodeBool(v bool)

EncodeBool encodes a boolean value to the Writer.

func (*Writer) EncodeByte 

func (w *Writer) EncodeByte(v byte)

EncodeByte encodes a byte value to the Writer.

func (*Writer) EncodeBytes 

func (w *Writer) EncodeBytes(v []byte)

EncodeBytes encodes a byte slice value to the Writer.

func (*Writer) EncodeComplex128 

func (w *Writer) EncodeComplex128(v complex128)

EncodeComplex128 encodes a complex128 value to the Writer.

func (*Writer) EncodeComplex64 

func (w *Writer) EncodeComplex64(v complex64)

EncodeComplex64 encodes a complex64 value to the Writer.

func (*Writer) EncodeCorker 

func (w *Writer) EncodeCorker(v Corker)

EncodeCorker encodes a cork.Corker value to the Writer.

func (*Writer) EncodeFloat32 

func (w *Writer) EncodeFloat32(v float32)

EncodeFloat32 encodes a float32 value to the Writer.

func (*Writer) EncodeFloat64 

func (w *Writer) EncodeFloat64(v float64)

EncodeFloat64 encodes a float64 value to the Writer.

func (*Writer) EncodeInt 

func (w *Writer) EncodeInt(v int)

EncodeInt encodes an int value to the Writer.

func (*Writer) EncodeInt16 

func (w *Writer) EncodeInt16(v int16)

EncodeInt16 encodes an int16 value to the Writer.

func (*Writer) EncodeInt32 

func (w *Writer) EncodeInt32(v int32)

EncodeInt32 encodes an int32 value to the Writer.

func (*Writer) EncodeInt64 

func (w *Writer) EncodeInt64(v int64)

EncodeInt64 encodes an int64 value to the Writer.

func (*Writer) EncodeInt8 

func (w *Writer) EncodeInt8(v int8)

EncodeInt8 encodes an int8 value to the Writer.

func (*Writer) EncodeMap 

func (w *Writer) EncodeMap(v interface{})

EncodeMap encodes a map to the Writer.

func (*Writer) EncodeNil 

func (w *Writer) EncodeNil()

EncodeBool writes a nil value to the Writer.

func (*Writer) EncodeReflect 

func (w *Writer) EncodeReflect(v reflect.Value)

EncodeReflect encodes a reflect.Value value to the Writer.

func (*Writer) EncodeSelfer 

func (w *Writer) EncodeSelfer(v Selfer)

EncodeSelfer encodes a cork.Selfer value to the Writer.

func (*Writer) EncodeString 

func (w *Writer) EncodeString(v string)

EncodeString encodes a string value to the Writer.

func (*Writer) EncodeTime 

func (w *Writer) EncodeTime(v time.Time)

EncodeInt encodes a time.Time value to the Writer.

func (*Writer) EncodeUint 

func (w *Writer) EncodeUint(v uint)

EncodeUint encodes a uint value to the Writer.

func (*Writer) EncodeUint16 

func (w *Writer) EncodeUint16(v uint16)

EncodeUint16 encodes a uint16 value to the Writer.

func (*Writer) EncodeUint32 

func (w *Writer) EncodeUint32(v uint32)

EncodeUint32 encodes a uint32 value to the Writer.

func (*Writer) EncodeUint64 

func (w *Writer) EncodeUint64(v uint64)

EncodeUint64 encodes a uint64 value to the Writer.

func (*Writer) EncodeUint8 

func (w *Writer) EncodeUint8(v uint8)

EncodeUint8 encodes a uint8 value to the Writer.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.