wire

package module
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 19, 2021 License: Apache-2.0 Imports: 17 Imported by: 15

README

Wire encoding for Golang

This software implements Go bindings for the Wire encoding protocol. The goal of the Wire encoding protocol is to be a simple language-agnostic encoding protocol for rapid prototyping of blockchain applications.

This package also includes a compatible (and slower) JSON codec.

Supported types

Primary types: uvarint, varint, byte, uint[8,16,32,64], int[8,16,32,64], string, and time types are supported

Arrays: Arrays can hold items of any arbitrary type. For example, byte-arrays and byte-array-arrays are supported.

Structs: Struct fields are encoded by value (without the key name) in the order that they are declared in the struct. In this way it is similar to Apache Avro.

Interfaces: Interfaces are like union types where the value can be any non-interface type. The actual value is preceded by a single "type byte" that shows which concrete is encoded.

Pointers: Pointers are like optional fields. The first byte is 0x00 to denote a null pointer (e.g. no value), otherwise it is 0x01.

Unsupported types

Maps: Maps are not supported because for most languages, key orders are nondeterministic. If you need to encode/decode maps of arbitrary key-value pairs, encode an array of {key,value} structs instead.

Floating points: Floating point number types are discouraged because of reasons. If you need to use them, use the field tag wire:"unsafe".

Enums: Enum types are not supported in all languages, and they're simple enough to model as integers anyways.

A struct example

Struct types can be automatically encoded with reflection. Unlike json-encoding, no field name or type information is encoded. Field values are simply encoded in order.

package main

import (
  "bytes"
  "fmt"
  "math"
  "github.com/Gessiux/go-wire"
)

type Foo struct {
  MyString       string
  MyUint32       uint32
  myPrivateBytes []byte
}

func main() {

  foo := Foo{"my string", math.MaxUint32, []byte("my private bytes")}

  buf, n, err := new(bytes.Buffer), int(0), error(nil)
  wire.WriteBinary(foo, buf, &n, &err)

  fmt.Printf("%X\n", buf.Bytes())
}

The above example prints:

01096D7920737472696E67FFFFFFFF, where

0109                            is the varint encoding of the length of string "my string"
    6D7920737472696E67          is the bytes of string "my string"
                      FFFFFFFF  is the bytes for math.MaxUint32, a uint32

Note that the unexported "myPrivateBytes" isn't encoded.

An interface example

Here's an example with interfaces.

package main

import (
  "bytes"
  "fmt"
  "github.com/Gessiux/go-wire"
)

type Animal interface{}
type Dog struct{ Name string }
type Cat struct{ Name string }
type Cow struct{ Name string }

var _ = wire.RegisterInterface(
  struct{ Animal }{},
  wire.ConcreteType{Dog{}, 0x01}, // type-byte of 0x01 for Dogs
  wire.ConcreteType{Cat{}, 0x02}, // type-byte of 0x02 for Cats
  wire.ConcreteType{Cow{}, 0x03}, // type-byte of 0x03 for Cows
)

func main() {

  animals := []Animal{
    Dog{"Snoopy"},
    Cow{"Daisy"},
  }

  buf, n, err := new(bytes.Buffer), int(0), error(nil)
  wire.WriteBinary(animals, buf, &n, &err)

  fmt.Printf("%X\n", buf.Bytes())
}

The above example prints:

0102010106536E6F6F70790301054461697379, where

0102                                    is the varint encoding of the length of the array
    01                                  is the type-byte for a Dog
      0106                              is the varint encoding of the length of the Dog's name
          536E6F6F7079                  is the Dog's name "Snoopy"
                      03                is the type-byte for a Cow
                        0105            is the varint encoding of the length of the Cow's name
                            4461697379  is the Cow's name "Daisy"
A pointer example

Here's an example with pointers (and interfaces too).

package main

import (
	"bytes"
	"fmt"
	"github.com/Gessiux/go-wire"
)

type Animal interface{}
type Dog struct{ Name string }
type Cat struct{ Name string }
type Cow struct{ Name string }

var _ = wire.RegisterInterface(
	struct{ Animal }{},
	wire.ConcreteType{Dog{}, 0x01},  // type-byte of 0x01 for Dogs
	wire.ConcreteType{&Dog{}, 0x02}, // type-byte of 0x02 for Dog pointers
)

type MyStruct struct {
	Field1 Animal
	Field2 *Dog
	Field3 *Dog
}

func main() {

	myStruct := MyStruct{
		Field1: &Dog{"Snoopy"},
		Field2: &Dog{"Smappy"},
		Field3: (*Dog)(nil),
	}

	buf, n, err := new(bytes.Buffer), int(0), error(nil)
	wire.WriteBinary(myStruct, buf, &n, &err)

	fmt.Printf("%X\n", buf.Bytes())
}

The above example prints:

020106536E6F6F7079010106536D6170707900, where

02                                      is the type-byte for a Dog pointer for Field1
  0106                                  is the varint encoding of the length of the Dog's name
      536E6F6F7079                      is the Dog's name "Snoopy"
                  01                    is a byte indicating a non-null pointer for Field2
                    0106                is the varint encoding of the length of the Dog's name
                        536D61707079    is the Dog's name "Smappy"
                                    00  is a byte indicating a null pointer for Field3

Notice that in Field1, that the value is non-null is implied in the type-byte of 0x02. While Golang lets you have nil-pointers as interface values, this is a Golang-specific feature that is absent in other OOP languages such as Java. So, Go-Wire does not support nil-pointers for interface values. The following example would return an error:

myStruct := MyStruct{
  Field1: (*Dog)(nil),    // Error!
  Field2: &Dog{"Smappy"}, // Ok!
  Field3: (*Dog)(nil),    // Ok!
}

buf, n, err := new(bytes.Buffer), int(0), error(nil)
wire.WriteBinary(myStruct, buf, &n, &err)
fmt.Println(err)

// Unexpected nil-pointer of type main.Dog for registered interface Animal.
// For compatibility with other languages, nil-pointer interface values are forbidden.

Documentation

Index

Constants

View Source 
const (
	RFC3339Millis = "2006-01-02T15:04:05.000Z" // forced microseconds
)
View Source 
const (
	ReadSliceChunkSize = 1024
)
View Source 
const Version = "0.6.1"

Variables

View Source 
var BasicCodec = Codec{
	Encode:  BasicCodecEncoder,
	Decode:  BasicCodecDecoder,
	Compare: BasicCodecComparator,
}
View Source 
var BytesCodec = Codec{
	Encode:  BytesCodecEncoder,
	Decode:  BytesCodecDecoder,
	Compare: BytesCodecComparator,
}
View Source 
var ErrBinaryReadInvalidLength = errors.New("Error: binary read invalid length")
View Source 
var ErrBinaryReadOverflow = errors.New("Error: binary read overflow")
View Source 
var ErrBinaryWriteOverflow = errors.New("Error: binary write overflow")

Functions

func BasicCodecComparator 

func BasicCodecComparator(o1 interface{}, o2 interface{}) int

Contract: Caller must ensure that types match.

func BasicCodecDecoder 

func BasicCodecDecoder(r io.Reader, n *int, err *error) (o interface{})

func BasicCodecEncoder 

func BasicCodecEncoder(o interface{}, w io.Writer, n *int, err *error)

func BinaryBytes 

func BinaryBytes(o interface{}) []byte

func BinaryCompare 

func BinaryCompare(a, b interface{}) int

NOTE: does not care about the type, only the binary representation.

func BinaryEqual 

func BinaryEqual(a, b interface{}) bool

NOTE: does not care about the type, only the binary representation.

func BinaryRipemd160 

func BinaryRipemd160(o interface{}) []byte

NOTE: The default hash function is Ripemd160.

func BinarySha256 

func BinarySha256(o interface{}) []byte

NOTE: only use this if you need 32 bytes.

func ByteSliceSize 

func ByteSliceSize(bz []byte) int

Returns the total encoded size of a byteslice

func BytesCodecComparator 

func BytesCodecComparator(o1 interface{}, o2 interface{}) int

func BytesCodecDecoder 

func BytesCodecDecoder(r io.Reader, n *int, err *error) (o interface{})

func BytesCodecEncoder 

func BytesCodecEncoder(o interface{}, w io.Writer, n *int, err *error)

func GetBool 

func GetBool(buf []byte) (bool, error)

func GetByteSlice 

func GetByteSlice(buf []byte) (bz []byte, n int, err error)

func GetInt16 

func GetInt16(buf []byte) int16

func GetInt32 

func GetInt32(buf []byte) int32

func GetInt64 

func GetInt64(buf []byte) int64

func GetString 

func GetString(buf []byte) (s string, n int, err error)

func GetTypeFromStructDeclaration 

func GetTypeFromStructDeclaration(o interface{}) reflect.Type

e.g. If o is struct{Foo}{}, return is the Foo reflection type.

func GetUint16 

func GetUint16(buf []byte) uint16

func GetUint32 

func GetUint32(buf []byte) uint32

func GetUint64 

func GetUint64(buf []byte) uint64

func GetUvarint 

func GetUvarint(buf []byte) (i uint, n int, err error)

func GetVarint 

func GetVarint(buf []byte) (i int, n int, err error)

func JSONBytes 

func JSONBytes(o interface{}) []byte

func JSONBytesPretty 

func JSONBytesPretty(o interface{}) []byte

NOTE: inefficient

func PutBool 

func PutBool(buf []byte, b bool)

func PutByteSlice 

func PutByteSlice(buf []byte, bz []byte) (n int, err error)

func PutInt16 

func PutInt16(buf []byte, i int16)

func PutInt32 

func PutInt32(buf []byte, i int32)

func PutInt64 

func PutInt64(buf []byte, i int64)

func PutString 

func PutString(buf []byte, s string) (n int, err error)

func PutUint16 

func PutUint16(buf []byte, i uint16)

func PutUint32 

func PutUint32(buf []byte, i uint32)

func PutUint64 

func PutUint64(buf []byte, i uint64)

func PutUvarint 

func PutUvarint(buf []byte, i uint) (n int, err error)

func PutVarint 

func PutVarint(buf []byte, i int) (n int, err error)

func ReadBinary 

func ReadBinary(o interface{}, r io.Reader, lmt int, n *int, err *error) (res interface{})

func ReadBinaryBytes 

func ReadBinaryBytes(d []byte, ptr interface{}) error

ptr: a pointer to the object to be filled

func ReadBinaryPtr 

func ReadBinaryPtr(o interface{}, r io.Reader, lmt int, n *int, err *error) (res interface{})

func ReadBinaryPtrLengthPrefixed 

func ReadBinaryPtrLengthPrefixed(o interface{}, r io.Reader, lmt int, n *int, err *error) (res interface{})

func ReadBool 

func ReadBool(r io.Reader, n *int, err *error) bool

func ReadByte 

func ReadByte(r io.Reader, n *int, err *error) byte

func ReadByteSlice 

func ReadByteSlice(r io.Reader, lmt int, n *int, err *error) []byte

func ReadByteSlices 

func ReadByteSlices(r io.Reader, lmt int, n *int, err *error) [][]byte

func ReadFloat32 

func ReadFloat32(r io.Reader, n *int, err *error) float32

func ReadFloat64 

func ReadFloat64(r io.Reader, n *int, err *error) float64

func ReadFull 

func ReadFull(buf []byte, r io.Reader, n *int, err *error)

Read len(buf) from r Increment n and set err accordingly.

func ReadInt16 

func ReadInt16(r io.Reader, n *int, err *error) int16

func ReadInt32 

func ReadInt32(r io.Reader, n *int, err *error) int32

func ReadInt64 

func ReadInt64(r io.Reader, n *int, err *error) int64

func ReadInt8 

func ReadInt8(r io.Reader, n *int, err *error) int8

func ReadJSON 

func ReadJSON(o interface{}, bytes []byte, err *error) interface{}

func ReadJSONBytes 

func ReadJSONBytes(d []byte, ptr interface{}) (err error)

ptr: a pointer to the object to be filled

func ReadJSONObject 

func ReadJSONObject(o interface{}, object interface{}, err *error) interface{}

o is the ultimate destination, object is the result of json unmarshal

func ReadJSONObjectPtr 

func ReadJSONObjectPtr(o interface{}, object interface{}, err *error) interface{}

func ReadJSONPtr 

func ReadJSONPtr(o interface{}, bytes []byte, err *error) interface{}

func ReadString 

func ReadString(r io.Reader, lmt int, n *int, err *error) string

func ReadTime 

func ReadTime(r io.Reader, n *int, err *error) time.Time

func ReadUint16 

func ReadUint16(r io.Reader, n *int, err *error) uint16

func ReadUint16s 

func ReadUint16s(r io.Reader, n *int, err *error) []uint16

func ReadUint32 

func ReadUint32(r io.Reader, n *int, err *error) uint32

func ReadUint64 

func ReadUint64(r io.Reader, n *int, err *error) uint64

func ReadUint8 

func ReadUint8(r io.Reader, n *int, err *error) uint8

func ReadUvarint 

func ReadUvarint(r io.Reader, n *int, err *error) uint

func ReadVarint 

func ReadVarint(r io.Reader, n *int, err *error) int

func UvarintSize 

func UvarintSize(i uint64) int

func WriteBinary 

func WriteBinary(o interface{}, w io.Writer, n *int, err *error)

func WriteBinaryLengthPrefixed 

func WriteBinaryLengthPrefixed(o interface{}, w io.Writer, n *int, err *error)

func WriteBool 

func WriteBool(b bool, w io.Writer, n *int, err *error)

func WriteByte 

func WriteByte(b byte, w io.Writer, n *int, err *error)

func WriteByteSlice 

func WriteByteSlice(bz []byte, w io.Writer, n *int, err *error)

func WriteByteSlices 

func WriteByteSlices(bzz [][]byte, w io.Writer, n *int, err *error)

func WriteFloat32 

func WriteFloat32(f float32, w io.Writer, n *int, err *error)

func WriteFloat64 

func WriteFloat64(f float64, w io.Writer, n *int, err *error)

func WriteInt16 

func WriteInt16(i int16, w io.Writer, n *int, err *error)

func WriteInt32 

func WriteInt32(i int32, w io.Writer, n *int, err *error)

func WriteInt64 

func WriteInt64(i int64, w io.Writer, n *int, err *error)

func WriteInt8 

func WriteInt8(i int8, w io.Writer, n *int, err *error)

func WriteJSON 

func WriteJSON(o interface{}, w io.Writer, n *int, err *error)

func WriteString 

func WriteString(s string, w io.Writer, n *int, err *error)

func WriteTime 

func WriteTime(t time.Time, w io.Writer, n *int, err *error)

func WriteTo 

func WriteTo(bz []byte, w io.Writer, n *int, err *error)

Write all of bz to w Increment n and set err accordingly.

func WriteUint16 

func WriteUint16(i uint16, w io.Writer, n *int, err *error)

func WriteUint16s 

func WriteUint16s(iz []uint16, w io.Writer, n *int, err *error)

func WriteUint32 

func WriteUint32(i uint32, w io.Writer, n *int, err *error)

func WriteUint64 

func WriteUint64(i uint64, w io.Writer, n *int, err *error)

func WriteUint8 

func WriteUint8(i uint8, w io.Writer, n *int, err *error)

func WriteUvarint 

func WriteUvarint(i uint, w io.Writer, n *int, err *error)

func WriteVarint 

func WriteVarint(i int, w io.Writer, n *int, err *error)

Types

type Codec 

type Codec struct {
	Encode  Encoder
	Decode  Decoder
	Compare Comparator
}

type Comparator 

type Comparator func(o1 interface{}, o2 interface{}) int

type ConcreteType 

type ConcreteType struct {
	O    interface{}
	Byte byte
}

For use with the RegisterInterface declaration

type Decoder 

type Decoder func(r io.Reader, n *int, err *error) interface{}

type Encoder 

type Encoder func(o interface{}, w io.Writer, n *int, err *error)

type Options 

type Options struct {
	JSONName      string      // (JSON) Corresponding JSON field name. (override with `json=""`)
	JSONOmitEmpty bool        // (JSON) Omit field if value is empty
	Varint        bool        // (Binary) Use length-prefixed encoding for (u)int64
	Unsafe        bool        // (JSON/Binary) Explicitly enable support for floats or maps
	ZeroValue     interface{} // Prototype zero object
}

type StructFieldInfo 

type StructFieldInfo struct {
	Index   int          // Struct field index
	Type    reflect.Type // Struct field type
	Options              // Encoding options
}

type TypeInfo 

type TypeInfo struct {
	Type reflect.Type // The type

	// If Type is kind reflect.Interface, is registered
	IsRegisteredInterface bool
	ByteToType            map[byte]reflect.Type
	TypeToByte            map[reflect.Type]byte

	// If Type is kind reflect.Struct
	Fields []StructFieldInfo
	Unwrap bool // if struct has only one field and its an anonymous interface
}

func GetTypeInfo 

func GetTypeInfo(rt reflect.Type) *TypeInfo

func MakeTypeInfo 

func MakeTypeInfo(rt reflect.Type) *TypeInfo

func RegisterInterface 

func RegisterInterface(o interface{}, ctypes ...ConcreteType) *TypeInfo

Must use this to register an interface to properly decode the underlying concrete type.

Source Files

View all Source files

Directories

Path Synopsis
cmd
wire

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.