dwire

README

dwire

A library for encoding and decoding things using DDRP's wire format.

Usage

To encode a single value:

import (
	"github.com/ddrp-org/dwire"
	"ddrp"
)

value := "a string value"
var w bytes.Buffer
if err := dwire.EncodeField(&w, value); err != nil {
	log.Fatal(err)
}

To encode a struct:

import (
	"github.com/ddrp-org/dwire"
	"bytes"
)

type Foo struct {
	StrField string
	Uint8Field uint8
}

foo := &Foo{
	StrField: "some string",
	Uint8Field: 1
}

var w bytes.Buffer
if err := dwire.EncodeFields(&w, foo.StrField, foo.Uint8Field); err != nil {
	log.Fatal(err)
}

Decoding works on pointer values. For example, to decode a single value:

import (
	"github.com/ddrp-org/dwire"
	"bytes"
)

var uint8Value uint8
r := bytes.NewReader([]byte{ 0x01 })
if err := dwire.DecodeField(r, &uint8Value); err != nil {
	log.Fatal(err)
}

To decode a struct:

import (
	"github.com/ddrp-org/dwire"
	"bytes"
)

type Foo struct {
	Uint8Field uint8
}

var foo Foo
r := bytes.NewReader([]byte{0x01})

if err := dwire.DecodeFields(r, &foo.Uint8Field); err != nil {
	log.Fatal(err)
}

For convenience, objects that implement the Encoder and Decoder interfaces can be passed directly to dwire's encoding/decodig methods. For example, the first struct example above could be implemented as follows:

import (
	"github.com/ddrp-org/dwire"
	"bytes"
)

type Foo struct {
	StrField string
	Uint8Field uint8
}

func (f *Foo) Encode(w io.Writer) error {
	return dwire.EncodeFields(
		w,
		f.StrField,
		f.Uint8Field,
	)
}

func (f *Foo) Decode(r io.Reader) error {
	return dwire.DecodeFields(
		r,
		&f.StrField,
		&f.Uint8Field,
	)
}

foo := &Foo{
	StrField: "some string",
	Uint8Field: 1
}

var w bytes.Buffer
if err := dwire.EncodeField(&w, foo); err != nil {
	log.Fatal(err)
}

Wire Format Definition

This definition is lifted from the canonical specification described in PIP-1. For clarity, we will use the function Encode(t) = b, where t is the inputted field and b represents the outputted bytes, to provide example encodings where necessary.

dwire defines encodings for the following types:

1. bool: Encoded as 0x01 or 0x00 if the value is true or false, respectively.
2. uint8, uint16, uint32, uint64: Encoded as big-endian unsigned integers.
3. byte: Encoded as uint8.
4. [N]<T> (i.e., fixed-length arrays of element T): Encoded as the concatenation of Encode(<T>) for each array element.
5. []<T> (i.e., variable-length arrays of element T): Encoded as the concatenation of a Uvarint length prefix and Encode(<T>) for each array element.
6. string: Encoded as []byte.

Well-Known Types

Certain complex types are considered "well known," and are encoded/decoded directly in this package without implementing the Encoder/Decoder interfaces. These types are:

1. time.Time: Encoded as a uint64 Unix timestamp.

Benchmarks

Benchmarks recorded on a mid-2018 MacBook Pro with the following specifications:

1. 2.2GHz 6-Core Intel i7
2. 32GB 2400MHz DDR4

goos: darwin
goarch: amd64
pkg: github.com/ddrp-org/dwire
BenchmarkUint8Encoding-12                 	  711883	      1469 ns/op
BenchmarkUint16Encoding-12                	  867793	      1472 ns/op
BenchmarkUint32Encoding-12                	  841219	      1446 ns/op
BenchmarkUint64Encoding-12                	  845859	      1544 ns/op
BenchmarkByteSliceEncoding1024-12         	  744835	      1623 ns/op
BenchmarkStringEncoding1024-12            	  710882	      1845 ns/op
BenchmarkByteArrayEncoding32-12           	  748132	      1551 ns/op
BenchmarkByteArrayEncodingReflect-12      	  577855	      2178 ns/op
BenchmarkUint16ArrayEncodingReflect-12    	   30091	     41010 ns/op
BenchmarkStringArrayEncodingReflect-12    	   10000	    121458 ns/op
BenchmarkWellKnownTimeEncoding-12         	  701935	      1845 ns/op

Acknowledgements

Much of this library was directly inspired by lnwire, the wire encoding library used by the Lightning Network. As such, we would like to extend our deepest thanks to the Lightning team for their trailblazing work.

Documentation

Overview

Package dwire implements the DDRP message encoding scheme as defined in PIP-1.

Fundamental types:

• bool: Encoded as 0x00 or 0x01 if the value is true or false, respectively.
• uint8: Encoded as a single byte in the range 0x00-0xff.
• uint16: Encoded as two big-endian bytes in the range 0x0000-0xffff.
• uint32: Encoded as four big-endian bytes in the range 0x00000000-0xffffffff
• uint64: Encoded as eight big-endian bytes in the range 0x0000000000000000-0xffffffffffffffff.
• string: Encoded as a UTF-8 []byte.
• [N]T: Encoded as the concatenation of the encoding of T.
• []T: Encoded as a binary.Uvarint length prefix followed by the concatenation of the encoding of T.

Well-known types:

• time.Time: Encoded as uint32(time.Unix()).

The easiest way to use this library is to call the EncodeField/DecodeField family of methods. To encode a value into a Writer:

value1 := "this is my value"
value2 := 2
err := dwire.EncodeFields(w, value1, value2)

To decode a value from a reader:

var value1 string
var value2 string
err := dwire.DecodeFields(r, &value1, &value2)

Note that values passed to DecodeField/DecodeFields MUST be pointers.

dwire exposes Encoder and Decoder interfaces, which allow arbitrary types to be encoded and decoded by EncodeField/DecodeField. For example:

type Foo struct {
	Value string
}

func (f *Foo) Encode(w io.Writer) error {
	return dwire.EncodeFields(w, f.Value)
}

func (f *Foo) Decode(r io.Reader) error {
	return dwire.DecodeFields(r, &f.Value)
}

Index

• Constants
• func DecodeField(r io.Reader, item interface{}) error
• func DecodeFields(r io.Reader, items ...interface{}) error
• func DecodeTime(r io.Reader, val interface{}) error
• func EncodeField(w io.Writer, item interface{}) error
• func EncodeFields(w io.Writer, items ...interface{}) error
• func EncodeTime(w io.Writer, val interface{}) error
• type ConfiguredEncoder
  • func (c *ConfiguredEncoder) DecodeField(r io.Reader, item interface{}) error
  • func (c *ConfiguredEncoder) DecodeFields(r io.Reader, items ...interface{}) error
  • func (c *ConfiguredEncoder) EncodeField(w io.Writer, item interface{}) error
  • func (c *ConfiguredEncoder) EncodeFields(w io.Writer, items ...interface{}) error
• type Decoder
• type EncodeDecoder
• type Encoder

Constants

const (
	DefaultMaxVariableArrayLen = 1024
	DefaultMaxByteFieldLen     = 256 * 1024
)

Functions

func DecodeField

func DecodeField(r io.Reader, item interface{}) error

DecodeFields decodes the field in the item argument from the Reader using the default Encoder. The item provided to DecodeField must be a pointer type.

func DecodeFields

func DecodeFields(r io.Reader, items ...interface{}) error

DecodeFields decodes each field in the variadic items argument from the Reader using the default Encoder. Items provided to DecodeFields must be pointer types.

func DecodeTime

func DecodeTime(r io.Reader, val interface{}) error

Decodes a time.Time value from the Reader. Since time.Time is a well-known type, you likely do not need to call this method directly. Instead, call EncodeField or EncodeFields with a &time.Time item.

func EncodeField

func EncodeField(w io.Writer, item interface{}) error

EncodeField encodes a single field into the Writer using the default Encoder.

func EncodeFields

func EncodeFields(w io.Writer, items ...interface{}) error

EncodeFields encodes each field in the variadic items argument into the Writer using the default Encoder.

func EncodeTime

func EncodeTime(w io.Writer, val interface{}) error

Encodes a time.Time value into the Writer. Since time.Time is a well-known type, you likely do not need to call this method directly. Instead, provide the time value to EncodeField or EncodeFields.

Types

type ConfiguredEncoder

type ConfiguredEncoder struct {
	// MaxVariableArrayLen is the maximum length of a variable-length array dwire
	// will decode before stopping early.
	MaxVariableArrayLen int

	// MaxByteFieldLen is the maximum length of a variable-length byte array field
	// dwire will decode before stopping early.
	MaxByteFieldLen uint64
}

func (*ConfiguredEncoder) DecodeField

func (c *ConfiguredEncoder) DecodeField(r io.Reader, item interface{}) error

DecodeField decodes the field in the item argument from the Reader. The item provided to DecodeField must be a pointer type.

func (*ConfiguredEncoder) DecodeFields

func (c *ConfiguredEncoder) DecodeFields(r io.Reader, items ...interface{}) error

DecodeFields decodes each field in the variadic items argument from the Reader. Items provided to DecodeFields must be pointer types.

func (*ConfiguredEncoder) EncodeField

func (c *ConfiguredEncoder) EncodeField(w io.Writer, item interface{}) error

EncodeFields encodes a single field into the Writer.

func (*ConfiguredEncoder) EncodeFields

func (c *ConfiguredEncoder) EncodeFields(w io.Writer, items ...interface{}) error

EncodeFields encodes each field in the variadic items argument into the Writer.

type Decoder

type Decoder interface {
	Decode(r io.Reader) error
}

Decoder is an interface that allows arbitrary types to be decoded. Types implementing the Decoder interface can be decoded using DecodeField or DecodeFields.

type EncodeDecoder

type EncodeDecoder interface {
	Encoder
	Decoder
}

type Encoder

type Encoder interface {
	Encode(w io.Writer) error
}

Encoder is an interface that allows arbitrary types to be encoded. Types implementing the Encoder interface can be encoded using EncodeField or EncodeFields.