encoding

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 24, 2019 License: MIT Imports: 4 Imported by: 0

README

Encoding

The encoding library provides methods to serialize different data types for storage and transfer over the wire protocol. The library wraps around the standard Read and Write functions of an io.Reader or io.Writer, to provide simple methods for any basic data type. The library is built to mimick the way the binary.Read and binary.Write functions work, but aims to simplify the code to gain processing speed, and avoid reflection-based encoding/decoding. As a general rule, for encoding a variable, you should pass it by value, and for decoding a value, you should pass it by reference. For an example of this, please refer to the tests or any implementation of this library in the codebase.

Methods

Integers (integers.go)

Functions are provided for integers of any size (between 8 and 64 bit). All integers will need to be passed and retrieved as their unsigned format. Signed integers can be converted to and from their unsigned format by simple type conversion.

The functions are declared like this:

Reading

  • ReadUint8(r io.Reader, v *uint8) error
  • ReadUint16(r io.Reader, o binary.ByteOrder, v *uint16) error
  • ReadUint32(r io.Reader, o binary.ByteOrder, v *uint32) error
  • ReadUint64(r io.Reader, o binary.ByteOrder, v *uint64) error

Writing

  • WriteUint8(w io.Writer, v uint8) error
  • WriteUint16(w io.Writer, o binary.ByteOrder, v uint16) error
  • WriteUint32(w io.Writer, o binary.ByteOrder, v uint32) error
  • WriteUint64(w io.Writer, o binary.ByteOrder, v uint64) error

Usage of this module will be as follows. Let's take a uint64 for example:

// Error handling omitted for clarity
var x uint64 = 5

buf := new(bytes.Buffer)

err := encoding.PutUint64(buf, binary.LittleEndian, x)

// buf.Bytes() = [5 0 0 0 0 0 0 0]

var y uint64
err := encoding.Uint64(buf, binary.LittleEndian, &y)

// y = 5

For an int64 you would write:

// Error handling omitted for clarity
var x int64 = -5

buf := new(bytes.Buffer)

err := encoding.PutUint64(buf, binary.LittleEndian, uint64(x))

// buf.Bytes() = [251 255 255 255 255 255 255 255]

var y uint64
err := encoding.Uint64(buf, binary.LittleEndian, &y)
z := int64(y)

// z = -5

The encoding module uses this same function template for all other data structures, except for CompactSize integers, which are explained below.

CompactSize integers (varint.go)

The library provides an implementation of CompactSize integers as described in the Bitcoin Developer Reference. The functions implemented are as follows:

  • ReadVarInt(r io.Reader) (uint64, error) for reading CompactSize integers
  • WriteVarInt(w io.Writer, v uint64) error for writing CompactSize integers
  • VarIntEncodeSize(v uint64) error for determining how many bytes an integer will take up through CompactSize encoding

For reading and writing CompactSize integers, the functions make use of the aforementioned integer serialization functions. CompactSize integers are best used when serializing an array of variable size, for example when serializing the inputs and outputs of a transaction.

A quick usage example:

// Error handling omitted for clarity

x := []int{5, 2, 98, 200}
l := len(x)
buf := new(bytes.Buffer)
err := encoding.WriteVarInt(buf, uint64(l))

// And then, to get it out...
n, err := encoding.ReadVarInt(buf)
Booleans (miscdata.go)

Booleans can be written and read much like integers, but they have their own function so that they can be passed without any conversion beforehand. The functions are:

  • ReadBool(r io.Reader, b *bool) error to read a bool
  • WriteBool(w io.Writer, b bool) error to write a bool

Boolean functions use the standard template as demonstrated at the Integers section.

256-bit and 512-bit data structures (miscdata.go)

For data that is either 256 or 512 bits long (such as hashes and signatures), separate functions are defined.

For 256-bit data:

  • Read256(r io.Reader, b *[]byte) error to read 256 bits of data
  • Write256(w io.Writer, b []byte) error to write 256 bits of data

For 512-bit data:

  • Read512(r io.Reader, b *[]byte) error to read 512 bits of data
  • Write512(w io.Writer, b []byte) error to write 512 bits of data

256-bit and 512-bit functions use the standard template as demonstrated at the Integers section.

Strings and byte slices (vardata.go)

Functions in this source file will make use of CompactSize integers for embedding and reading length prefixes. For byte slices of variable length there are two functions.

  • ReadVarBytes(r io.Reader, b *[]byte) error to read byte data
  • WriteVarBytes(w io.Writer, b []byte) error to write byte data

For strings of variable length (like a message) there are two convenience functions, which point to the functions above.

  • ReadString(r io.Reader, s *string) error
  • WriteString(w io.Writer, s string) error

Writing a string will simply convert it to a byte slice, and reading it will take the byte slice and convert it to a string.

String and byte slice functions use the standard template as demonstrated at the Integers section.

###Arrays/slices

For arrays or slices, you should use a CompactSize integer to denote the length of the array, followed by the data, serialized in a for loop.

The method is as follows (I will take a slice of strings for this example):

// Error handling omitted for clarity

slice := []string{"foo", "bar", "baz"}

buf := new(bytes.Buffer)
err := encoding.WriteVarInt(buf, uint64(len(slice)))
// Handle error

for _, str := range slice {
	err := encoding.WriteString(buf, str)
}

// buf.Bytes() = [3 3 102 111 111 3 98 97 114 3 98 97 122]

count, err := ReadVarInt(buf)
// Handle error

// Make new array with len 0 and cap count
newSlice := make([]string, 0, count)
for i := uint64(0); i < count; i++ {
	err := encoding.ReadString(buf, *newSlice[i])
}

// newArr = ["foo" "bar" "baz"]

###Structs

For structs, the best way of doing this is to create an Encode() and Decode() method. In these functions you should go off each field one by one, and encode them into the buffer, or decode them one by one from the buffer, and then populate the respective fields.

A more advanced example can be found in the transactions folder. I will demonstrate a small one here:

// Error handling omitted for clarity

type S struct {
	Str string
	A []uint64
	B bool
}

func (s *S) Encode(w io.Writer) error {
	// Str
	err := encoding.WriteString(w, s.Str)
	
	// A
	err := encoding.WriteVarInt(w, uint64(len(s.A)))
	for _, n := range s.A {
		err := encoding.WriteUint64(w, binary.LittleEndian, n)
	}
	
	// B
	err := encoding.WriteBool(w, s.B)
}

func (s *S) Decode(r io.Reader) error {
	// Str
	err := encoding.ReadString(r, &s.Str)
	
	// A
	count, err := encoding.ReadVarInt(r)
	
	s.A = make([]uint64, 0, count)
	for i := uint64(0); i < length; i++ {
		n, err := encoding.ReadUint64(r, binary.LittleEndian, &s.A[i])
	}
	
	// B
	err := encoding.ReadBool(r, &s.B)
	
    return nil
}

As a sidenote for structs, it's advisable to include a function like GetEncodeSize() as seen in the stealthtx source file. Knowing how long the buffer has to be before running Encode() will reduce allocations during encoding, and increase performance (especially with transactions and blocks as they will contain many different fields).

count := StealthTX.GetEncodeSize()
bs := make([]byte, 0, count)
buf := bytes.NewBuffer(bs)
err := StealthTX.Encode(buf)

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Read256 

func Read256(r *bytes.Buffer, b []byte) error

Read256 will read 32 bytes from r and return them as a slice of bytes.

func Read512 

func Read512(r *bytes.Buffer, b []byte) error

Read512 will read 64 bytes from r and return them as a slice of bytes.

func ReadBLS 

func ReadBLS(r *bytes.Buffer, b []byte) error

ReadBLS will read a compressed bls signature (33 bytes) from r and return it as a slice of bytes.

func ReadBool 

func ReadBool(r *bytes.Buffer, b *bool) error

ReadBool will read a single byte from r, turn it into a bool and return it.

func ReadString 

func ReadString(r *bytes.Buffer) (string, error)

ReadString reads the data with ReadVarBytes and returns it as a string by simple type conversion.

func ReadUint16LE added in v0.2.0 

func ReadUint16LE(r *bytes.Buffer, v *uint16) error

ReadUint16LE will read two bytes and convert them to a uint16 assuming little-endian byte order. The result is put into v.

func ReadUint32LE added in v0.2.0 

func ReadUint32LE(r *bytes.Buffer, v *uint32) error

ReadUint32LE will read four bytes and convert them to a uint32 assuming little-endian byte order. The result is put into v.

func ReadUint64LE added in v0.2.0 

func ReadUint64LE(r *bytes.Buffer, v *uint64) error

ReadUint64LE will read eight bytes and convert them to a uint64 assuming little-endian byte order. The result is put into v.

func ReadUint8 

func ReadUint8(r *bytes.Buffer, v *uint8) error

ReadUint8 will read a single byte into v.

func ReadVarBytes 

func ReadVarBytes(r *bytes.Buffer, b *[]byte) error

ReadVarBytes will read a CompactSize int denoting the length, then proceeds to read that amount of bytes from r into b.

func ReadVarInt 

func ReadVarInt(r *bytes.Buffer) (uint64, error)

ReadVarInt reads the discriminator byte of a CompactSize int, and then deserializes the number accordingly.

func VarIntEncodeSize 

func VarIntEncodeSize(v uint64) uint64

VarIntEncodeSize returns the number of bytes needed to serialize a CompactSize int of size v

func Write256 

func Write256(w *bytes.Buffer, b []byte) error

Write256 will check the length of b and then write to w.

func Write512 

func Write512(w *bytes.Buffer, b []byte) error

Write512 will check the length of b and then write to w.

func WriteBLS 

func WriteBLS(w *bytes.Buffer, b []byte) error

WriteBLS will write a compressed bls signature (33 bytes) to w.

func WriteBool 

func WriteBool(w *bytes.Buffer, b bool) error

WriteBool will write a boolean value as a single byte into w.

func WriteString 

func WriteString(w *bytes.Buffer, s string) error

WriteString will write string s as a slice of bytes through WriteVarBytes.

func WriteUint16LE added in v0.2.0 

func WriteUint16LE(w *bytes.Buffer, v uint16) error

WriteUint16LE will write two bytes in little-endian byte order.

func WriteUint32LE added in v0.2.0 

func WriteUint32LE(w *bytes.Buffer, v uint32) error

WriteUint32LE will write four bytes in little-endian byte order.

func WriteUint64LE added in v0.2.0 

func WriteUint64LE(w *bytes.Buffer, v uint64) error

WriteUint64LE will write eight bytes in little-endian byte order.

func WriteUint8 

func WriteUint8(w *bytes.Buffer, v uint8) error

WriteUint8 will write a single byte.

func WriteVarBytes 

func WriteVarBytes(w *bytes.Buffer, b []byte) error

WriteVarBytes will serialize a CompactSize int denoting the length, then proceeds to write b into w.

func WriteVarInt 

func WriteVarInt(w *bytes.Buffer, v uint64) error

WriteVarInt writes a CompactSize integer with a number of bytes depending on it's value

Types

This section is empty.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.