restruct

README

restruct

restruct is a library for reading and writing binary data in Go. Similar to lunixbochs struc and encoding/binary, this library reads data based on the layout of structures and, like struc, based on what is contained in struct tags.

To install Restruct, use the following command:

go get github.com/go-restruct/restruct

restruct aims to provide a clean, flexible, robust implementation of struct packing. In the future, through fast-path optimizations and code generation, it also aims to be quick, but it is currently very slow.

restruct currently requires Go 1.7+.

Status

• As of writing, coverage is hovering around 95%, but more thorough testing is always useful and desirable.
• Unpacking and packing are fully functional.
• More optimizations are probably possible.

Example

package main

import (
	"encoding/binary"
	"io/ioutil"
	"os"

	"github.com/go-restruct/restruct"
)

type Record struct {
	Message string `struct:"[128]byte"`
}

type Container struct {
	Version   int `struct:"int32"`
	NumRecord int `struct:"int32,sizeof=Records"`
	Records   []Record
}

func main() {
	var c Container

	file, _ := os.Open("records")
	defer file.Close()
	data, _ := ioutil.ReadAll(file)

	restruct.Unpack(data, binary.LittleEndian, &c)
}

Documentation

Overview

Package restruct implements packing and unpacking of raw binary formats.

Structures can be created with struct tags annotating the on-disk or in-memory layout of the structure, using the "struct" struct tag, like so:

struct {
	Length int `struct:"int32,sizeof=Packets"`
	Packets []struct{
		Source    string    `struct:"[16]byte"`
		Timestamp int       `struct:"int32,big"`
		Data      [256]byte `struct:"skip=8"`
	}
}

To unpack data in memory to this structure, simply use Unpack with a byte slice:

msg := Message{}
restruct.Unpack(data, binary.LittleEndian, &msg)

Index

• Variables
• func BitSize(v interface{}) (size int, err error)
• func EnableExprBeta()
• func Pack(order binary.ByteOrder, v interface{}) (data []byte, err error)
• func RegisterArrayType(array interface{})
• func SizeOf(v interface{}) (size int, err error)
• func Unpack(data []byte, order binary.ByteOrder, v interface{}) (err error)
• type BitSizer
• type FieldFlags
• type Packer
• type Sizer
• type Unpacker

Variables

var ErrInvalidBits = errors.New("bits specified on non-bitwise type")

ErrInvalidBits is returned when bits is used on an invalid type.

var ErrInvalidSize = errors.New("size specified on fixed size type")

ErrInvalidSize is returned when sizefrom is used on an invalid type.

var ErrInvalidSizeFrom = errors.New("sizefrom specified on fixed size type")

ErrInvalidSizeFrom is returned when sizefrom is used on an invalid type.

var ErrInvalidSizeOf = errors.New("sizeof specified on fixed size type")

ErrInvalidSizeOf is returned when sizefrom is used on an invalid type.

Functions

func BitSize

func BitSize(v interface{}) (size int, err error)

BitSize returns the binary encoded size of the given value, in bits.

func EnableExprBeta

func EnableExprBeta()

EnableExprBeta enables you to use restruct expr while it is still in beta. Use at your own risk. Functionality may change in unforeseen, incompatible ways at any time.

func Pack

func Pack(order binary.ByteOrder, v interface{}) (data []byte, err error)

Pack writes data from a datastructure into a byteslice.

Two types of values are directly supported here: Packers and structs. You can pass them by value or by pointer.

Each structure is serialized in the same way it would be deserialized with Unpack. See Unpack documentation for the struct tag format.

func RegisterArrayType

func RegisterArrayType(array interface{})

RegisterArrayType is deprecated; it is now a noop.

func SizeOf

func SizeOf(v interface{}) (size int, err error)

SizeOf returns the binary encoded size of the given value, in bytes.

func Unpack

func Unpack(data []byte, order binary.ByteOrder, v interface{}) (err error)

Unpack reads data from a byteslice into a value.

Two types of values are directly supported here: Unpackers and structs. You can pass them by value or by pointer, although it is an error if Restruct is unable to set a value because it is unaddressable.

For structs, each field will be read sequentially based on a straightforward interpretation of the type. For example, an int32 will be read as a 32-bit signed integer, taking 4 bytes of memory. Structures and arrays are laid out flat with no padding or metadata.

Unexported fields are ignored, except for fields named _ - those fields will be treated purely as padding. Padding will not be preserved through packing and unpacking.

The behavior of deserialization can be customized using struct tags. The following struct tag syntax is supported:

`struct:"[flags...]"`

Flags are comma-separated keys. The following are available:

type              A bare type name, e.g. int32 or []string. For integer
                  types, it is possible to specify the number of bits,
                  allowing the definition of bitfields, by appending a
                  colon followed by the number of bits. For example,
                  uint32:20 would specify a field that is 20 bits long.

sizeof=[Field]    Specifies that the field should be treated as a count of
                  the number of elements in Field.

sizefrom=[Field]  Specifies that the field should determine the number of
                  elements in itself by reading the counter in Field.

skip=[Count]      Skips Count bytes before the field. You can use this to
                  e.g. emulate C structure alignment.

big,msb           Specifies big endian byte order. When applied to
                  structs, this will apply to all fields under the struct.

little,lsb        Specifies little endian byte order. When applied to
                  structs, this will apply to all fields under the struct.

variantbool       Specifies that the boolean `true` value should be
                  encoded as -1 instead of 1.

invertedbool      Specifies that the `true` and `false` encodings for
                  boolean should be swapped.

Types

type BitSizer

type BitSizer interface {
	BitSize() int
}

BitSizer is an interface for types that need to specify their own size in bit-level granularity. It has the same effect as Sizer.

type FieldFlags

type FieldFlags uint64

FieldFlags is a type for flags that can be applied to fields individually.

const (
	// VariantBoolFlag causes the true value of a boolean to be ~0 instead of
	// just 1 (all bits are set.) This emulates the behavior of VARIANT_BOOL.
	VariantBoolFlag FieldFlags = 1 << iota

	// InvertedBoolFlag causes the true and false states of a boolean to be
	// flipped in binary.
	InvertedBoolFlag

	// RootFlag is set when the field points to the root struct.
	RootFlag

	// ParentFlag is set when the field points to the parent struct.
	ParentFlag

	// DefaultFlag is set when the field is designated as a switch case default.
	DefaultFlag
)

type Packer

type Packer interface {
	Sizer
	Pack(buf []byte, order binary.ByteOrder) ([]byte, error)
}

Packer is a type capable of packing a native value into a binary representation. The Pack function is expected to overwrite a number of bytes in buf then return a slice of the remaining buffer. Note that you must also implement SizeOf, and returning an incorrect SizeOf will cause the encoder to crash. The SizeOf should be equal to the number of bytes consumed from the buffer slice in Pack. You may use a pointer receiver even if the type is used by value.

type Sizer

type Sizer interface {
	SizeOf() int
}

Sizer is a type which has a defined size in binary. The SizeOf function returns how many bytes the type will consume in memory. This is used during encoding for allocation and therefore must equal the exact number of bytes the encoded form needs. You may use a pointer receiver even if the type is used by value.

type Unpacker

type Unpacker interface {
	Unpack(buf []byte, order binary.ByteOrder) ([]byte, error)
}

Unpacker is a type capable of unpacking a binary representation of itself into a native representation. The Unpack function is expected to consume a number of bytes from the buffer, then return a slice of the remaining bytes in the buffer. You may use a pointer receiver even if the type is used by value.