restruct

README

restruct

Added UnPackPartial to get remaining buffer after unpacking.

restruct is a library for reading and writing binary data in Go. Similar to lunixbochs struc and encoding/binary, this library reads data based on the layout of structures and, like struc, based on what is contained in struct tags.

To install Restruct, use the following command:

go get github.com/go-restruct/restruct

restruct aims to provide a clean, flexible, robust implementation of struct packing. In the future, through fast-path optimizations and code generation, it also aims to be quick, but it is currently very slow.

restruct currently requires Go 1.7+.

Status

• As of writing, coverage is hovering around 95%, but more thorough testing is always useful and desirable.
• Unpacking and packing are fully functional.
• More optimizations are probably possible.

Example

package main

import (
	"encoding/binary"
	"io/ioutil"
	"os"

	"github.com/go-restruct/restruct"
)

type Record struct {
	Message string `struct:"[128]byte"`
}

type Container struct {
	Version   int `struct:"int32"`
	NumRecord int `struct:"int32,sizeof=Records"`
	Records   []Record
}

func main() {
	var c Container

	file, _ := os.Open("records")
	defer file.Close()
	data, _ := ioutil.ReadAll(file)

	restruct.Unpack(data, binary.LittleEndian, &c)
}

Documentation

Overview

Package restruct implements packing and unpacking of raw binary formats.

Structures can be created with struct tags annotating the on-disk or in-memory layout of the structure, using the "struct" struct tag, like so:

struct {
	Length int `struct:"int32,sizeof=Packets"`
	Packets []struct{
		Source    string    `struct:"[16]byte"`
		Timestamp int       `struct:"int32,big"`
		Data      [256]byte `struct:"skip=8"`
	}
}

To unpack data in memory to this structure, simply use Unpack with a byte slice:

msg := Message{}
restruct.Unpack(data, binary.LittleEndian, &msg)

Index

• Variables
• func BitSize(v interface{}) (size int, err error)
• func EnableExprBeta()
• func Pack(order binary.ByteOrder, v interface{}) (data []byte, err error)
• func RegisterArrayType(array interface{})
• func SizeOf(v interface{}) (size int, err error)
• func Unpack(data []byte, order binary.ByteOrder, v interface{}) (err error)
• func UnpackPartial(data []byte, order binary.ByteOrder, v interface{}) (remaining []byte, err error)
• type BitSizer
• type FieldFlags
• type Packer
• type Sizer
• type Unpacker

Variables

var ErrInvalidBits = errors.New("bits specified on non-bitwise type")

ErrInvalidBits is returned when bits is used on an invalid type.

var ErrInvalidSize = errors.New("size specified on fixed size type")

ErrInvalidSize is returned when sizefrom is used on an invalid type.

var ErrInvalidSizeFrom = errors.New("sizefrom specified on fixed size type")

ErrInvalidSizeFrom is returned when sizefrom is used on an invalid type.

var ErrInvalidSizeOf = errors.New("sizeof specified on fixed size type")

ErrInvalidSizeOf is returned when sizefrom is used on an invalid type.

Functions

func BitSize

func BitSize(v interface{}) (size int, err error)

BitSize returns the binary encoded size of the given value, in bits.

func EnableExprBeta

func EnableExprBeta()

EnableExprBeta enables you to use restruct expr while it is still in beta. Use at your own risk. Functionality may change in unforeseen, incompatible ways at any time.

func Pack

func Pack(order binary.ByteOrder, v interface{}) (data []byte, err error)

Pack writes data from a datastructure into a byteslice.

Two types of values are directly supported here: Packers and structs. You can pass them by value or by pointer.

Each structure is serialized in the same way it would be deserialized with Unpack. See Unpack documentation for the struct tag format.

func RegisterArrayType

func RegisterArrayType(array interface{})

RegisterArrayType is deprecated; it is now a noop.

func SizeOf

func SizeOf(v interface{}) (size int, err error)

SizeOf returns the binary encoded size of the given value, in bytes.

func Unpack

func Unpack(data []byte, order binary.ByteOrder, v interface{}) (err error)

func UnpackPartial

func UnpackPartial(data []byte, order binary.ByteOrder, v interface{}) (remaining []byte, err error)

Types

type BitSizer

type BitSizer interface {
	BitSize() int
}

BitSizer is an interface for types that need to specify their own size in bit-level granularity. It has the same effect as Sizer.

type FieldFlags

type FieldFlags uint64

FieldFlags is a type for flags that can be applied to fields individually.

const (
	// VariantBoolFlag causes the true value of a boolean to be ~0 instead of
	// just 1 (all bits are set.) This emulates the behavior of VARIANT_BOOL.
	VariantBoolFlag FieldFlags = 1 << iota

	// InvertedBoolFlag causes the true and false states of a boolean to be
	// flipped in binary.
	InvertedBoolFlag

	// RootFlag is set when the field points to the root struct.
	RootFlag

	// ParentFlag is set when the field points to the parent struct.
	ParentFlag

	// DefaultFlag is set when the field is designated as a switch case default.
	DefaultFlag
)

type Packer

type Packer interface {
	Sizer
	Pack(buf []byte, order binary.ByteOrder) ([]byte, error)
}

Packer is a type capable of packing a native value into a binary representation. The Pack function is expected to overwrite a number of bytes in buf then return a slice of the remaining buffer. Note that you must also implement SizeOf, and returning an incorrect SizeOf will cause the encoder to crash. The SizeOf should be equal to the number of bytes consumed from the buffer slice in Pack. You may use a pointer receiver even if the type is used by value.

type Sizer

type Sizer interface {
	SizeOf() int
}

Sizer is a type which has a defined size in binary. The SizeOf function returns how many bytes the type will consume in memory. This is used during encoding for allocation and therefore must equal the exact number of bytes the encoded form needs. You may use a pointer receiver even if the type is used by value.

type Unpacker

type Unpacker interface {
	Unpack(buf []byte, order binary.ByteOrder) ([]byte, error)
}

Unpacker is a type capable of unpacking a binary representation of itself into a native representation. The Unpack function is expected to consume a number of bytes from the buffer, then return a slice of the remaining bytes in the buffer. You may use a pointer receiver even if the type is used by value.