gblob

package module
v0.2.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 29, 2023 License: MIT Imports: 4 Imported by: 6

README

Go Blob Build Status Go Report Card GoDoc

A package that provides utilities for writing and reading primitive types to and from byte sequences.

User's Guide

For a more complete documentation, check the GoDocs.

Block

The Block API allows one to place values of concrete primitive types at specific offsets inside a byte slice.

Example:

block := make(gblob.LittleEndianBlock, 32)
block.SetFloat32(4, 3.5)

There are two implementations available - LittleEndianBlock and BigEndianBlock.

Following is a benchmark comparison between Block and Go's binary.ByteOrder API.

Approach Time per Operation Allocated Memory per Operation Allocation Count per Operation
Block 0.2327 ns/op 0 B/op 0 allocs/op
binary.ByteOrder 0.2281 ns/op 0 B/op 0 allocs/op

Both APIs provide similar performance.

As with any benchmark result, take it with a grain of salt and do your own measurements according to your own requirements.

TypedWriter

The TypedWriter API allows one to write concrete primitive types to an io.Writer.

Example:

var buffer bytes.Buffer
writer := gblob.NewLittleEndianWriter(&buffer)
writer.WriteUint64(0x13743521FA954321)

There are two implementations available - NewLittleEndianWriter and NewBigEndianWriter.

Following is a benchmark comparison between TypedWriter and Go's binary.Write functions.

Approach Time per Operation Allocated Memory per Operation Allocation Count per Operation
TypedWriter 56.97 ns/op 0 B/op 0 allocs/op
binary.Write 175.5 ns/op 32 B/op 6 allocs/op

As can be seen, not only does the TypedWriter not allocate any memory, but it also runs about 3 times faster.

As with any benchmark result, take it with a grain of salt and do your own measurements according to your own requirements.

NOTE: In reality the TypedWriter does allocate around 8 bytes but only when first created (uses it for a write buffer). It does not allocate during write operations.

TypedReader

The TypedReader API allows one to read concrete primitive types from an io.Reader.

Example:

buffer := bytes.NewBuffer([]uint8{
  0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x20,
})
reader := gblob.NewLittleEndianReader(buffer)
value, err := reader.ReadUint64()

There are two implementations available - NewLittleEndianReader and NewBigEndianReader.

Following is a benchmark comparison between TypedReader and Go's binary.Read functions.

Approach Time per Operation Allocated Memory per Operation Allocation Count per Operation
TypedReader 61.69 ns/op 0 B/op 0 allocs/op
binary.Read 226.3 ns/op 56 B/op 12 allocs/op

As can be seen, not only does the TypedReader not allocate any memory, but it also runs nearly 4 times faster.

As with any benchmark result, take it with a grain of salt and do your own measurements according to your own requirements.

NOTE: In reality the TypedReader does allocate around 8 bytes but only when first created (uses it for a read buffer). It does not allocate during read operations.

PackedEncoder

The PackedEncoder API allows one to marshal a data structure to a binary sequence.

Example:

var buffer bytes.Buffer
gblob.NewLittleEndianPackedEncoder(&buffer).Encode(struct{
  A float32
  B uint64
}{
  A: 3.14,
  B: 0xFF003344FF003344,
})

There are two implementations available - NewLittleEndianPackedEncoder and NewBigEndianPackedEncoder.

It is difficult to compare the PackedEncoder API with binary.Write or gob.NewEncoder, since the former has less flexibility to the types it can read and the latter is much more flexible - it allows for forward compatible serialization.

Nevertheless, following is a benchmark comparison between PackedEncoder and Go's gob.NewEncoder functions.

Approach Time per Operation Allocated Memory per Operation Allocation Count per Operation
PackedEncoder 242.6 ns/op 72 B/op 2 allocs/op
gob.NewEncoder 408.7 ns/op 72 B/op 2 allocs/op

The PackedEncoder is almost twice faster. Memory allocation is equal.

And following is a benchmark comparison between PackedEncoder and Go's binary.Write functions.

Approach Time per Operation Allocated Memory per Operation Allocation Count per Operation
PackedEncoder 133.7 ns/op 32 B/op 1 allocs/op
binary.Write 442.9 ns/op 112 B/op 8 allocs/op

The PackedEncoder is significantly quicker and it allocates less memory.

As with any benchmark result, take it with a grain of salt and do your own measurements according to your own requirements.

PackedDecoder

The PackedDecoder API allows one to unmarshal a data structure from a binary sequence that was previously marshaled through the PackedEncoder API.

Example:

buffer := bytes.NewBuffer([]uint8{
  0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x20,
})
var target uint64
gblob.NewLittleEndianPackedDecoder(buffer).Decode(&target)

There are two implementations available - NewLittleEndianPackedDecoder and NewBigEndianPackedDecoder.

It is difficult to compare the PackedDecoder API with binary.Read or gob.NewDecoder, since the former has less flexibility to the types it can read and the latter is much more flexible - it allows for backward compatible deserialization.

Nevertheless, following is a benchmark comparison between PackedDecoder and Go's gob.NewDecoder functions.

Approach Time per Operation Allocated Memory per Operation Allocation Count per Operation
PackedDecoder 434.6 ns/op 152 B/op 5 allocs/op
gob.NewDecoder 19480 ns/op 7928 B/op 212 allocs/op

As can be seen, the PackedDecoder is much faster and barely allocates memory. On the downside, a change to the target type would break decoding, which is not the case with gob.NewDecoder.

And following is a benchmark comparison between PackedDecoder and Go's binary.Read functions.

Approach Time per Operation Allocated Memory per Operation Allocation Count per Operation
PackedDecoder 187.7 ns/op 32 B/op 1 allocs/op
binary.Read 173.5 ns/op 64 B/op 2 allocs/op

The PackedDecoder is marginably slower, though it allocates slightly less memory and has the added benefit of supporing types like slice and map.

As with any benchmark result, take it with a grain of salt and do your own measurements according to your own requirements.

Documentation

Overview

Package gblob (Go Blob) provides utilities for writing and reading binary primitives. While Go does have encoding/binary and encoding/gob packages, they have certain limitations that this package attempts to overcome at the cost of other limitations.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type BigEndianBlock 

type BigEndianBlock []byte

BigEndianBlock represents a fixed-size block of bytes that holds values encoded in Big Endian order.

func (BigEndianBlock) Float32 

func (b BigEndianBlock) Float32(offset int) float32

Float32 returns the float32 value at the specified offset.

func (BigEndianBlock) Float64 

func (b BigEndianBlock) Float64(offset int) float64

Float64 returns the float64 value at the specified offset.

func (BigEndianBlock) Int16 

func (b BigEndianBlock) Int16(offset int) int16

Int16 returns the int16 value at the specified offset.

func (BigEndianBlock) Int32 

func (b BigEndianBlock) Int32(offset int) int32

Int32 returns the int32 value at the specified offset.

func (BigEndianBlock) Int64 

func (b BigEndianBlock) Int64(offset int) int64

Int64 returns the int64 value at the specified offset.

func (BigEndianBlock) Int8 

func (b BigEndianBlock) Int8(offset int) int8

Int8 returns the int8 value at the specified offset.

func (BigEndianBlock) SetFloat32 

func (b BigEndianBlock) SetFloat32(offset int, value float32)

SetFloat32 places the float32 value at the specified offset.

func (BigEndianBlock) SetFloat64 

func (b BigEndianBlock) SetFloat64(offset int, value float64)

SetFloat64 places the float64 value at the specified offset.

func (BigEndianBlock) SetInt16 

func (b BigEndianBlock) SetInt16(offset int, value int16)

SetInt16 places the int16 value at the specified offset.

func (BigEndianBlock) SetInt32 

func (b BigEndianBlock) SetInt32(offset int, value int32)

SetInt32 places the int32 value at the specified offset.

func (BigEndianBlock) SetInt64 

func (b BigEndianBlock) SetInt64(offset int, value int64)

SetInt64 places the int64 value at the specified offset.

func (BigEndianBlock) SetInt8 

func (b BigEndianBlock) SetInt8(offset int, value int8)

SetInt8 places the int8 value at the specified offset.

func (BigEndianBlock) SetUint16 

func (b BigEndianBlock) SetUint16(offset int, value uint16)

SetUint16 places the uint16 value at the specified offset.

func (BigEndianBlock) SetUint32 

func (b BigEndianBlock) SetUint32(offset int, value uint32)

SetUint32 places the uint32 value at the specified offset.

func (BigEndianBlock) SetUint64 

func (b BigEndianBlock) SetUint64(offset int, value uint64)

SetUint64 places the uint64 value at the specified offset.

func (BigEndianBlock) SetUint8 

func (b BigEndianBlock) SetUint8(offset int, value uint8)

SetUint8 places the uint8 value at the specified offset.

func (BigEndianBlock) Uint16 

func (b BigEndianBlock) Uint16(offset int) uint16

Uint16 returns the uint16 value at the specified offset.

func (BigEndianBlock) Uint32 

func (b BigEndianBlock) Uint32(offset int) uint32

Uint32 returns the uint32 value at the specified offset.

func (BigEndianBlock) Uint64 

func (b BigEndianBlock) Uint64(offset int) uint64

Uint64 returns the uint64 value at the specified offset.

func (BigEndianBlock) Uint8 

func (b BigEndianBlock) Uint8(offset int) uint8

Uint8 returns the uint8 value at the specified offset.

type Block 

type Block interface {

	// Uint8 returns the uint8 value at the specified offset.
	Uint8(offset int) uint8

	// SetUint8 places the uint8 value at the specified offset.
	SetUint8(offset int, value uint8)

	// Int8 returns the int8 value at the specified offset.
	Int8(offset int) int8

	// SetInt8 places the int8 value at the specified offset.
	SetInt8(offset int, value int8)

	// Uint16 returns the uint16 value at the specified offset.
	Uint16(offset int) uint16

	// SetUint16 places the uint16 value at the specified offset.
	SetUint16(offset int, value uint16)

	// Int16 returns the int16 value at the specified offset.
	Int16(offset int) int16

	// SetInt16 places the int16 value at the specified offset.
	SetInt16(offset int, value int16)

	// Uint32 returns the uint32 value at the specified offset.
	Uint32(offset int) uint32

	// SetUint32 places the uint32 value at the specified offset.
	SetUint32(offset int, value uint32)

	// Int32 returns the int32 value at the specified offset.
	Int32(offset int) int32

	// SetInt32 places the int32 value at the specified offset.
	SetInt32(offset int, value int32)

	// Uint64 returns the uint64 value at the specified offset.
	Uint64(offset int) uint64

	// SetUint64 places the uint64 value at the specified offset.
	SetUint64(offset int, value uint64)

	// Int64 returns the int64 value at the specified offset.
	Int64(offset int) int64

	// SetInt64 places the int64 value at the specified offset.
	SetInt64(offset int, value int64)

	// Float32 returns the float32 value at the specified offset.
	Float32(offset int) float32

	// SetFloat32 places the float32 value at the specified offset.
	SetFloat32(offset int, value float32)

	// Float64 returns the float64 value at the specified offset.
	Float64(offset int) float64

	// SetFloat64 places the float64 value at the specified offset.
	SetFloat64(offset int, value float64)
}

Block represents a fixed-size block of bytes that holds values encoded in a particular order.

type LittleEndianBlock 

type LittleEndianBlock []byte

LittleEndianBlock represents a fixed-size block of bytes that holds values encoded in Little Endian order.

func (LittleEndianBlock) Float32 

func (b LittleEndianBlock) Float32(offset int) float32

Float32 returns the float32 value at the specified offset.

func (LittleEndianBlock) Float64 

func (b LittleEndianBlock) Float64(offset int) float64

Float64 returns the float64 value at the specified offset.

func (LittleEndianBlock) Int16 

func (b LittleEndianBlock) Int16(offset int) int16

Int16 returns the int16 value at the specified offset.

func (LittleEndianBlock) Int32 

func (b LittleEndianBlock) Int32(offset int) int32

Int32 returns the int32 value at the specified offset.

func (LittleEndianBlock) Int64 

func (b LittleEndianBlock) Int64(offset int) int64

Int64 returns the int64 value at the specified offset.

func (LittleEndianBlock) Int8 

func (b LittleEndianBlock) Int8(offset int) int8

Int8 returns the int8 value at the specified offset.

func (LittleEndianBlock) SetFloat32 

func (b LittleEndianBlock) SetFloat32(offset int, value float32)

SetFloat32 places the float32 value at the specified offset.

func (LittleEndianBlock) SetFloat64 

func (b LittleEndianBlock) SetFloat64(offset int, value float64)

SetFloat64 places the float64 value at the specified offset.

func (LittleEndianBlock) SetInt16 

func (b LittleEndianBlock) SetInt16(offset int, value int16)

SetInt16 places the int16 value at the specified offset.

func (LittleEndianBlock) SetInt32 

func (b LittleEndianBlock) SetInt32(offset int, value int32)

SetInt32 places the int32 value at the specified offset.

func (LittleEndianBlock) SetInt64 

func (b LittleEndianBlock) SetInt64(offset int, value int64)

SetInt64 places the int64 value at the specified offset.

func (LittleEndianBlock) SetInt8 

func (b LittleEndianBlock) SetInt8(offset int, value int8)

SetInt8 places the int8 value at the specified offset.

func (LittleEndianBlock) SetUint16 

func (b LittleEndianBlock) SetUint16(offset int, value uint16)

SetUint16 places the uint16 value at the specified offset.

func (LittleEndianBlock) SetUint32 

func (b LittleEndianBlock) SetUint32(offset int, value uint32)

SetUint32 places the uint32 value at the specified offset.

func (LittleEndianBlock) SetUint64 

func (b LittleEndianBlock) SetUint64(offset int, value uint64)

SetUint64 places the uint64 value at the specified offset.

func (LittleEndianBlock) SetUint8 

func (b LittleEndianBlock) SetUint8(offset int, value uint8)

SetUint8 places the uint8 value at the specified offset.

func (LittleEndianBlock) Uint16 

func (b LittleEndianBlock) Uint16(offset int) uint16

Uint16 returns the uint16 value at the specified offset.

func (LittleEndianBlock) Uint32 

func (b LittleEndianBlock) Uint32(offset int) uint32

Uint32 returns the uint32 value at the specified offset.

func (LittleEndianBlock) Uint64 

func (b LittleEndianBlock) Uint64(offset int) uint64

Uint64 returns the uint64 value at the specified offset.

func (LittleEndianBlock) Uint8 

func (b LittleEndianBlock) Uint8(offset int) uint8

Uint8 returns the uint8 value at the specified offset.

type PackedDecoder added in v0.2.0 

type PackedDecoder struct {
	// contains filtered or unexported fields
}

PackedDecoder decodes arbitrary Go objects from binary form by going through each field in sequence and deserializing it without any padding.

func NewBigEndianPackedDecoder added in v0.2.0 

func NewBigEndianPackedDecoder(in io.Reader) *PackedDecoder

NewBigEndianPackedDecoder creates a new PackedDecoder that is configured to read its input in Big Endian order.

func NewLittleEndianPackedDecoder added in v0.2.0 

func NewLittleEndianPackedDecoder(in io.Reader) *PackedDecoder

NewLittleEndianPackedDecoder creates a new PackedDecoder that is configured to read its input in Little Endian order.

func (*PackedDecoder) Decode added in v0.2.0 

func (d *PackedDecoder) Decode(target interface{}) error

Decode decodes the specified target value from the Reader.

type PackedEncoder added in v0.2.0 

type PackedEncoder struct {
	// contains filtered or unexported fields
}

PackedEncoder encodes arbitrary Go objects in binary form by going through each field in sequence and serializing it without any padding.

func NewBigEndianPackedEncoder added in v0.2.0 

func NewBigEndianPackedEncoder(out io.Writer) *PackedEncoder

NewBigEndianPackedEncoder creates a new PackedEncoder that is configured to write its output in Big Endian order.

func NewLittleEndianPackedEncoder added in v0.2.0 

func NewLittleEndianPackedEncoder(out io.Writer) *PackedEncoder

NewLittleEndianPackedEncoder creates a new PackedEncoder that is configured to write its output in Little Endian order.

func (*PackedEncoder) Encode added in v0.2.0 

func (e *PackedEncoder) Encode(source any) error

Encode encodes the specified source value into the Writer.

type TypedReader 

type TypedReader interface {

	// ReadUint8 reads a single uint8 from the source.
	ReadUint8() (uint8, error)

	// ReadInt8 reads a single int8 from the source.
	ReadInt8() (int8, error)

	// ReadUint16 reads a single uint16 from the source.
	ReadUint16() (uint16, error)

	// ReadInt16 reads a single int16 from the source.
	ReadInt16() (int16, error)

	// ReadUint32 reads a single uint32 from the source.
	ReadUint32() (uint32, error)

	// ReadInt32 reads a single int32 from the source.
	ReadInt32() (int32, error)

	// ReadUint64 reads a single uint64 from the source.
	ReadUint64() (uint64, error)

	// ReadInt64 reads a single int64 from the source.
	ReadInt64() (int64, error)

	// ReadFloat32 reads a single float32 from the source.
	ReadFloat32() (float32, error)

	// ReadFloat64 reads a single float64 from the source.
	ReadFloat64() (float64, error)

	// ReadBytes reads exactly len(target) bytes from the source and places
	// them inside target.
	ReadBytes(target []byte) error
}

TypedReader represents a reader that can parse specific Go types from a byte sequence.

The endianness depends on the actual implementation.

func NewBigEndianReader 

func NewBigEndianReader(in io.Reader) TypedReader

NewBigEndianReader returns an implementation of TypedReader that reads from the specified in Reader in Big Endian order.

func NewLittleEndianReader 

func NewLittleEndianReader(in io.Reader) TypedReader

NewLittleEndianReader returns an implementation of TypedReader that reads from the specified in Reader in Little Endian order.

type TypedWriter 

type TypedWriter interface {

	// WriteUint8 writes a single uint8 to the target.
	WriteUint8(uint8) error

	// WriteInt8 writes a single int8 to the target.
	WriteInt8(int8) error

	// WriteUint16 writes a single uint16 to the target.
	WriteUint16(uint16) error

	// WriteInt16 writes a single int16 to the target.
	WriteInt16(int16) error

	// WriteUint32 writes a single uint32 to the target.
	WriteUint32(uint32) error

	// WriteInt32 writes a single int32 to the target.
	WriteInt32(int32) error

	// WriteUint64 writes a single uint64 to the target.
	WriteUint64(uint64) error

	// WriteInt64 writes a single int64 to the target.
	WriteInt64(int64) error

	// WriteFloat32 writes a single float32 to the target.
	WriteFloat32(float32) error

	// WriteFloat64 writes a single float64 to the target.
	WriteFloat64(float64) error

	// WriteBytes writes len(bytes) from source to the target.
	WriteBytes(source []byte) error
}

TypedWriter represents a writer that can serialize specific Go types to a byte sequence.

The endianness depends on the actual implementation.

func NewBigEndianWriter 

func NewBigEndianWriter(out io.Writer) TypedWriter

NewBigEndianWriter returns an implementation of TypedWriter that writes to the specified out Writer in Big Endian order.

func NewLittleEndianWriter 

func NewLittleEndianWriter(out io.Writer) TypedWriter

NewLittleEndianWriter returns an implementation of TypedWriter that writes to the specified out Writer in Little Endian order.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.