vamp

package module
v0.8.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 18, 2024 License: AGPL-3.0 Imports: 19 Imported by: 3

README

Vampire

Binary Data Serialization

Go Reference

Although it is not zero-copy, Vampire provides "direct access" to values, which means you can navigate down any path in a nested data structure without having to analyze too many other parts that come before the value you are looking for.

This ability to reach each data field quickly makes the format accessible – “perviam” in Latin. Just because “Vampire” sounds much cooler the format is called Vampire in the end. But vamp is the package name to Go for because its shorter and still associated with vampires.

What is the point of this property for a serialization format?

Message brokers, queuing systems, and other types of middleware often have to make decisions about the data they manage based on tiny pieces of that data. This is often solved by artificially and redundantly adding the desired information as out-of-band data to the normal payload data. Alternatively, one could decode the data first and then find the relevant parts within the payload. If you choose the second approach, it is more efficient to decode only the relevant parts of the payload.

That was the driving idea for the design of Vampire.

Data Types

To make Vampire work, it requires strict typing of formats. Types must be known before writing or reading data. This reduces the amount of structural information in the encoded data and makes Vampire data quite compact. Vampire types have a Vampire encoding that allows us to create self-contained and self-descriptive data. For convenience, types also have a concise text encoding, which is described here.

TYPE ::= BASIC_TYPE | BYTES_TYPE | ARRAY_TYPE | RECORD_TYPE | LIST_TYPE |
         UNION_TYPE | ANY_TYPE

SIZE ::= 'Size8' | 'Size16' | 'Size32'

  • Basic Types: bool; signed and unsigned integers with 8, 16, 32 and 64 bit; 32-bit and 64-bit float

    BASIC_TYPE ::= 'bool' | 'int8' | 'int16' | 'int32' | 'int64' | 'uint8' |
               'uint16' | 'uint32' | 'unit64' | 'float32' | 'float64'

  • Bytes Type: Byte strings with either 8-, 16- or 32-bit size limits (size in Byte)

    BYTES_TYPE ::= '"' SIZE '"'`

  • Array Types: Variable length array with 8-bit, 16-bit and 32-bit length limits. Each element has the same type. Length is part of the array value not the array type. The length of an array must be known before writing an array value.

    ARRAY_TYPE ::= '[' SIZE ' ' TYPE ']'

  • Record Types: Fixed number of fields where each field has it's own type. Field names are optional and are not part of record values.

    RECORD_TYPE ::= '{' [ FIELD { ' ' FIELD } ] '}'

    FIELD ::= [ FIELD_NAME ':' ] TYPE

  • List Types: Unbounded (theoretically) sequence of values of the same type. Final length does not need to be known before writing. A 8-, 16- or 32-bit length fields is provided. If the final length fits, it might be recorded in the list value.

    LIST_TYPE ::= '(' SIZE ' ' TYPE ')'

  • Union Types: A tagged union with a predefined set of alternative value types. Tags can have 8-, 16- or 32-bit size.

    UNION_TYPE ::= '<' SIZE ' ' TYPE { '|' TYPE } '>'

  • Any Type: Write any Vampire value along with its Vampire type.

    ANY_TYPE ::= 'any'

Performance

Full Record Write and Read

Using this Go data structure with Vampire type {Age:int8 Name:"Size8" Owners:[Size8 "Size8"] Male:bool}

type Animal struct {
	Age    int      `vamp:",size=8"`
	Name   string   `vamp:",size=8"`
	Owners []string `vamp:",size=8:8"`
	Male   bool
}

Animal{Age: 4, Name: "Candy", Owners: []string{"Mary", "Joe"}, Male:   true}
Format Encoded Bytes Write&Read-Time / Best
Vampire (8-bit offsets) 22 –
Vampire (16-bit offsets) 26 100%
Vampire (32-bit offsets) 34 –
JSON (Go std. lib) 60 233%
JSON (jsoniter) 60 110%
GOB 113 2,130%
CBOR 40 112%
Partial Read Performance

This is a test from the vamp-benchmarks project. It runs on 100,000 randomly generated orders with the structure:

type Order struct {
	CustName string
	CustNum  uint32
	Date     time.Time
	Lines    []OrderLine
}

type OrderLine struct {
	MatName string
	MatNum  uint32
	Count   int
	Price   float64
}

The task is to filter all orders that have at least one order line with the MatNum 0. The test targets Vampire's core property of being able to access individual attributes as directly as possible.

Format File Size real user sys total orders/s
JSON 124,023,274 byte 1.447 s 1.022 s 0.401 s 2.869 s 34,855
vamp 52,374,536 byte 0.220 s 0.056 s 0.163 s 0.438 s 228,311
vamp/JSON 42.23% 15.19% 5.43% 40.70% 15.28% 6.55

JSON timing was measured with Jsoniter. File sizes may vary with new test runs becaus of randomly generated data.

Documentation

Overview

Package vamp implements the Vampire binary data serialization format for Go.

While Vampire does not provide zero-copy characteristics it gives you "direct access" to values i.e., you can navigate any path down into nested structures without parsing too many other parts that come before the value you are looking for. This feature makes any value directly accessible – “perviam” in Latin. Just because “Vampire” sounds much cooler the format is called Vampire in the end. But vamp is the package name to Go for because its shorter and still associated with vampires.

Type Promotion

When reading and writing data, Vamp supports type promotion between certain types to facilitate the preservation of backwards compatibility.

  • Type promotion between integers is supported if a source type can be stored in a target type without loss. E.g. uint16 can be promoted to int32 but no signed integer type can be promoted to any unsigned integer type.

  • float32 will be promoted to `float64`

For the machine-dependent integers int and uint, it depends on the current value whether it can be read from or written to a Vampire integer. If the current value is within the range of the target, the read or write is successful. Otherwise, it results in a runtime error.

Date and Duration

By default as int64 <-> Unix time with millis.

Example 
type Animal struct {
	Age    int      `vamp:"age,size=8"`
	Name   string   `vamp:",size=8"`
	Owners []string `vamp:",size=8:8"`
	Male   bool
}
// Use reflection to get the Vampire type; can be created with API, too
animalType, _, _ := ReflectType(Animal{})
// We use a 16-bit offset for the buffers
const BufferOffset = Size16
// Create write buffer starting with a Vampire header
wbuf := NewWriteBuffer(
	animalType,
	BufferOffset,
	HeaderSize,                       // Start writing data behind the header
	HeaderFor(BufferOffset).Bytes(0), // Pass Vampire header as buffer
)
// Write a record of type t to the buffer. Value has to match type t.
animalType.Write(wbuf, Animal{
	Age:    4,
	Name:   "Lassie",
	Owners: []string{"Timmy", "Martin"},
	Male:   false,
})
// How many bytes did we write
fmt.Printf("With header: %d; Value data only: %d\n", len(wbuf.AllBytes()), len(wbuf.Bytes()))
// Now extract the complete buffer, incl. header
data := wbuf.AllBytes()
// Wrap data into a read buffer that expects the header at byte 0
// Reads buffer's offset size from header (We could also directly read from wbuf)
rbuf, _ := NewReadBufferHeader(animalType, data, false)
// We need the specific type – record – to select a field
rt := animalType.(*Record)
// Slice the read buffer to field "Owners" with index 2 (3rd field)
revert := rt.Field(rbuf, 2)
// Read the Owners into a string array
var owners []string
rt.FieldType(2).Read(rbuf, &owners)
// We want to unslice the buffer to return to the complete record
rbuf.Unslice(revert)
// Print what we got from the binary data
fmt.Println(strings.Join(owners, " & "))

Output:

With header: 35; Value data only: 31
Timmy & Martin

Index

Examples

Constants

View Source 
const (
	// Version of the implemented Vampire encoding schema
	Version = 0

	// Size of Vampire data header in byte
	HeaderSize = len(Header{})
)

Variables

View Source 
var (
	Bool    boolType
	Int8    int8Type
	Int16   int16Type
	Int32   int32Type
	Int64   int64Type
	Uint8   uint8Type
	Uint16  uint16Type
	Uint32  uint32Type
	Uint64  uint64Type
	Float32 float32Type
	Float64 float64Type
)
  • Vampire Basic Types

Vampire Basic Types

Each <N>Type variable implements Type and represents a Vampire type that corresponds directly to a Go type <n> where <n> is <N> folded to lower case e.g., Bool corresponds to Go's bool type.

Marshaler and Unmarshaler interfaces are not explicitly defined for basic types. To implement user defined (un-)marshaling to or from <N>Type a value has to implement methods that match the following convention:

The marshaler for writing <n> to <N>Type: 

interface{ VampTo<N>() (<n>, error) }

The unmarshaler for reading <n> from <N>Type: 

interface{ VampFrom<N>(<n>) error }
View Source 
var Any anyType
View Source 
var IOBytesMode = CheckUTF8

Functions

func Equal added in v0.8.0 

func Equal(s, t Type, opts CompareOpts) bool

func MustReflectType added in v0.7.0 

func MustReflectType(val any) (Type, Fields)

func ReflectType added in v0.7.0 

func ReflectType(val any) (Type, Fields, error)

func ResetTypeBuffer 

func ResetTypeBuffer(buf *Buffer, off SizeRange, prefix int)

func TypeName added in v0.4.0 

func TypeName(t Type) string

TypeName returns the name attached to a type, if any. Otherwise TypeName returns the empty string.

func UseFieldsForStruct added in v0.8.0 

func UseFieldsForStruct(s any, fs Fields) error

func ValidName added in v0.6.0 

func ValidName(s string) bool

ValidName checks whether s is a valid name.

func WriteType 

func WriteType(buf *Buffer, t Type) error

func WriteTypeFile added in v0.7.1 

func WriteTypeFile(name string, off SizeRange, t Type) error

Types

type Alt 

type Alt struct {
	S uint
	V any
}

func (Alt) VampUnionAlt 

func (a Alt) VampUnionAlt() (uint, any)

type AnyMarshaler 

type AnyMarshaler interface {
	VampAnyType() (Type, any)
}

type Array added in v0.5.0 

type Array struct {
	// contains filtered or unexported fields
}
Example 
a := []string{"boo", "bar", "baz"}
t, _, _ := ReflectType(a)
fmt.Println(t)
buf := NewWriteBuffer(t, Size16, 0, nil)
t.Write(buf, a)
var out []string
t.Read(&buf.ReadBuffer, &out)
fmt.Printf("%[1]T=%[1]v\n", out)

Output:

[Size32 "Size16"]
[]string=[boo bar baz]

func NewArray added in v0.5.0 

func NewArray(length SizeRange, elem Type) *Array

func NewOptional added in v0.5.0 

func NewOptional(elem Type) *Array

NewOptional returns an ArrayType with length of Size8 – the smallest sufficient size range. This is the recommended way to represent optional values.

Example 
et := NewBytes(Size8, ReadRaw)
ot := NewOptional(et)
buf := NewWriteBuffer(ot, Size8, 0, nil)
ot.WriteOptOne(buf, "Hello, optional!")
var out string
ok, _ := ot.ReadOpt(&buf.ReadBuffer, &out)
fmt.Println(ok, out)
buf.Reset(ot, Size16, 0)
ot.WriteOptNone(buf)
out = "-"
ok, _ = ot.ReadOpt(&buf.ReadBuffer, &out)
fmt.Println(ok, out)

Output:

true Hello, optional!
false -

func (*Array) Compare added in v0.8.0 

func (a *Array) Compare(t Type, opts CompareOpts) (bool, int)

func (*Array) DynNo added in v0.5.0 

func (t *Array) DynNo() uint

func (*Array) Elem added in v0.5.0 

func (t *Array) Elem() Type

func (*Array) FixSize added in v0.5.0 

func (t *Array) FixSize() uint

func (*Array) Hash added in v0.5.0 

func (a *Array) Hash(h hash.Hash, opts CompareOpts)

func (*Array) Index added in v0.5.0 

func (t *Array) Index(buf *ReadBuffer, i uint) (revert uint)

func (*Array) Len added in v0.5.0 

func (t *Array) Len(buf *ReadBuffer) uint

func (*Array) LenRange added in v0.5.0 

func (t *Array) LenRange() SizeRange

func (*Array) PrintTo added in v0.8.0 

func (t *Array) PrintTo(w io.Writer) (n int, err error)

func (*Array) Read added in v0.5.0 

func (t *Array) Read(buf *ReadBuffer, into any) error

func (*Array) ReadArray added in v0.5.0 

func (t *Array) ReadArray(buf *ReadBuffer, ls ArrayUnmarshaler) error

func (*Array) ReadMap added in v0.8.0 

func (t *Array) ReadMap(buf *ReadBuffer, m any) error

func (*Array) ReadOpt added in v0.5.0 

func (t *Array) ReadOpt(buf *ReadBuffer, into any) (ok bool, err error)

func (*Array) String added in v0.5.0 

func (t *Array) String() string

func (*Array) VampRecordRead added in v0.5.0 

func (t *Array) VampRecordRead(rt *Record, buf *ReadBuffer) error

func (*Array) VampRecordWrite added in v0.5.0 

func (t *Array) VampRecordWrite(rt *Record, buf *Buffer) error

func (*Array) VampUnionAlt added in v0.5.0 

func (t *Array) VampUnionAlt() (uint, any)

func (*Array) Write added in v0.5.0 

func (t *Array) Write(buf *Buffer, ls any) (err error)

func (*Array) WriteArray added in v0.5.0 

func (t *Array) WriteArray(buf *Buffer, ls ArrayMarshaler) error

func (*Array) WriteMap added in v0.8.0 

func (t *Array) WriteMap(buf *Buffer, m any) error

func (*Array) WriteOptNone added in v0.5.0 

func (t *Array) WriteOptNone(buf *Buffer) error

func (*Array) WriteOptOne added in v0.5.0 

func (t *Array) WriteOptOne(buf *Buffer, value any) error

type ArrayMarshaler 

type ArrayMarshaler interface {
	VampArrayLen() uint
	VampArrayWriter(i uint) (any, error)
}

type ArrayUnmarshaler 

type ArrayUnmarshaler interface {
	VampArrayResize(l uint) error
	VampArrayReader(i uint) (any, error)
	VampArrayFinish() error
}

type Buffer 

type Buffer struct {
	ReadBuffer
}

func NewTypeWriteBuffer 

func NewTypeWriteBuffer(off SizeRange, at int, buf []byte) *Buffer

func NewWriteBuffer 

func NewWriteBuffer(t Type, off SizeRange, at int, buf []byte) *Buffer

func (*Buffer) Reset 

func (b *Buffer) Reset(t Type, off SizeRange, prefix int)

type Bytes added in v0.5.0 

type Bytes struct {
	// contains filtered or unexported fields
}

mode is a vamp but not a Vampire thing (Hash; Equals)

func NewBytes added in v0.5.0 

func NewBytes(size SizeRange, mode BytesReadMode) Bytes

func NewString added in v0.5.0 

func NewString(size SizeRange) Bytes

func (Bytes) Compare added in v0.8.0 

func (s Bytes) Compare(t Type, _ CompareOpts) (bool, int)

func (Bytes) DynNo added in v0.5.0 

func (t Bytes) DynNo() uint

func (Bytes) FixSize added in v0.5.0 

func (t Bytes) FixSize() uint

func (Bytes) Hash added in v0.5.0 

func (s Bytes) Hash(h hash.Hash, _ CompareOpts)

func (Bytes) LenRange added in v0.5.0 

func (t Bytes) LenRange() SizeRange

func (Bytes) PrintTo added in v0.8.0 

func (t Bytes) PrintTo(w io.Writer) (int, error)

func (Bytes) Read added in v0.5.0 

func (t Bytes) Read(buf *ReadBuffer, into any) (err error)

User Defined Unmarshaling

Unmarshaling from byte array: 

interface{ VampFromBytes([]byte) error } // must copy its argument

Unmarshaling from string: 

interface{ VampFromString(string) error }

func (Bytes) ReadBytes added in v0.5.0 

func (t Bytes) ReadBytes(buf *ReadBuffer) []byte

func (Bytes) ReadString added in v0.5.0 

func (t Bytes) ReadString(buf *ReadBuffer) string

func (Bytes) String added in v0.5.0 

func (t Bytes) String() string

func (*Bytes) VampFromUint8 added in v0.5.0 

func (t *Bytes) VampFromUint8(v uint8) error

func (Bytes) VampToUint8 added in v0.5.0 

func (t Bytes) VampToUint8() (uint8, error)

func (Bytes) VampUnionAlt added in v0.5.0 

func (t Bytes) VampUnionAlt() (uint, any)

func (Bytes) Write added in v0.5.0 

func (t Bytes) Write(buf *Buffer, value any) error

User Defined Marshaling

Marshaling to byte array: 

interface{ VampToBytes() ([]byte, error) }

Marshaling to string: 

interface{ VampToString() (string, error) }

func (Bytes) WriteBytes added in v0.5.0 

func (t Bytes) WriteBytes(buf *Buffer, data []byte) error

func (Bytes) WriteString added in v0.5.0 

func (t Bytes) WriteString(buf *Buffer, s string) error

type BytesReadMode added in v0.7.0 

type BytesReadMode uint8
const (
	// Bytes will be read as string if the bytes are valid UTF-8
	CheckUTF8 BytesReadMode = iota

	// Bytes will be read as []byte
	ReadRaw

	// Bytes will be read as string. It is not checked if bytes are valid UTF-8.
	ReadString
)
  • Bytes Reading Mode

Bytes Reading Mode

The bytes mode controls how a Vampire array is unmarshaled when the target type is not determined e.g., read into an *any.

type CompareOpts added in v0.8.0 

type CompareOpts int
const (
	// Compare takes Record field names into account
	FieldNames CompareOpts = (1 << iota)

	// Compare takes Union alt names into account
	AltNames
)

type Fields added in v0.8.0 

type Fields struct {
	// contains filtered or unexported fields
}

Fields denotes a set of fields from a Go struct. Fields are computed when a vamp record type is computed by reflection from a Go struct, e.g. with Reflect.Type.

func (Fields) Empty added in v0.8.0 

func (fs Fields) Empty() bool

func (Fields) Get added in v0.8.0 

func (fs Fields) Get(from any, reuse []any) []any

Get copies the field values of from to an any slice and returs it. If reuse is not nil and has sufficient capacty it will be used to store the values. The result of Get is meant to be used with Record.WriteFields.

func (Fields) Set added in v0.8.0 

func (fs Fields) Set(to any, reuse []any) []any

Set writes pointers to the fields of to to an any slice and returns it. If reuse is not nil and has sufficient capacty it will be used to store the pointers. The result of Set is meant to be used with Record.ReadFields. 

type Header [4]byte

Header is the VMP header that can be written to buffers to mark the data to be Vampire encoded and to provide the offsetSize used for the data.

The VMP header is "VMPx" where x encodes the vampire version and the offset size.s

Example 
fmt.Println(HeaderFor(Size16))

Output:

Vampire-v0+16bit

func HeaderFor 

func HeaderFor(s SizeRange) Header

HeaderFor clapms s to valid range

func ParseHeader 

func ParseHeader(mem []byte) (h Header, pos int)

func (Header) Bytes 

func (h Header) Bytes(cap int) []byte

Bytes returns the header as byte slice with added capacity of cap byte.

func (Header) SizeRange 

func (h Header) SizeRange() SizeRange

func (Header) String 

func (h Header) String() string

func (Header) Version 

func (h Header) Version() int

type List added in v0.5.0 

type List struct {
	// contains filtered or unexported fields
}
Example 
a := []string{"boo", "bar", "baz"}
t := NewList(Size32, NewString(Size16))
fmt.Println(t)
buf := NewWriteBuffer(t, Size16, 0, nil)
t.Write(buf, a)
var out any
t.Read(&buf.ReadBuffer, &out)
fmt.Printf("%[1]T=%[1]v\n", out)

Output:

(Size32 "Size16")
[]interface {}=[boo bar baz]

func NewList added in v0.5.0 

func NewList(length SizeRange, elem Type) *List

func (*List) Compare added in v0.8.0 

func (l *List) Compare(t Type, opts CompareOpts) (bool, int)

func (*List) DynNo added in v0.5.0 

func (t *List) DynNo() uint

func (*List) Elem added in v0.5.0 

func (t *List) Elem() Type

func (*List) FixSize added in v0.5.0 

func (t *List) FixSize() uint

func (*List) Hash added in v0.5.0 

func (a *List) Hash(h hash.Hash, opts CompareOpts)

func (*List) LenRange added in v0.5.0 

func (t *List) LenRange() SizeRange

func (*List) ListLen added in v0.5.0 

func (t *List) ListLen(buf *ReadBuffer) (len uint, ok bool)

func (*List) Next added in v0.5.0 

func (t *List) Next(buf *ReadBuffer) (revert uint, ok bool)

TODO Test ListType.Next()

func (*List) PrintTo added in v0.8.0 

func (t *List) PrintTo(w io.Writer) (n int, err error)

func (*List) Read added in v0.5.0 

func (t *List) Read(buf *ReadBuffer, into any) error

func (*List) ReadList added in v0.5.0 

func (t *List) ReadList(buf *ReadBuffer, into ListUnmarshaler) error

func (*List) ReadStream added in v0.5.0 

func (l *List) ReadStream(r io.Reader, offSize SizeRange, buffer []byte) *ReadStream

func (*List) String added in v0.5.0 

func (t *List) String() string

func (*List) VampRecordRead added in v0.5.0 

func (t *List) VampRecordRead(rt *Record, buf *ReadBuffer) error

func (*List) VampRecordWrite added in v0.5.0 

func (t *List) VampRecordWrite(rt *Record, buf *Buffer) error

func (*List) VampUnionAlt added in v0.5.0 

func (t *List) VampUnionAlt() (uint, any)

func (*List) Write added in v0.5.0 

func (t *List) Write(buf *Buffer, val any) error

func (*List) WriteList added in v0.5.0 

func (t *List) WriteList(buf *Buffer, ls ListMarshaler) error

func (*List) WriteStream added in v0.5.0 

func (l *List) WriteStream(w io.Writer, offSize SizeRange, buffer []byte) (*WriteStream, error)

type ListIterator 

type ListIterator interface {
	VampListNext() bool
	VampListGet() any
}

type ListMarshaler 

type ListMarshaler interface {
	VampListIter() ListIterator
}

type ListUnmarshaler 

type ListUnmarshaler interface {
	VampListReader() (any, error)
	VampListAppend(any) error
}

type Path added in v0.6.0 

type Path struct {
	// contains filtered or unexported fields
}

func NewPath added in v0.6.0 

func NewPath(t Type, p ...uint) (res Path, err error)

func (Path) Select added in v0.6.0 

func (p Path) Select(b *ReadBuffer) (rev uint, err error)

type PathElement added in v0.2.0 

type PathElement struct {
	T   Type
	Idx uint
}

type PathError added in v0.2.0 

type PathError struct {
	// contains filtered or unexported fields
}
Example 
rt := NewRecord(
	NewString(Size16),
	Float64,
	Bool,
)
buf := NewWriteBuffer(rt, Size16, 0, nil)
err := rt.WriteFields(buf, "next fails", "", true)
fmt.Println(err)

Output:

.1:vampire cannot write string as float64

func (PathError) Error added in v0.2.0 

func (e PathError) Error() string

func (PathError) Unwrap added in v0.2.0 

func (e PathError) Unwrap() error

type ReadBuffer 

type ReadBuffer struct {
	// contains filtered or unexported fields
}

ReadBuffer adapts a Vampire encoded byte array to be used for read-only access. One has to use the specific Type that matches the encoded data to read the data from the buffer.

func NewReadBuffer 

func NewReadBuffer(t Type, off SizeRange, buf []byte) *ReadBuffer

NewReadBuffer wraps buf for read access when the offsetSize used for the encoded data is known. The first byte of the data of Type t must be buf[0].

func NewReadBufferHeader added in v0.8.0 

func NewReadBufferHeader(t Type, buf []byte, search bool) (*ReadBuffer, error)

NewReadBufferHeader wraps buf for read access when the offsetSize is provided by a VMP Header in buf. When search is true NewReadBufferHeader scans for the first valid VMP header and uses it's offsetSize. Otherwise the VMP header must start at buf[0].

func NewTypeReadBuffer 

func NewTypeReadBuffer(buf []byte, search bool) (*ReadBuffer, error)

func NewTypeReadBufferOffSize 

func NewTypeReadBufferOffSize(off SizeRange, buf []byte) *ReadBuffer

func (*ReadBuffer) AllBytes 

func (b *ReadBuffer) AllBytes() []byte

func (*ReadBuffer) Bytes 

func (b *ReadBuffer) Bytes() []byte

func (*ReadBuffer) Offset 

func (b *ReadBuffer) Offset() SizeRange

func (*ReadBuffer) Unslice 

func (b *ReadBuffer) Unslice(revert uint)

type ReadStream added in v0.5.0 

type ReadStream struct {
	// contains filtered or unexported fields
}

func (*ReadStream) Buffer added in v0.5.0 

func (rs *ReadStream) Buffer() *ReadBuffer

func (*ReadStream) Err added in v0.5.0 

func (rs *ReadStream) Err() error

func (*ReadStream) Head added in v0.5.0 

func (rs *ReadStream) Head() (length int64, data []byte, err error)

Head returns the list's length hint and the raw head data. If the list's actual length is unknown or exceeds the SizeRange of the List, the length hint will be -1.

func (*ReadStream) Next added in v0.5.0 

func (rs *ReadStream) Next() bool

func (*ReadStream) Read added in v0.5.0 

func (rs *ReadStream) Read(into any) error

type Record added in v0.5.0 

type Record struct {
	// contains filtered or unexported fields
}

TODO Slice to field

Example 
r := struct {
	Name string
	Age  uint8
}{
	Name: "John Doe",
	Age:  44,
}
t, _, _ := ReflectType(r)
fmt.Println(t)
buf := NewWriteBuffer(t, Size16, 0, nil)
t.Write(buf, r)
var out any
t.Read(&buf.ReadBuffer, &out)
fmt.Printf("%[1]T=%[1]v\n", out)

Output:

{Name:"Size16" Age:uint8}
map[string]interface {}=map[Age:44 Name:John Doe]

func MustUseStruct added in v0.8.0 

func MustUseStruct(s any) *Record

func NewRecord added in v0.5.0 

func NewRecord(fields ...Type) *Record

func UseStruct added in v0.8.0 

func UseStruct(s any) (*Record, error)

func (*Record) Compare added in v0.8.0 

func (r *Record) Compare(t Type, opts CompareOpts) (bool, int)

func (*Record) DynNo added in v0.5.0 

func (t *Record) DynNo() uint

func (*Record) Field added in v0.5.0 

func (t *Record) Field(buf *ReadBuffer, i uint) (revert uint)

func (*Record) FieldName added in v0.5.0 

func (t *Record) FieldName(i uint) string

func (*Record) FieldType added in v0.5.0 

func (t *Record) FieldType(i uint) Type

func (*Record) FixSize added in v0.5.0 

func (t *Record) FixSize() uint

func (*Record) HasNames added in v0.7.0 

func (t *Record) HasNames() bool

func (*Record) Hash added in v0.5.0 

func (r *Record) Hash(h hash.Hash, opts CompareOpts)

func (*Record) NumField added in v0.5.0 

func (t *Record) NumField() uint

func (*Record) PrintTo added in v0.8.0 

func (t *Record) PrintTo(w io.Writer) (n int, err error)

func (*Record) Read added in v0.5.0 

func (t *Record) Read(buf *ReadBuffer, out any) (err error)

func (*Record) ReadFields added in v0.5.0 

func (t *Record) ReadFields(buf *ReadBuffer, outs ...any) error

func (*Record) String added in v0.5.0 

func (t *Record) String() string

func (*Record) VampArrayFinish added in v0.5.0 

func (t *Record) VampArrayFinish() error

func (*Record) VampArrayLen added in v0.5.0 

func (t *Record) VampArrayLen() uint

func (*Record) VampArrayReader added in v0.5.0 

func (t *Record) VampArrayReader(i uint) (any, error)

func (*Record) VampArrayResize added in v0.5.0 

func (t *Record) VampArrayResize(l uint) error

func (*Record) VampArrayWriter added in v0.5.0 

func (t *Record) VampArrayWriter(i uint) (any, error)

func (*Record) VampUnionAlt added in v0.5.0 

func (t *Record) VampUnionAlt() (uint, any)

func (*Record) Write added in v0.5.0 

func (t *Record) Write(buf *Buffer, value any) error

func (*Record) WriteFields added in v0.5.0 

func (t *Record) WriteFields(buf *Buffer, values ...any) error

type RecordMarshaler 

type RecordMarshaler interface {
	VampRecordWrite(*Record, *Buffer) error
}

type RecordUnmarshaler 

type RecordUnmarshaler interface {
	VampRecordRead(*Record, *ReadBuffer) error
}

type Reflect 

type Reflect struct {
	// contains filtered or unexported fields
}

func DefaultReflect 

func DefaultReflect() Reflect

DefaultReflect uses Size16 for strings and Size32 for arrays.

func (Reflect) ArraySize 

func (r Reflect) ArraySize(s SizeRange) (res Reflect)

func (Reflect) MustType added in v0.7.0 

func (r Reflect) MustType(val any) (Type, Fields)

func (Reflect) MustUseStruct added in v0.8.0 

func (r Reflect) MustUseStruct(s any) *Record

func (Reflect) StringSize 

func (r Reflect) StringSize(s SizeRange) Reflect

func (Reflect) Type added in v0.7.0 

func (r Reflect) Type(val any) (Type, Fields, error)

func (Reflect) UseStruct added in v0.8.0 

func (r Reflect) UseStruct(s any) (*Record, error)

UseStruct computes the vamp record type of a Go struct by reflection and calls UseFieldsForStruct on the computed fields for eficcient use without implementing RecordMarshaler and RecordUnmarshaler.

type SizeRange 

type SizeRange uint8

SizeRanges are used in vamp to define the valid range for buffer offsets, string length and any sort of ranges of size in byte that is used in Vampire encoding. 

const (
	// Size8 is a 8-bit size range with values 0…255
	Size8 SizeRange = iota

	// Size16 is a 16-bit size range with values 0…65535
	Size16

	// Size32 is a 32-bit size range with values 0…4294967295
	Size32
)

func SizeRangeFromBits added in v0.7.1 

func SizeRangeFromBits(n int) (SizeRange, error)

func SizeRangeFromMax added in v0.8.0 

func SizeRangeFromMax(max uint) (SizeRange, error)

func (SizeRange) Check 

func (s SizeRange) Check(size uint) error

Check checks if size is withing the SizeRange s. Only if size is not in s' range an error is returned.

func (SizeRange) Max 

func (s SizeRange) Max() uint

Max returns the maximum uint of SizeRange s. SizeRange s is expected to be s.Valid()==true.

func (SizeRange) Size 

func (s SizeRange) Size() uint

Size returns the number of bytes used for the size range in Vampire encoding. SizeRange s is expected to be s.Valid()==true.

func (SizeRange) String 

func (i SizeRange) String() string

func (*SizeRange) VampFromUint8 

func (s *SizeRange) VampFromUint8(i uint8) error

SizeRange can be decoded from a Vampire Uint8.

func (SizeRange) VampToUint8 

func (s SizeRange) VampToUint8() (uint8, error)

SizeRange can be encoded as Vampire Uint8.

type Type 

type Type interface {
	FixSize() uint // fix size of the type in byte
	DynNo() uint   // number of dynamic elements of the type

	// Write the value to buffer if its structure matches Type
	Write(b *Buffer, value any) error

	// Read buffers content into argument if its structure matches Type
	Read(b *ReadBuffer, into any) error

	// Compare this type with type t. A type w is less than or equal to type r
	// if vampire data written by w can be ready with type r. Only if neither w
	// is less than or equal to r nor r is less than or equal to w is comparable
	// false. Otherwise, comparison is 0 for equal types, less than 0 if this
	// type is less than t or greater than 0 if this t is less than this type.
	Compare(t Type, opts CompareOpts) (comparable bool, comparison int)

	Hash(h hash.Hash, opts CompareOpts)

	fmt.Stringer
	PrintTo(w io.Writer) (int, error)
}

Type is the interface of all Vampire types that are used to define the structure of marshaled data. Types can be created as Array, List, Record or Union form other types. Basic types are booleans, signed and unsigned integers with 8, 16, 32 or 64 bit and float with 32 or 64 bit. The special Bytes types are used to hold any binary data, especially UTF-8 strings. Finally there is the Any type that can hold any Vampire value as a tuple of a value and its type.

func Anonymized added in v0.7.0 

func Anonymized(t Type) Type

Anonymized returns the unnamed type of t. If t is already unnamed it is returned.

func MustNamed added in v0.6.0 

func MustNamed(n string, t Type) Type

func Named added in v0.4.0 

func Named(n string, t Type) (Type, error)

Named attaches a name to a type. Type names are irrelevant for the encoded data end exist for documentation only. Passing a named type to the constructors of record or union types will assign the name to the respective record field or union variant.

func ParseType added in v0.7.0 

func ParseType(rd io.Reader) (typ Type, err error)

func ParseTypeString added in v0.7.0 

func ParseTypeString(s string) (Type, error)
Example 
typ, _ := ParseTypeString(`<Size8
		Foo:[Size8 "Size8"]
	  | float32
	  | {ID:uint32 Name:"Size16"}
	>`)
fmt.Println(typ)

Output:

<Size8 Foo:[Size8 "Size8"]|float32|{ID:uint32 Name:"Size16"}>

func ReadType 

func ReadType(buf *ReadBuffer) (t Type, err error)

func ReadTypeFile added in v0.7.1 

func ReadTypeFile(name string) (Type, error)

type Union added in v0.5.0 

type Union struct {
	// When not nil it will be used in [Union.Read] to select the read target
	// depending on the union's alt field. The original into argument of the
	// Read method is passed to AltTarget and the returend reader then is used
	// to read the alt type into. If reader is nil the union is skipped.
	AltTarget func(alt uint, into any) (reader any, err error)
	// contains filtered or unexported fields
}
Example 
type address struct {
	Street string
	No     string
}
type geocoos struct {
	Lat, Lon float64
}
addrType, _, _ := ReflectType(address{})
geoType, _, _ := ReflectType(geocoos{})
placeType := NewUnion(Size8,
	addrType,
	MustNamed("geo", geoType),
)
fmt.Println(placeType)
buf := NewWriteBuffer(placeType, Size16, 0, nil)
placeType.Write(buf, Alt{S: 0, V: address{Street: "Justroad", No: "33a"}})
var out any
placeType.Read(&buf.ReadBuffer, &out)
fmt.Printf("%[1]T=%[1]v\n", out)
buf.Reset(placeType, Size16, 0)
placeType.Write(buf, Alt{S: 1, V: geocoos{Lat: 33.321, Lon: 44.123}})
placeType.Read(&buf.ReadBuffer, &out)
fmt.Printf("%[1]T=%[1]v\n", out)

Output:

<Size8 {Street:"Size16" No:"Size16"}|geo:{Lat:float64 Lon:float64}>
map[string]interface {}=map[No:33a Street:Justroad]
map[string]interface {}=map[Lat:33.321 Lon:44.123]

func NewUnion added in v0.5.0 

func NewUnion(altRange SizeRange, alts ...Type) *Union

func (*Union) Alt added in v0.5.0 

func (t *Union) Alt(i uint) Type

func (*Union) AltName added in v0.5.0 

func (t *Union) AltName(i uint) string

func (*Union) AltRange added in v0.5.0 

func (t *Union) AltRange() SizeRange

func (*Union) Compare added in v0.8.0 

func (u *Union) Compare(t Type, opts CompareOpts) (bool, int)

func (*Union) DynNo added in v0.5.0 

func (t *Union) DynNo() uint

func (*Union) FixSize added in v0.5.0 

func (t *Union) FixSize() uint

func (*Union) Hash added in v0.5.0 

func (u *Union) Hash(h hash.Hash, opts CompareOpts)

func (*Union) NumAlts added in v0.5.0 

func (t *Union) NumAlts() uint

func (*Union) PrintTo added in v0.8.0 

func (t *Union) PrintTo(w io.Writer) (n int, err error)

func (*Union) Read added in v0.5.0 

func (t *Union) Read(buf *ReadBuffer, into any) error

func (*Union) ReadAlt added in v0.5.0 

func (t *Union) ReadAlt(buf *ReadBuffer, altInto func(uint) (any, error)) (any, error)

func (*Union) SetAltReader added in v0.7.0 

func (t *Union) SetAltReader(f func(uint, any) (any, error)) *Union

func (*Union) String added in v0.5.0 

func (t *Union) String() string

func (*Union) VampRecordRead added in v0.5.0 

func (t *Union) VampRecordRead(rt *Record, buf *ReadBuffer) error

func (*Union) VampRecordWrite added in v0.5.0 

func (t *Union) VampRecordWrite(rt *Record, buf *Buffer) error

func (*Union) VampUnionAlt added in v0.5.0 

func (t *Union) VampUnionAlt() (uint, any)

func (*Union) Write added in v0.5.0 

func (t *Union) Write(buf *Buffer, value any) error

func (*Union) WriteAlt added in v0.5.0 

func (t *Union) WriteAlt(buf *Buffer, alt uint, value any) error

type UnionMarshaler 

type UnionMarshaler interface {
	VampUnionAlt() (uint, any)
}

type UnionUnmarshaler 

type UnionUnmarshaler interface {
	VampUnionReader(alt uint, into any) (any, error)
}

type WriteStream added in v0.5.0 

type WriteStream struct {
	// contains filtered or unexported fields
}

func (*WriteStream) Flush added in v0.5.0 

func (ws *WriteStream) Flush() error

func (*WriteStream) Write added in v0.5.0 

func (ws *WriteStream) Write(elem any) error

func (*WriteStream) WriteLast added in v0.5.0 

func (ws *WriteStream) WriteLast(elem any) error

func (*WriteStream) WriteWithNext added in v0.5.0 

func (ws *WriteStream) WriteWithNext(elem any) error

Source Files

View all Source files

Directories

Path Synopsis
cmd
vamp
The vamp CLI tool converts type definitions between vamp and text encoding and outputs vamp data as JSON.
 The vamp CLI tool converts type definitions between vamp and text encoding and outputs vamp data as JSON.
examples
cs
Package cs demonstartes how to use Vampire for client/server protocol.
 Package cs demonstartes how to use Vampire for client/server protocol.
cs/client
cs/protocol
cs/server
polymorphic
Package polymorphic demonstartes how to use unions.
 Package polymorphic demonstartes how to use unions.
selfcontained
Package selfcontained demonstrates how to use the any type to create a self-contained vmp file that can be read without an external type definition.
 Package selfcontained demonstrates how to use the any type to create a self-contained vmp file that can be read without an external type definition.
stream
wrind

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.