dynobuffers

package module
v0.0.0-...-b1cab92 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 13, 2023 License: MIT Imports: 14 Imported by: 1

README

codecov

Dyno Buffers

  • Codegen-less wrapper for FlatBuffers with get\set by name feature
  • Status: under development

Features

  • Uses FlatBuffer to read\write values from\to byte array converting to the required type described in the Scheme.
  • No codegen, no compilers, no (de)serialization. Just fields description and get\set by name.
  • In contrast to FlatBuffers tracks if the field was unset or initially not set
  • Supported types
    • int32, int64, float32, float64, bool, string, byte
    • nested objects
    • arrays
  • Empty strings, nested objects and arrays are not stored (Get() returns nil)
  • Strings could be set as both string and []byte, string arrays - as both []string and [][]byte. Get(), ToJSON() and ToJSONMap() returns string or array of strings
  • Scheme versioning
    • Any data written with Scheme of any version will be correctly read using Scheme of any other version
      • Written in old Scheme, read in New Scheme -> nil result on new field read, field considered as unset
      • Written in new Scheme, read in old Scheme -> no errors
  • Data could be loaded from JSON (using gojay) or from map[string]interface{}

Limitations

  • Only 2 cases of scheme modification are allowed: field rename and append fields to the end. This is necessary to have ability to read byte buffers in Scheme of any version
  • Written in New -> read in Old -> write in Old -> New fields are lost (todo)

Installation

go get github.com/untillpro/dynobuffers

Usage

  • Describe Scheme
    • By yaml. Field types:
      • int32
      • int64
      • float32
      • float64
      • bool
      • string
      • byte
      var schemeStr = `
name: string
price: float32
quantity: int32
Id: int64 # first letter is capital -> field is mandatory
`
scheme, err := dynobuffers.YamlToScheme(schemeStr)
if err != nil {
	panic(err)
}
    • Manually 
      scheme := dynobuffers.NewScheme()
scheme.AddField("name", dynobuffers.FieldTypeString, false)
scheme.AddField("price", dynobuffers.FieldTypeFloat, false)
scheme.AddField("quantity", dynobuffers.FieldTypeInt, false)
scheme.AddField("id", dynobuffers.FieldTypeLong, true)
  • Create empty Dyno Buffer using Scheme 
    b := dynobuffers.NewBuffer(scheme)
    • panics if nil provided
  • Set\modify fields according to the Scheme 
    b.Set("price", float32(0.123))
b.Set("name", nil) // unset field
  • To bytes array 
    bytes, err := b.ToBytes()
if err != nil {
	panic(err)
}
  • To JSON key-value 
    jsonStr := b.ToJSON()
    Note: arrays of byte are encoded to base64 strings
  • To map key-value (JSON-compatible) 
    jsonMap := b.ToJSONMap()
  • Read Buffer from bytes using Scheme 
    b = dynobuffers.ReadBuffer(bytes, scheme)
    • panics if nil Scheme provided
  • Work with Buffer 
    value, ok := b.GetFloat32("price") // read typed. !ok -> field is unset or no such field in the scheme. Works faster and takes less memory allocations than Get()
b.Get("price") // read untyped. nil -> field is unset or no such field in the scheme
b.Set("price", nil) // set to nil means unset
bytes = b.ToBytes()
  • Load data from JSON key-value and to bytes array 
    bytes, nilled, err := b.ApplyJSONAndToBytes([]byte(`{"name": "str", "price": 0.123, "fld": null}`))
if err != nil {
	panic(err)
}
    • nilled will contain list of field names whose values were effectively set to nil, i.e. fields names whose values were provided as null or as an empty object, array or string
      • note: nils are not stored in bytes
    • value is nil and field is mandatory -> error
    • value type and field type are incompatible (e.g. string provided for numeric field) -> error
    • float value is provided for an integer field -> no error, integer part is considered only. E.g. 0.123 value in JSON is met -> integer field value is 0
    • no such field in the scheme -> error
    • array element value is nil -> error (not supported)
    • values for byte arrays are expected to be base64 strings
  • Load data from map[string]interface{} 
    m := map[string]interface{} {
	"name": "str",
	"price": 0.123,
	"fld": nil,
}
if err := b.ApplyMap(m); err != nil {
	panic(err)
}
bytes, err := b.ToBytes()
if err != nil {
	panic(err)
}
    • value type and field type differs but value fits into field (e.g. float64(255) fits into float, double, int, long, byte; float64(256) does not fit into byte etc) -> ok
    • the rest is the same as for ApplyJSONAndToBytes()
  • Check if a field exists in the scheme and is set to non-nil 
    b.HasValue("name")
  • Return Buffer to pool to prevent additional memory allocations 
    b.Release()
// b itself, all objects created manually and used in b.Set(), all objects got using `b.Get()` are released also.
// nor these objects neither result of `b.ToBytes()` must not be used from now on
  • Iterate over fields which has value 
    b.IterateFields(nil, func(name string, value interface{}) bool {
    return true // continue iterating on true, stop on false
})
  • Iterate over specified fields only. Will iterate over each specified name if according field has a value. Unknown field name -> no iteration 
    b.IterateFields([]string{"name", "price", "unknown"}, func(name string, value interface{}) bool {
    return true // continue iterating on true, stop on false
})
  • See dynobuffers_test.go for usage examples

Nested objects

  • Declare scheme
    • by yaml 
      var schemeStr := `
name: string
Nested:
  nes1: int32
  Nes2: int32
Id: int64
`
schemeRoot := dynobuffers.YamlToScheme(schemeStr)
    • manually 
      schemeNested := dynobuffers.NewScheme()
schemeNested.AddField("nes1", dynobuffers.FieldTypeInt, false)
schemeNested.AddField("nes2", dynobuffers.FieldTypeInt, true)
schemeRoot := dynobuffers.NewScheme()
schemeRoot.AddField("name", dynobuffers.FieldTypeString, false)
schemeRoot.AddNested("nested", schemeNested, true)
schemeRoot.AddField("id", dynobuffers.FieldTypeLong, true)
  • Create Buffer, fill and to bytes 
    bRoot := dynobuffers.NewBuffer(schemeRoot)
bNested := dynobuffers.NewBuffer(schemeNested)
bNested.Set("nes1", 1)
bNested.Set("nes2", 2)
bRoot.Set("name", "str")
bRoot.Set("nested", bNested)
bytes, err := bRoot.ToBytes()
if err != nil {
	panic(err)
}
  • Read from bytes, modify and to bytes again 
    bRoot = dynobuffers.ReadBuffer(bytes, schemeRoot)
bRoot.Set("name", "str modified")
bNested := bRoot.Get("nested").(*dynobuffers.Buffer)
bNested.Set("nes2", 3)
bytes, err := bRoot.ToBytes()
if err != nil {
	panic(err)
}
bRoot = dynobuffers.ReadBuffer(bytes, scheme)
// note: bNested is obsolete here. Need to obtain it again from bRoot
  • Empty nested objects are not stored
  • Unmodified nested objects are copied field by field on ToBytes()
  • See dynobuffers_test.go for usage examples

Arrays

  • Declare scheme
    • by yaml. Append .. to field name to make it array 
      var schemeStr = `
name: string
Nested..:
  nes1: int32
  Nes2: int32
Ids..: int64
`
schemeRoot := dynobuffers.YamlToScheme(schemeStr)
    • manually 
      schemeNested := dynobuffers.NewScheme()
schemeNested.AddField("nes1", dynobuffers.FieldTypeInt, false)
schemeNested.AddField("nes2", dynobuffers.FieldTypeInt, true)
schemeRoot := dynobuffers.NewScheme()
schemeRoot.AddField("name", dynobuffers.FieldTypeString, false)
schemeRoot.AddNestedArray("nested", schemeNested, true)
schemeRoot.AddArray("ids", dynobuffers.FieldTypeLong, true)
  • Create Buffer, fill and to bytes 
    bRoot := dynobuffers.NewBuffer(schemeRoot)
buffersNested := make([]*Buffer, 2)

bNested := dynobuffers.NewBuffer(schemeNested)
bNested.Set("nes1", 1)
bNested.Set("nes2", 2)
buffersNested = append(buffersNested, bNested)

bNested = dynobuffers.NewBuffer(schemeNested)
bNested.Set("nes1", 3)
bNested.Set("nes2", 4)
buffersNested = append(buffersNested, bNested)

ids := []int64{5,6}
bRoot.Set("name", "str")
bRoot.Set("nested", buffersNested)
bRoot.Set("ids", ids)
bytes, err := bRoot.ToBytes()
if err != nil {
	panic(err)
}
  • Read array
    • By iterator 
      bRoot = dynobuffers.ReadBuffer(bytes, schemeRoot)
int64Arr := assert.Equal(t, int64(5), bRoot.GetInt64Array("ids"))
for i := 0; i < int64Arr.Len(); i++ {
	assertEqual(t, ids[i], int64Arr.At(i))
}
    • Read filled array of non-objects 
      bRoot = dynobuffers.ReadBuffer(bytes, schemeRoot)
arr := b.Get("ids").([]int64)
    • Read array of objects using iterator 
      bRoot = dynobuffers.ReadBuffer(bytes, schemeRoot)
arr := bRoot.Get("nested").(*dynobuffers.ObjectArray) // ObjectArray is iterator over nested entities
for arr.Next() {
	// arr.Buffer is switched on each arr.Next()
	assert.Equal(t, int32(1), arr.Buffer.Get("nes1"))
}
// note: not need to release `arr`. It will be released on `b.Release()`
  • Modify array and to bytes
    • Set 
      bRoot = dynobuffers.ReadBuffer(bytes, schemeRoot)

ids := []int64{5,6}
bRoot.Set("ids", ids)

arr := bRoot.Get("nested").(*dynobuffers.ObjectArray)
arr.Next()
arr.Buffer.Set("nes1", -1)
bRoot.Set("nested", arr)
bytes, err := bRoot.ToBytes()
if err != nil {
	panic(err)
}
bRoot = dynobuffers.ReadBuffer(bytes, scheme)
// note: `arr` is obsolete here. Need to obtain the array again from bRoot
    • Append 
      bRoot = dynobuffers.ReadBuffer(bytes, schemeRoot)
// if wasn't set then equivalent to bRoot.Set()
bRoot.Append("ids", []int32{7, 8}) // 7 and 8 will be appended

buffersNested := []*Buffer{}
bNested = dynobuffers.NewBuffer(schemeNested)
bNested.Set("nes1", 5)
bNested.Set("nes2", 6)
buffersNested = append(buffersNested, bNested)
bRoot.Append("nested", buffersNested) // bNested will be appended
bytes, err := bRoot.ToBytes()
if err != nil {
	panic(err)
}
  • Null\nil array element is met on ApplyJSONAndToBytes(), Set(), Append() or ApplyMap() -> error, not supported
  • Arrays are appended (set if there is nothing to append to) if met on ApplyJSONAndToBytes() and ApplyMap()
  • Byte arrays are decoded to JSON as base64 strings
  • Byte array value could be set from either byte array and base64-encoded string
  • Empty array -> no array, Get() will return nil, HasValue() will return false
  • Append() or Set() nil or epmty array means unset the array
  • See dynobuffers_test.go for usage examples

TODO

  • For now there are 2 same methods: ApplyMapBuffer() and ApplyJSONAndToBytes(). Need to get rid of one of them.
  • For now ToBytes() result must not be stored if Release() is used because on next ToBytes() the stored previous ToBytes() result will be damaged. See TestPreviousResultDamageOnReuse(). The better soultion is to make ToBytes() return not []byte but an interface {Bytes() []byte; Release()}.
  • ToJSON(): use bytebufferpool?
  • Array of nested entities modification is not supported

Benchmarks

Reading Many Fields

  • cd benchmarks
  • go test -bench=ReadAllArticleFields -benchmem
goos: windows
goarch: amd64
pkg: github.com/untillpro/dynobuffers/benchmarks
Benchmark_ReadAllArticleFields_Avro-8                      71046             23394 ns/op           11257 B/op        149 allocs/op
Benchmark_ReadAllArticleFields_Dyno_Untyped-8             195801              6437 ns/op             808 B/op         84 allocs/op
Benchmark_ReadAllArticleFields_Dyno_Typed-8               313376              3776 ns/op               0 B/op          0 allocs/op
Benchmark_ReadAllArticleFields_Flat-8                    1000000              1132 ns/op               0 B/op          0 allocs/op
Benchmark_ReadAllArticleFields_Json-8                      14431             87331 ns/op           14145 B/op        603 allocs/op

Reading Few of Many Fields

  • cd benchmarks
  • go test -bench=ReadFewArticleFields -benchmem
goos: windows
goarch: amd64
pkg: github.com/untillpro/dynobuffers/benchmarks
Benchmark_ReadFewArticleFields_Avro-8              98437             19311 ns/op           11257 B/op        149 allocs/op
Benchmark_ReadFewArticleFields_Dyno_Typed-8     18520500                62.2 ns/op             0 B/op          0 allocs/op
Benchmark_ReadFewArticleFields_Flat-8           60032416                19.8 ns/op             0 B/op          0 allocs/op
Benchmark_ReadFewArticleFields_Json-8              15333             83824 ns/op           11073 B/op        603 allocs/op

Reading Few Fields

  • cd benchmarks
  • go test -bench=ReadSimple -benchmem
goos: windows
goarch: amd64
pkg: github.com/untillpro/dynobuffers/benchmarks
Benchmark_ReadSimple_Avro-8              2038466              1193 ns/op             744 B/op         10 allocs/op
Benchmark_ReadSimple_Dyno_Typed-8       20017480                55.2 ns/op             0 B/op          0 allocs/op
Benchmark_ReadSimple_Flat-8             59981404                20.2 ns/op             0 B/op          0 allocs/op
Benchmark_ReadSimple_Flat_String-8      29275360                40.7 ns/op             0 B/op          0 allocs/op
Benchmark_ReadSimple_Json-8               545769              2423 ns/op             776 B/op         21 allocs/op

NOTE: DynoBuffers allocs caused by string types

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func GetObjectsInUse 

func GetObjectsInUse() uint64

GetObjectsInUse returns pooled objects amount which are currently in use, i.e. not released useful for testing and metrics accounting

func IsFloat64ValueFitsIntoField 

func IsFloat64ValueFitsIntoField(f *Field, float64Src float64) bool

IsFloat64ValueFitsIntoField checks if target type of field enough to fit float64 value e.g. float64(1) could be applied to any numeric field, float64(256) to any numeric except FieldTypeByte etc Useful to check float64 values came from JSON

Types

type Buffer 

type Buffer struct {
	Scheme *Scheme
	// contains filtered or unexported fields
}

Buffer is wrapper for FlatBuffers

func NewBuffer 

func NewBuffer(Scheme *Scheme) *Buffer

NewBuffer creates new empty Buffer

func ReadBuffer 

func ReadBuffer(bytes []byte, Scheme *Scheme) *Buffer

ReadBuffer creates Buffer from bytes using provided Scheme

func (*Buffer) Append 

func (b *Buffer) Append(name string, toAppend interface{})

Append appends an array field. toAppend could be a single value or an array of values Value for byte array field could be base64 string or []byte Rewrites previous modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap `Get()` will not consider modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap Nil or empty array is provided -> equals to unset the field

func (*Buffer) ApplyJSONAndToBytes 

func (b *Buffer) ApplyJSONAndToBytes(jsonBytes []byte) (res []byte, nilled []string, err error)

ApplyJSONAndToBytes sets field values described by provided json and returns new FlatBuffer byte array with inital + applied data See `ApplyMapBuffer` for details

func (*Buffer) ApplyMap 

func (b *Buffer) ApplyMap(data map[string]interface{}) error

ApplyMap sets field values described by provided map[string]interface{} Resulting buffer has no value (or has nil value) for a mandatory field -> error Value type and field type are incompatible (e.g. string for numberic field) -> error Value and field types differs but value fits into field -> no error. Examples: 

255 fits into float, double, int, long, byte;
256 does not fit into byte
math.MaxInt64 does not fit into int32

Unexisting field is provided -> error Byte arrays could be base64 strings or []byte Array element is nil -> error (not supported) Note: float is provided for an int field -> error, whereas no error on ApplyJSONAndToBytes() (gojay feature) `Get()` will not consider modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap Rewrites previous modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap Non-empty array is applied over an existing array -> incomming array is appended to the existing Nil or empty array is applied over an existing array -> existing array is unset

func (*Buffer) ApplyMapBuffer 

func (b *Buffer) ApplyMapBuffer(jsonMap []byte) error

ApplyMapBuffer modifies Buffer with JSON specified by jsonMap ToBytes() will return byte array with initial + applied data float is provided for an int field -> no error, integer part only is used (gojay feature), whereas is an error on ApplyMap() Values for byte arrays are expected to be base64 strings `Get()` will not consider modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap Rewrites previous modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap

func (*Buffer) CommitChanges 

func (b *Buffer) CommitChanges() error

applies all current modifications to the underlying byte array and clears all modifications note: takes one byte array allocation

func (*Buffer) Get 

func (b *Buffer) Get(name string) interface{}

Get returns stored field value by name. field is scalar -> scalar is returned field is an array of scalars -> []T is returned field is a nested object -> *dynobuffers.Buffer is returned. field is an array of nested objects -> *dynobuffers.ObjectArray is returned. field is not set, set to nil or no such field in the Scheme -> nil `Get()` will not consider modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap

func (*Buffer) GetBool 

func (b *Buffer) GetBool(name string) (bool, bool)

GetBool returns bool value by name and if the Scheme contains the field and if the value was set to non-nil

func (*Buffer) GetBoolArray 

func (b *Buffer) GetBoolArray(name string) IBoolArray

func (*Buffer) GetByField 

func (b *Buffer) GetByField(f *Field) interface{}

GetByField is an analogue of Get() but accepts a known Field

func (*Buffer) GetByte 

func (b *Buffer) GetByte(name string) (byte, bool)

GetByte returns byte value by name and if the Scheme contains the field and if the value was set to non-nil

func (*Buffer) GetByteArray 

func (b *Buffer) GetByteArray(name string) IByteArray

func (*Buffer) GetBytes 

func (b *Buffer) GetBytes() []byte

GetBytes is an alias for ToBytes(). Simply returns underlying buffer if no modifications. Returns nil on error

func (*Buffer) GetFloat32 

func (b *Buffer) GetFloat32(name string) (float32, bool)

GetFloat32 returns float32 value by name and if the Scheme contains the field and if the value was set to non-nil

func (*Buffer) GetFloat32Array 

func (b *Buffer) GetFloat32Array(name string) IFloat32Array

func (*Buffer) GetFloat64 

func (b *Buffer) GetFloat64(name string) (float64, bool)

GetFloat64 returns float64 value by name and if the Scheme contains the field and if the value was set to non-nil

func (*Buffer) GetFloat64Array 

func (b *Buffer) GetFloat64Array(name string) IFloat64Array

func (*Buffer) GetInt32 

func (b *Buffer) GetInt32(name string) (int32, bool)

GetInt32 returns int32 value by name and if the Scheme contains the field and the value was set to non-nil

func (*Buffer) GetInt32Array 

func (b *Buffer) GetInt32Array(name string) IInt32Array

func (*Buffer) GetInt64 

func (b *Buffer) GetInt64(name string) (int64, bool)

GetInt64 returns int64 value by name and if the Scheme contains the field and if the value was set to non-nil

func (*Buffer) GetInt64Array 

func (b *Buffer) GetInt64Array(name string) IInt64Array

func (*Buffer) GetString 

func (b *Buffer) GetString(name string) (string, bool)

GetString returns string value by name and if the Scheme contains the field and if the value was set to non-nil

func (*Buffer) GetStringArray 

func (b *Buffer) GetStringArray(name string) IStringArray

func (*Buffer) HasValue 

func (b *Buffer) HasValue(name string) bool

HasValue returns if specified field exists in the scheme and its value is set to non-nil

func (*Buffer) IsModified 

func (b *Buffer) IsModified() bool

func (*Buffer) IsNil 

func (b *Buffer) IsNil() bool

IsNil returns if current buffer means nothing need to comply to gojay.MarshalerJSONObject

func (*Buffer) IterateFields 

func (b *Buffer) IterateFields(names []string, callback func(name string, value interface{}) bool)

IterateFields calls `callback` for each fields which has a value. `names` empty -> calback is called for all fields which has a value `names` not empty -> callback is called for each specified name if according field has a value callbeck returns false -> iteration stops

func (*Buffer) MarshalJSONObject 

func (b *Buffer) MarshalJSONObject(enc *gojay.Encoder)

MarshalJSONObject encodes current Buffer into JSON using gojay. Complies to gojay.MarshalerJSONObject interface

func (*Buffer) NKeys 

func (b *Buffer) NKeys() int

NKeys returns Schemes's root fields amount. Conforms to gojay.UnmarshalerJSONObject interface

func (*Buffer) Release 

func (b *Buffer) Release()

Release returns used Buffer into pool Note: Buffer instance itself, result of ToBytes() must not be used after Release()

func (*Buffer) Reset 

func (b *Buffer) Reset(bytes []byte)

Reset sets current underlying byte array and clears modified fields. Useful for *Buffer instance reuse Note: bytes must match the Buffer's scheme

func (*Buffer) Set 

func (b *Buffer) Set(name string, value interface{})

Set sets field value by name. Call ToBytes() to get modified byte array Value for byte array field could be base64 string or []byte `Get()` will not consider modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap Rewrites previous modifications made by Set, Append, ApplyJSONAndToBytes, ApplyMapBuffer, ApplyMap

func (*Buffer) ToBytes 

func (b *Buffer) ToBytes() ([]byte, error)

ToBytes returns new FlatBuffer byte array with fields modified by Set() and fields which initially had values Note: initial byte array and current modifications are kept

func (*Buffer) ToBytesNilled 

func (b *Buffer) ToBytesNilled() (res []byte, nilledFields []string, err error)

ToBytesNilled constructs resulting byte array (nil values are not stored) and list of field names which were nilled using Set, ApplyJSON, ApplyMap useful in cases when we should know which fields were null on load from map or JSON nilled fields of nested objects are not considered

func (*Buffer) ToBytesWithBuilder 

func (b *Buffer) ToBytesWithBuilder(builder *flatbuffers.Builder) error

ToBytesWithBuilder same as ToBytes but uses builder builder.Reset() is invoked

func (*Buffer) ToJSON 

func (b *Buffer) ToJSON() []byte

ToJSON returns JSON key->value string empty buffer -> "{}"

func (*Buffer) ToJSONMap 

func (b *Buffer) ToJSONMap() map[string]interface{}

ToJSONMap returns map[string]interface{} representation of the buffer compatible to json result map is built using inital data + current modifications numeric field types are kept (not float64 as json.Unmarshal() does) nested object, array, array element is empty or nil -> skip empty buffer -> empty map is returned

func (*Buffer) UnmarshalJSONObject 

func (b *Buffer) UnmarshalJSONObject(dec *gojay.Decoder, fn string) (err error)

UnmarshalJSONObject unmarshals JSON into the current Buffer using Gojay. Conforms to gojay.UnmarshalerJSONObject interface

type Field 

type Field struct {
	Name        string
	Ft          FieldType
	Order       int
	IsMandatory bool
	FieldScheme *Scheme // != nil for FieldTypeObject only

	IsArray bool
	// contains filtered or unexported fields
}

Field describes a Scheme field

func (*Field) QualifiedName 

func (f *Field) QualifiedName() string

QualifiedName returns ownerScheme.fieldName

type FieldType 

type FieldType int

FieldType s.e. 

const (
	// FieldTypeUnspecified - wrong type
	FieldTypeUnspecified FieldType = iota
	// FieldTypeObject field is nested Scheme
	FieldTypeObject
	// FieldTypeInt32 int32
	FieldTypeInt32
	// FieldTypeInt64 int64
	FieldTypeInt64
	// FieldTypeFloat32 float32
	FieldTypeFloat32
	// FieldTypeFloat64 float64
	FieldTypeFloat64
	// FieldTypeString variable length
	FieldTypeString
	// FieldTypeBool bool
	FieldTypeBool
	// FieldTypeByte byte
	FieldTypeByte
)

type IBoolArray 

type IBoolArray interface {
	Len() int
	At(idx int) bool
}

type IByteArray 

type IByteArray interface {
	Bytes() []byte
}

type IFloat32Array 

type IFloat32Array interface {
	Len() int
	At(idx int) float32
}

type IFloat64Array 

type IFloat64Array interface {
	Len() int
	At(idx int) float64
}

type IInt32Array 

type IInt32Array interface {
	Len() int
	At(idx int) int32
}

type IInt64Array 

type IInt64Array interface {
	Len() int
	At(idx int) int64
}

type IStringArray 

type IStringArray interface {
	Len() int
	At(idx int) string
}

type ObjectArray 

type ObjectArray struct {
	Buffer *Buffer
	Len    int
	// contains filtered or unexported fields
}

ObjectArray used to iterate over array of nested objects ObjectArray.Buffer should be used for reading only

func (*ObjectArray) Next 

func (oa *ObjectArray) Next() bool

Next proceeds to a next nested object in the array. If true then .Buffer represents the next element

func (*ObjectArray) Release 

func (oa *ObjectArray) Release()

Release returns used ObjectArray instance to the pool. Releases also ObjectArray.Buffer Note: ObjectArray instance itself, ObjectArray.Buffer, result of ObjectArray.Buffer.ToBytes() must not be used after Release()

func (*ObjectArray) Value 

func (oa *ObjectArray) Value() interface{}

Value returns *dynobuffers.Buffer instance as current element

type Scheme 

type Scheme struct {
	Name      string
	FieldsMap map[string]*Field
	Fields    []*Field
}

Scheme describes fields and theirs order in byte array

func MapSliceToScheme 

func MapSliceToScheme(mapSlice yaml.MapSlice) (*Scheme, error)

MapSliceToScheme s.e.

func NewScheme 

func NewScheme() *Scheme

NewScheme creates new empty Scheme

func YamlToScheme 

func YamlToScheme(yamlStr string) (*Scheme, error)

YamlToScheme creates Scheme by provided yaml `fieldName: yamlFieldType` Field types:

  • `int` -> `int32`
  • `long` -> `int64`
  • `float` -> `float32`
  • `double` -> `float64`
  • `bool` -> `bool`
  • `string` -> `string`
  • `byte` -> `byte`

Field name starts with the capital letter -> field is mandatory Field name ends with `..` -> field is an array See [dynobuffers_test.go](dynobuffers_test.go) for examples

func (*Scheme) AddArray 

func (s *Scheme) AddArray(name string, elementType FieldType, isMandatory bool) *Scheme

AddArray adds array field

func (*Scheme) AddField 

func (s *Scheme) AddField(name string, ft FieldType, isMandatory bool) *Scheme

AddField adds field

func (*Scheme) AddFieldC 

func (s *Scheme) AddFieldC(name string, ft FieldType, nested *Scheme, isMandatory bool, IsArray bool) *Scheme

AddFieldC adds new finely-tuned field

func (*Scheme) AddNested 

func (s *Scheme) AddNested(name string, nested *Scheme, isMandatory bool) *Scheme

AddNested adds nested object field

func (*Scheme) AddNestedArray 

func (s *Scheme) AddNestedArray(name string, nested *Scheme, isMandatory bool) *Scheme

AddNestedArray adds array of nested objects field

func (*Scheme) GetNestedScheme 

func (s *Scheme) GetNestedScheme(nestedObjectField string) *Scheme

GetNestedScheme returns Scheme of nested object if the field has FieldTypeObject type, nil otherwise

func (*Scheme) MarshalYAML 

func (s *Scheme) MarshalYAML() (interface{}, error)

MarshalYAML marshals Scheme to yaml. Needs to conform to yaml.Marshaler interface

func (*Scheme) UnmarshalYAML 

func (s *Scheme) UnmarshalYAML(unmarshal func(interface{}) error) error

UnmarshalYAML unmarshals Scheme from yaml. Conforms to yaml.Unmarshaler interface fields will be replaced with the ones came from the yaml

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.