anyenc

package
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 26, 2024 License: MIT Imports: 11 Imported by: 10

README

AnyEnc Binary Format

AnyEnc is a high-performance binary format used by Any-Store. The format is designed to handle various data types, including raw binary, and allows direct byte-level comparisons via bytes.Compare. Its structure is similar to JSON. The Go library’s interface closely resembles that of fastjson, providing a high-performance, memory-efficient API.

Data Format Structure

Each serialized item starts with a byte that defines its type. Supported data types and their corresponding byte identifiers:

  • 0x01 — Null value
  • 0x02 — Number (float64)
  • 0x03 — String
  • 0x04 — Boolean False
  • 0x05 — Boolean True
  • 0x06 — Array
  • 0x07 — Object
  • 0x08 — Binary data
Serialization Details:
  • Null, True, False: Represented by a single byte indicating the type (0x01, 0x05, or 0x04).
  • Number: A type byte (0x02) followed by an 8-byte float64.
  • String: A type byte (0x03), followed by the string data, terminated by 0x00.
  • Array: A type byte (0x06), followed by serialized elements, ending with 0x00.
  • Object: A type byte (0x07), followed by key-value pairs (key string terminated by 0x00, followed by the value), and ending with 0x00 for both key and value.
  • Binary: A type byte (0x08), followed by a 4-byte big-endian uint64 length, and the binary data.

Example Encodings

  • A string "hello":
    0x03 + "hello" + 0x00

  • An array [42, true]:
    0x06 + 0x02 + <8-byte float for 42> + 0x05 + 0x00

  • An object {"key": "value"}:
    0x07 + "key" + 0x00 + 0x03 + "value" + 0x00 + 0x00

Documentation

Index

Constants

View Source 
const (
	TypeNull   = Type(1)
	TypeNumber = Type(2)
	TypeString = Type(3)
	TypeFalse  = Type(4)
	TypeTrue   = Type(5)
	TypeArray  = Type(6)
	TypeObject = Type(7)
	TypeBinary = Type(8)
)
View Source 
const EOS = byte(0)

Variables

This section is empty.

Functions

func AppendAnyValue 

func AppendAnyValue(b []byte, v any) []byte

AppendAnyValue appends encoded value to b. Supports only simple go types.

func AppendFloat64 

func AppendFloat64(b []byte, f float64) []byte

AppendFloat64 encodes float64 as bytes that will be correctly comparable with byte.Compare

func AppendNumber 

func AppendNumber[T constraints.Integer | constraints.Float](b []byte, n T) []byte

AppendNumber encodes number

func BytesToFloat64 

func BytesToFloat64(b []byte) float64

BytesToFloat64 decodes float64 from bytes encoded with AppendFloat64

Types

type Arena 

type Arena struct {
	// contains filtered or unexported fields
}

Arena may be used for fast creation and re-use of Values.

Typical Arena lifecycle:

  1. Construct Values via the Arena and Value.Set* calls.
  2. Marshal the constructed Values with Value.MarshalTo call.
  3. Reset all the constructed Values at once by Arena.Reset call.
  4. Go to 1 and re-use the Arena.

It is unsafe calling Arena methods from concurrent goroutines. Use per-goroutine Arenas or ArenaPool instead.

func (*Arena) ApproxSize 

func (a *Arena) ApproxSize() int

ApproxSize returns approximate size of arena cache

func (*Arena) NewArray 

func (a *Arena) NewArray() *Value

NewArray returns new empty array value.

New entries may be added to the returned array via Set* calls.

The returned array is valid until Reset is called on a.

func (*Arena) NewBinary 

func (a *Arena) NewBinary(b []byte) *Value

NewBinary returns new binary value containing b.

The returned value is valid until Reset is called on a.

func (*Arena) NewFalse 

func (a *Arena) NewFalse() *Value

NewFalse return false value.

func (*Arena) NewFromFastJson 

func (a *Arena) NewFromFastJson(jv *fastjson.Value) *Value

NewFromFastJson creates Value from passed fastjson.Value

func (*Arena) NewNull 

func (a *Arena) NewNull() *Value

NewNull returns null value.

func (*Arena) NewNumberFloat64 

func (a *Arena) NewNumberFloat64(f float64) *Value

NewNumberFloat64 returns new number value containing f.

The returned number is valid until Reset is called on a.

func (*Arena) NewNumberInt 

func (a *Arena) NewNumberInt(n int) *Value

NewNumberInt returns new number value containing n.

The returned number is valid until Reset is called on a.

func (*Arena) NewObject 

func (a *Arena) NewObject() *Value

NewObject returns new empty object value.

New entries may be added to the returned object via Set call.

The returned object is valid until Reset is called on a.

func (*Arena) NewString 

func (a *Arena) NewString(s string) *Value

NewString returns new string value containing s.

The returned string is valid until Reset is called on a.

func (*Arena) NewStringBytes 

func (a *Arena) NewStringBytes(s []byte) *Value

NewStringBytes returns new string value containing s.

The returned string is valid until Reset is called on a.

func (*Arena) NewTrue 

func (a *Arena) NewTrue() *Value

NewTrue returns true value.

func (*Arena) Reset 

func (a *Arena) Reset()

Reset resets all the Values allocated by a.

Values previously allocated by a cannot be used after the Reset call.

type ArenaPool 

type ArenaPool struct {
	// contains filtered or unexported fields
}

ArenaPool may be used for pooling Arenas for similarly typed values.

func (*ArenaPool) Get 

func (ap *ArenaPool) Get() *Arena

Get returns an Arena from ap.

The Arena must be Put to ap after use.

func (*ArenaPool) Put 

func (ap *ArenaPool) Put(a *Arena)

Put returns a to ap.

a and objects created by a cannot be used after a is put into ap.

type Object 

type Object struct {
	// contains filtered or unexported fields
}

Object represents an anyenc object

func (*Object) Del 

func (o *Object) Del(key string)

Del deletes value by key

func (*Object) Get 

func (o *Object) Get(key string) *Value

Get gets a value by key

func (*Object) Len 

func (o *Object) Len() int

Len returns object length

func (*Object) Set 

func (o *Object) Set(key string, value *Value)

Set sets value

func (*Object) Visit 

func (o *Object) Visit(visit func(k []byte, v *Value))

Visit calls visit func for every key-value pair

type Parser 

type Parser struct {
	// contains filtered or unexported fields
}

Parser parses encoded bytes.

Parser may be re-used for subsequent parsing.

Parser cannot be used from concurrent goroutines. Use per-goroutine parsers or ParserPool instead.

func (*Parser) ApproxSize 

func (p *Parser) ApproxSize() int

ApproxSize returns approximate size of parser cache

func (*Parser) Parse 

func (p *Parser) Parse(b []byte) (v *Value, err error)

Parse parses encoded bytes

The returned value is valid until the next call to Parse*.

type ParserPool 

type ParserPool struct {
	// contains filtered or unexported fields
}

ParserPool may be used for pooling Parsers for similarly typed values.

func (*ParserPool) Get 

func (pp *ParserPool) Get() *Parser

Get returns a Parser from pp.

The Parser must be Put to pp after use.

func (*ParserPool) Put 

func (pp *ParserPool) Put(p *Parser)

Put returns p to pp.

p and objects recursively returned from p cannot be used after p is put into pp.

type Tuple 

type Tuple []byte

Tuple represents an encoded sequence of values as a byte slice.

func (Tuple) Append 

func (t Tuple) Append(v *Value) Tuple

Append adds a new encoded value to the tuple.

func (Tuple) AppendInverted 

func (t Tuple) AppendInverted(v *Value) Tuple

AppendInverted adds a new encoded value and inverts bytes

func (Tuple) Copy 

func (t Tuple) Copy() Tuple

Copy creates and returns a copy of the tuple.

func (Tuple) ReadBytes 

func (t Tuple) ReadBytes(f func(b []byte) error) (err error)

ReadBytes iterates over every value in the tuple and passes the raw bytes to the provided function `f`. It continues until all values are processed.

func (Tuple) ReadValues 

func (t Tuple) ReadValues(p *Parser, f func(v *Value) error) error

ReadValues decodes and reads all values from the start of the tuple. The provided function `f` is called for each value.

func (Tuple) String 

func (t Tuple) String() string

String returns a string representation of the tuple.

type Type 

type Type uint8

func (Type) String 

func (t Type) String() string

type Value 

type Value struct {
	// contains filtered or unexported fields
}

Value represents one decoded value.

func MustParse 

func MustParse(b []byte) *Value

MustParse parses bytes to Value. Panics in case of error

func MustParseJson 

func MustParseJson(jsonString string) *Value

MustParseJson parses Value from json string. Panics in case of error

func Parse 

func Parse(b []byte) (*Value, error)

Parse parses bytes to Value

func ParseJson 

func ParseJson(jsonString string) (*Value, error)

ParseJson parses Value from json string

func (*Value) AppendBytes 

func (v *Value) AppendBytes(b []byte) ([]byte, error)

AppendBytes appends binary data to the given byte slice from a binary type value.

func (*Value) Array 

func (v *Value) Array() ([]*Value, error)

Array returns the value as an array of values or an error if it's not an array.

func (*Value) Bool 

func (v *Value) Bool() (bool, error)

Bool returns the value as a boolean or an error if it's not a boolean.

func (*Value) Bytes 

func (v *Value) Bytes() ([]byte, error)

Bytes returns the binary data for a binary type or an error if it's not binary. Warning: the returned value is not a copy.

func (*Value) Del 

func (v *Value) Del(key string)

Del deletes a value by key.

func (*Value) FastJson 

func (v *Value) FastJson(a *fastjson.Arena) *fastjson.Value

FastJson converts Value to fastjson.Value using the provided arena.

func (*Value) Float64 

func (v *Value) Float64() (float64, error)

Float64 returns the value as a float64 or an error if it's not a number.

func (*Value) Get 

func (v *Value) Get(keys ...string) *Value

Get returns a value by the given key path.

Array indexes may be represented as decimal numbers in keys.

Returns nil for non-existing key paths.

The returned value is valid until Parse is called on the parser that returned v.

func (*Value) GetArray 

func (v *Value) GetArray(keys ...string) []*Value

GetArray returns an array from the given path or nil if the value is not an array.

func (*Value) GetBool 

func (v *Value) GetBool(keys ...string) bool

GetBool returns true if the value at the path is a boolean true, or false otherwise.

func (*Value) GetBytes 

func (v *Value) GetBytes(keys ...string) []byte

GetBytes returns a byte slice from the given path or nil if the value is not binary.

func (*Value) GetFloat64 

func (v *Value) GetFloat64(keys ...string) float64

GetFloat64 returns a float64 from the given path or 0 if the value is not a number.

func (*Value) GetInt 

func (v *Value) GetInt(keys ...string) int

GetInt returns an int from the given path or 0 if the value is not a number.

func (*Value) GetObject 

func (v *Value) GetObject(keys ...string) *Object

GetObject returns an object from the given path or nil if the value is not an object.

func (*Value) GetString 

func (v *Value) GetString(keys ...string) string

GetString returns a string from the given path or an empty string if the value is not a string.

func (*Value) GetStringBytes 

func (v *Value) GetStringBytes(keys ...string) []byte

GetStringBytes returns a byte slice from the given path or nil if the value is not a string.

func (*Value) GoType 

func (v *Value) GoType() any

GoType converts the value to a Go primitive type.

Numbers are always float64. Arrays are []any. Objects are map[string]any.

func (*Value) Int 

func (v *Value) Int() (int, error)

Int returns the value as an int or an error if it's not a number.

func (*Value) MarshalTo 

func (v *Value) MarshalTo(dst []byte) []byte

MarshalTo appends the value to the given byte slice. You can pass nil as dst.

func (*Value) Object 

func (v *Value) Object() (*Object, error)

Object returns the value as an object or an error if it's not an object.

func (*Value) Set 

func (v *Value) Set(key string, value *Value)

Set sets a (key, value) entry in the array or object v.

func (*Value) SetArrayItem 

func (v *Value) SetArrayItem(idx int, value *Value)

SetArrayItem sets the value in the array v at the specified index.

func (*Value) String 

func (v *Value) String() string

String returns a string (JSON) representation of the value. Use it for debugging purposes only.

func (*Value) StringBytes 

func (v *Value) StringBytes() ([]byte, error)

StringBytes returns the value as a byte slice or an error if it's not a string. Warning: the returned value is not a copy.

func (*Value) Type 

func (v *Value) Type() Type

Type returns the type of the value.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.