gotiny

package module
v0.1.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 22, 2023 License: MIT Imports: 14 Imported by: 5

README

gotiny

Go Report Card CodeCov GoDoc

gotiny is an efficient Go serialization library. By pre-generating encoding machines and reducing the use of the reflect library, gotiny improves efficiency and is almost as fast as serialization libraries that generate code.

examples


 package main

 import (
 "fmt"
 "reflect"

 "github.com/raszia/gotiny"
 )

func main() {

    marshalUnmarshalExample()
    encodeDecodeExample()
    marshalUnmarshalCompressExample()
}

 // marshal src and unmarshel the returned data to dst
 // no compression
func marshalUnmarshalExample() {
    src1, src2 := "marshalUnmarshal", []byte(" Example!")
    dst1, dst2 := "", []byte{3, 4, 5}

    data := gotiny.Marshal(&src1, &src2)
    gotiny.Unmarshal(data, &dst1, &dst2)

    fmt.Println(dst1 + string(dst2)) // print "marshalUnmarshal Example!"
}

 // encode the data using encoder and decode the data using decoder
 // no compression
func encodeDecodeExample() {
    src1, src2 := "encodeDecode", []byte(" Example!")
    dst1, dst2 := "", []byte{3, 4, 5}

    enc := gotiny.NewEncoder(src1, src2)
    dec := gotiny.NewDecoder(dst1, dst2)

    dst1, dst2 = "", []byte{
    3, 3, 3, 3, 3,
    3, 3, 3, 3, 3,
    4, 5, 6, 7, 44,
    7, 5, 6, 4, 7}

    dec.DecodeValue(enc.EncodeValue(reflect.ValueOf(src1),
    reflect.ValueOf(src2)),
    reflect.ValueOf(&dst1).Elem(),
    reflect.ValueOf(&dst2).Elem())

    fmt.Println(dst1 + string(dst2)) // print "encodeDecode Example!"
}

 // marshal src and unmarshel the returned data to dst
 // with compression
func marshalUnmarshalCompressExample() {
    src1, src2 := "marshalUnmarshalCompress", []byte(" Example!")
    dst1, dst2 := "", []byte{3, 4, 5}

    data := gotiny.MarshalCompress(&src1, &src2)
    gotiny.UnmarshalCompress(data, &dst1, &dst2)

    fmt.Println(dst1 + string(dst2)) // print "marshalUnmarshalCompress Example!"
}

Features

  • High efficiency: gotiny is over three times as fast as gob, the serialization library that comes with Golang. It is on par with other serialization frameworks that generate code and is even faster than some of them.
  • Zero memory allocation except for map types.
  • Supports encoding all built-in types and custom types, except func and chan types.
  • Encodes non-exported fields of struct types. Non-encoding fields can be set using Golang tags.
  • Strict type conversion: only types that are exactly the same are correctly encoded and decoded.
  • Encodes nil values with types.
  • Can handle cyclic types but not cyclic values. It will stack overflow.
  • Decodes all types that can be encoded, regardless of the original and target values.
  • Encoded byte strings do not contain type information, resulting in very small byte arrays.
  • Encoded and Decode with compression (optional).

Cannot process cyclic values. Does not support circular references. TODO

type a *a var b a b = &b

install

go get -u github.com/raszia/gotiny

Encoding Protocol

Boolean type

bool type takes up one bit, with the true value encoded as 1 and the false value encoded as 0. When bool type is encountered for the first time, a byte is allocated to encode the value into the least significant bit. When encountered for the second time, it is encoded into the second least significant bit. The ninth time a bool value is encountered, another byte is allocated to encode the value into the least significant bit, and so on.

Integer type
  • uint8 and int8 types are encoded as the next byte of the string.
  • uint16,uint32,uint64,uint,uintptr are encoded using Varints Encoding method.
  • int16,int32,int64,int are converted to unsigned numbers using ZigZag and then encoded using Varints Encoding.
Floating point type

float32 and float64 are encoded using the encoding method for floating point types in gobEncoding method for floating-point types.

Complex number
  • The complex64 type is forced to be converted to a uint64 and encoded using uint64 encoding
  • complex128 type encodes the real and imaginary parts as float64 types.
String type

The string type first encodes the length of the string by casting it to uint64 type and then encoding it. After that, it encodes the byte array of the string as is.

Pointer type

For the pointer type, it checks whether it is nil. If it is nil, it encodes a false value of bool type and then ends. If it is not nil, it encodes a true value of bool type, and then dereferences the pointer and encodes it based on the type of the dereferenced object.

Array and Slice type

It first casts the length to uint64 and encodes it using uint64 encoding. After that, it encodes each element based on its type.

Map type

Similar to the above, it first encodes the length and then encodes each key and its corresponding value. It does this for each key-value pair in the map.

Struct type

It encodes all the members of the struct based on their types, whether they are exported or not. The struct is strictly reconstructed.

Types that implement interfaces
  • For types that implement the BinaryMarshaler/BinaryUnmarshaler interfaces in the encoding package or the GobEncoder/GobDecoder interfaces in the gob package, the encoding and decoding is done using their implementation methods.
  • For types that implement the GoTinySerialize interface in the gotiny.GoTinySerialize package, the encoding and decoding is done using their implementation methods.

benchmark

benchmark

License

MIT

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	Gzip = GzipPool{
		// contains filtered or unexported fields
	}
)

Functions

func GetName 

func GetName(obj any) string

func GetNameByType 

func GetNameByType(rt reflect.Type) string

func Gunziper 

func Gunziper(outPut *bytes.Buffer, input *bytes.Buffer) error

Gunziper unzip input and write to output

func Gziper 

func Gziper(outPut *bytes.Buffer, inPut []byte) error

Gziper compress input and write to output

func Marshal 

func Marshal(is ...any) []byte

Marshal marshal value

func MarshalCompress 

func MarshalCompress(is ...any) []byte

MarshalCompress marshal and compress value

func MarshalCompressEncrypt added in v0.1.1 

func MarshalCompressEncrypt(aesConfig *aesConfigStruct, is ...any) []byte

MarshalCompressEncrypt marshal and compress and encrypt value

func MarshalEncrypt added in v0.1.1 

func MarshalEncrypt(aesConfig *aesConfigStruct, is ...any) []byte

MarshalEncrypt marshal and encrypt value

func NewAES128config added in v0.1.1 

func NewAES128config(key [16]byte) *aesConfigStruct

create AES-128 config for encryption encoding key must be 16 byte

func NewAES192config added in v0.1.1 

func NewAES192config(key [24]byte) *aesConfigStruct

create AES-192 config for encryption encoding key must be 24 byte

func NewAES256config added in v0.1.1 

func NewAES256config(key [32]byte) *aesConfigStruct

create AES-256 config for encryption encoding key must be 32 byte

func Register 

func Register(i any) string

func RegisterName 

func RegisterName(name string, rt reflect.Type)

func Unmarshal 

func Unmarshal(buf []byte, is ...any) int

Unmarshal unmarshal data with no encryption and no compression

func UnmarshalCompress 

func UnmarshalCompress(buf []byte, is ...any) int

UnmarshalCompress unmarshal compressed data

func UnmarshalCompressEncrypt added in v0.1.1 

func UnmarshalCompressEncrypt(aesConfig *aesConfigStruct, buf []byte, is ...any) int

UnmarshalCompressEncrypt unmarshal compressed and encrypted data

func UnmarshalEncrypt added in v0.1.1 

func UnmarshalEncrypt(aesConfig *aesConfigStruct, buf []byte, is ...any) int

UnmarshalEncrypt unmarshal encrypted data

func UnusedUnixNanoEncodeTimeType 

func UnusedUnixNanoEncodeTimeType()

Types

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

func NewDecoder 

func NewDecoder(is ...any) *Decoder

func NewDecoderWithPtr 

func NewDecoderWithPtr(is ...any) *Decoder

func NewDecoderWithType 

func NewDecoderWithType(ts ...reflect.Type) *Decoder

func (*Decoder) Decode 

func (d *Decoder) Decode(buf []byte, is ...any) int

is is pointer of variable

func (*Decoder) DecodeCompress 

func (d *Decoder) DecodeCompress(buf []byte, is ...any) int

func (*Decoder) DecodeCompressEncrypt added in v0.1.1 

func (d *Decoder) DecodeCompressEncrypt(aesConfig *aesConfigStruct, buf []byte, is ...any) int

DecodeCompressEncrypt decompress and decrypt and decode the data

func (*Decoder) DecodeEncrypt added in v0.1.1 

func (d *Decoder) DecodeEncrypt(aesConfig *aesConfigStruct, buf []byte, is ...any) int

DecodeEncrypt decrypt and decode the data

func (*Decoder) DecodePtr 

func (d *Decoder) DecodePtr(buf []byte, ps ...unsafe.Pointer) int

ps is a unsafe.Pointer of the variable

func (*Decoder) DecodeValue 

func (d *Decoder) DecodeValue(buf []byte, vs ...reflect.Value) int

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

func NewEncoder 

func NewEncoder(is ...any) *Encoder

Create an encoder of type Encoder.

func NewEncoderWithPtr 

func NewEncoderWithPtr(ps ...any) *Encoder

Create an encoder that points to the encoding of the given type.

func NewEncoderWithType 

func NewEncoderWithType(ts ...reflect.Type) *Encoder

func (*Encoder) AppendTo 

func (e *Encoder) AppendTo(buf []byte)

The encoded data will be appended to buf.

func (*Encoder) Encode 

func (e *Encoder) Encode(is ...any) []byte

Encode encode value The input is a pointer to the value to be encoded.

func (*Encoder) EncodeCompress 

func (e *Encoder) EncodeCompress(is ...any) []byte

EncodeCompress compress and encode value The input is a pointer to the value to be encoded.

func (*Encoder) EncodeCompressEncrypt added in v0.1.1 

func (e *Encoder) EncodeCompressEncrypt(aesConfig *aesConfigStruct, is ...any) []byte

EncodeCompressEncrypt compress and ecrypt and encode value The input is a pointer to the value to be encoded.

func (*Encoder) EncodeEncrypt added in v0.1.1 

func (e *Encoder) EncodeEncrypt(aesConfig *aesConfigStruct, is ...any) []byte

EncodeEncrypt encrypt and encode value The input is a pointer to the value to be encoded.

func (*Encoder) EncodePtr 

func (e *Encoder) EncodePtr(ps ...unsafe.Pointer) []byte

an unsafe.Pointer to the value to be encoded.

func (*Encoder) EncodeValue 

func (e *Encoder) EncodeValue(vs ...reflect.Value) []byte

vs holds the value to be encoded.

type GoTinySerializer 

type GoTinySerializer interface {
	// Encoding method, appends the object's serialized result to the input parameter and returns it. The method should not modify the original value of the input parameter.
	GotinyEncode([]byte) []byte
	// Decoding method, decodes the input parameter to the object and returns the length used. The method starts using the 0th byte of the input parameter and should not modify any data in the input parameter.
	GotinyDecode([]byte) int
}

This interface should only be implemented by pointers.

type GzipPool 

type GzipPool struct {
	// contains filtered or unexported fields
}

GzipPool manages a pool of gzip.Writer. The pool uses sync.Pool internally.

func (*GzipPool) GetReader 

func (pool *GzipPool) GetReader(src io.Reader) (reader *gzip.Reader)

GetReader returns gzip.Reader from the pool, or creates a new one if the pool is empty.

func (*GzipPool) GetWriter 

func (pool *GzipPool) GetWriter(dst io.Writer) (writer *gzip.Writer)

GetWriter returns gzip.Writer from the pool, or creates a new one with gzip.BestCompression if the pool is empty.

func (*GzipPool) Getbuffer 

func (pool *GzipPool) Getbuffer() *bytes.Buffer

get a buffer from pool

func (*GzipPool) PutReader 

func (pool *GzipPool) PutReader(reader *gzip.Reader)

PutReader closes and returns a gzip.Reader to the pool so that it can be reused via GetReader.

func (*GzipPool) PutWriter 

func (pool *GzipPool) PutWriter(writer *gzip.Writer)

PutWriter closes and returns a gzip.Writer to the pool so that it can be reused via GetWriter.

func (*GzipPool) Putbuffer 

func (pool *GzipPool) Putbuffer(b *bytes.Buffer)

put back a buffer to the pool

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.