frugal

README

Frugal

English | 中文

A very fast dynamic Thrift serializer & deserializer based on just-in-time compilation.

Features

Code Generation Free

Traditional Thrift serializer and deserializer are based on generated code which is no longer needed since we can use JIT compilation to dynamically generate machine code.

High Performance

Thanks to JIT compilation, Frugal can generate better machine code than Go language compiler. In multi-core scenarios, Frugal's performance is about 5 times higher than that of traditional serializer and deserializer.

name                                 old time/op    new time/op     delta
MarshalAllSize_Parallel/small-16       78.8ns ± 0%     14.9ns ± 0%    -81.10%
MarshalAllSize_Parallel/medium-16      1.34µs ± 0%     0.32µs ± 0%    -76.32%
MarshalAllSize_Parallel/large-16       37.7µs ± 0%      9.4µs ± 0%    -75.02%
UnmarshalAllSize_Parallel/small-16      368ns ± 0%       30ns ± 0%    -91.90%
UnmarshalAllSize_Parallel/medium-16    11.9µs ± 0%      0.8µs ± 0%    -92.98%
UnmarshalAllSize_Parallel/large-16      233µs ± 0%       21µs ± 0%    -90.99%

name                                 old speed      new speed       delta
MarshalAllSize_Parallel/small-16     7.31GB/s ± 0%  38.65GB/s ± 0%   +428.84%
MarshalAllSize_Parallel/medium-16    12.9GB/s ± 0%   54.7GB/s ± 0%   +322.10%
MarshalAllSize_Parallel/large-16     11.7GB/s ± 0%   46.8GB/s ± 0%   +300.26%
UnmarshalAllSize_Parallel/small-16   1.56GB/s ± 0%  19.31GB/s ± 0%  +1134.41%
UnmarshalAllSize_Parallel/medium-16  1.46GB/s ± 0%  20.80GB/s ± 0%  +1324.55%
UnmarshalAllSize_Parallel/large-16   1.89GB/s ± 0%  20.98GB/s ± 0%  +1009.73%

name                                 old alloc/op   new alloc/op    delta
MarshalAllSize_Parallel/small-16         112B ± 0%         0B        -100.00%
MarshalAllSize_Parallel/medium-16        112B ± 0%         0B        -100.00%
MarshalAllSize_Parallel/large-16         779B ± 0%        57B ± 0%    -92.68%
UnmarshalAllSize_Parallel/small-16     1.31kB ± 0%     0.10kB ± 0%    -92.76%
UnmarshalAllSize_Parallel/medium-16      448B ± 0%      3022B ± 0%   +574.55%
UnmarshalAllSize_Parallel/large-16     1.13MB ± 0%     0.07MB ± 0%    -93.54%

name                                 old allocs/op  new allocs/op   delta
MarshalAllSize_Parallel/small-16         1.00 ± 0%       0.00        -100.00%
MarshalAllSize_Parallel/medium-16        1.00 ± 0%       0.00        -100.00%
MarshalAllSize_Parallel/large-16         1.00 ± 0%       0.00        -100.00%
UnmarshalAllSize_Parallel/small-16       6.00 ± 0%       1.00 ± 0%    -83.33%
UnmarshalAllSize_Parallel/medium-16      6.00 ± 0%      30.00 ± 0%   +400.00%
UnmarshalAllSize_Parallel/large-16      4.80k ± 0%      0.76k ± 0%    -84.10%

What can you do with Frugal ?

Use Frugal as Kitex serializer and deserializer

No more massive serialization and deserialization code, leads to a more tidy project. No more meaningless diff of generated code in code review.

Serialized and Deserialize struct generated by Thriftgo

If you have a Thrift file, and all you need is using Frugal to do serialization and deserialization. You can use thriftgo to generate Go struct, then you can use Frugal.

Serialization and deserialization on a customized Go struct

If you don't want any Thrift files, and you want serialize or deserialize a customized Go struct. You can add some struct field tag to the Go struct, then you can use Frugal.

Usage

Using with Kitex

1. Update Kitex to v0.4.2 or higher version

go get github.com/cloudwego/kitex@latest

2. Generate code with -thrift frugal_tag option

Example:

kitex -thrift frugal_tag -service a.b.c my.thrift

If you don't need codec code, you can use -thrift template=slim option.

kitex -thrift frugal_tag,template=slim -service a.b.c my.thrift

3. Init clients and servers with WithPayloadCodec(thrift.NewThriftFrugalCodec()) option

Client example:

package client

import (
    "context"

    "example.com/kitex_test/client/kitex_gen/a/b/c/echo"
    "github.com/cloudwego/kitex/client"
    "github.com/cloudwego/kitex/pkg/remote/codec/thrift"
)

func Echo() {
    code := thrift.NewThriftCodecWithConfig(thrift.FastRead | thrift.FastWrite | thrift.FrugalRead | thrift.FrugalWrite)
    cli := echo.MustNewClient("a.b.c", client.WithPayloadCodec(codec))
    ...
}

Server example:

package main

import (
    "log"

    "github.com/cloudwego/kitex/server"
    c "example.com/kitex_test/kitex_gen/a/b/c/echo"
    "github.com/cloudwego/kitex/pkg/remote/codec/thrift"
)

func main() {
    code := thrift.NewThriftCodecWithConfig(thrift.FastRead | thrift.FastWrite | thrift.FrugalRead | thrift.FrugalWrite)
    svr := c.NewServer(new(EchoImpl), server.WithPayloadCodec(code))

    err := svr.Run()
    if err != nil {
        log.Println(err.Error())
    }
}

Using with Thrift IDL

Prepare Thrift file

We can define a struct in Thrift file like below:

my.thrift:

struct MyStruct {
    1: string msg
    2: i64 code
}

Use Thriftgo to generate code

Now we have thrift file, we can use Thriftgo with frugal_tag option to generate Go code.

Example:

thriftgo -r -o thrift -g go:frugal_tag,package_prefix=example.com/kitex_test/thrift my.thrift

If you don't need codec code, you can use template=slim option

thriftgo -r -o thrift -g go:frugal_tag,template=slim,package_prefix=example.com/kitex_test/thrift my.thrift

Use Frugal to serialize or deserialize

Now we can use Frugal to serialize or deserialize the struct defined in thrift file.

Example:

package main

import (
    "github.com/cloudwego/frugal"

    "example.com/kitex_test/thrift"
)

func main() {
    ms := &thrift.MyStruct{
        Msg: "my message",
        Code: 1024,
    }
    ...
    buf := make([]byte, frugal.EncodedSize(ms))
    frugal.EncodeObject(buf, nil, ms)
    ...
    got := &thrift.MyStruct{}
    frugal.DecodeObject(buf, got)
    ...
}

Serialization and deserialization on a customized Go struct

Define a Go struct

We can define a struct like this:

type MyStruct struct {
    Msg     string
    Code    int64
    Numbers []int64 
}

Add Frugal tag to struct fields

Frugal tag is like frugal:"1,default,string", 1 is field ID, default is field requiredness, string is field type. Field ID and requiredness is always required, but field type is only required for list, set and enum.

You can add Frugal tag to MyStruct like below:

type MyStruct struct {
    Msg     string  `frugal:"1,default"`
    Code    int64   `frugal:"2,default"`
    Numbers []int64 `frugal:"3,default,list<i64>"`
}

All types example:

type MyEnum int64

type Example struct {
 MyOptBool         *bool            `frugal:"1,optional"`
 MyReqBool         bool             `frugal:"2,required"`
 MyOptByte         *int8            `frugal:"3,optional"`
 MyReqByte         int8             `frugal:"4,required"`
 MyOptI16          *int16           `frugal:"5,optional"`
 MyReqI16          int16            `frugal:"6,required"`
 MyOptI32          *int32           `frugal:"7,optional"`
 MyReqI32          int32            `frugal:"8,required"`
 MyOptI64          *int64           `frugal:"9,optional"`
 MyReqI64          int64            `frugal:"10,required"`
 MyOptString       *string          `frugal:"11,optional"`
 MyReqString       string           `frugal:"12,required"`
 MyOptBinary       []byte           `frugal:"13,optional"`
 MyReqBinary       []byte           `frugal:"14,required"`
 MyOptI64Set       []int64          `frugal:"15,optional,set<i64>"`
 MyReqI64Set       []int64          `frugal:"16,required,set<i64>"`
 MyOptI64List      []int64          `frugal:"17,optional,list<i64>"`
 MyReqI64List      []int64          `frugal:"18,required,list<i64>"`
 MyOptI64StringMap map[int64]string `frugal:"19,optional"`
 MyReqI64StringMap map[int64]string `frugal:"20,required"`
 MyOptEnum         *MyEnum          `frugal:"21,optional,i64"`
 MyReqEnum         *MyEnum          `frugal:"22,optional,i64"`
}

Use Frugal to serialize or deserialize

Example:

package main

import (
    "github.com/cloudwego/frugal"
)

func main() {
    ms := &thrift.MyStruct{
        Msg: "my message",
        Code: 1024,
        Numbers: []int64{0, 1, 2, 3, 4},
    }
    ...
    buf := make([]byte, frugal.EncodedSize(ms))
    frugal.EncodeObject(buf, nil, ms)
    ...
    got := &thrift.MyStruct{}
    frugal.DecodeObject(buf, got)
    ...
}

Documentation

Index

• func DecodeObject(buf []byte, val interface{}) (int, error)
• func EncodeObject(buf []byte, mem iov.BufferWriter, val interface{}) (int, error)
• func EncodedSize(val interface{}) int
• func Pretouch(vt reflect.Type, options ...Option) error
• func SetMaxInlineDepth(depth int) int
• func SetMaxInlineILSize(size int) int
• type Option
  • func WithMaxInlineDepth(depth int) Option
  • func WithMaxInlineILSize(size int) Option
  • func WithMaxPretouchDepth(depth int) Option

Functions

func DecodeObject

func DecodeObject(buf []byte, val interface{}) (int, error)

DecodeObject deserializes buf into val with Thrift Binary Protocol.

func EncodeObject

func EncodeObject(buf []byte, mem iov.BufferWriter, val interface{}) (int, error)

EncodeObject serializes val into buf with Thrift Binary Protocol, with optional Zero-Copy iov.BufferWriter. buf must be large enough to contain the entire serialization result.

func EncodedSize

func EncodedSize(val interface{}) int

EncodedSize measures the encoded size of val.

func Pretouch

func Pretouch(vt reflect.Type, options ...Option) error

Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in order to reduce the first-hit latency.

func SetMaxInlineDepth

func SetMaxInlineDepth(depth int) int

SetMaxInlineDepth sets the default maximum inlining depth for all types from now on.

This value can also be configured with the `FRUGAL_MAX_INLINE_DEPTH` environment variable.

The default value of this option is "5".

Returns the old opts.MaxInlineDepth value.

func SetMaxInlineILSize

func SetMaxInlineILSize(size int) int

SetMaxInlineILSize sets the default maximum inlining IL instructions for all types from now on.

This value can also be configured with the `FRUGAL_MAX_INLINE_IL_SIZE` environment variable.

The default value of this option is "50000".

Returns the old opts.MaxInlineILSize value.

Types

type Option

type Option func(*opts.Options)

Option is the property setter function for opts.Options.

func WithMaxInlineDepth

func WithMaxInlineDepth(depth int) Option

WithMaxInlineDepth sets the maximum inlining depth for the JIT compiler.

Increasing of this option makes the compiler inline more aggressively, which gives better runtime performance at the cost of a longer compilation time, and vice versa.

Set this option to "0" disables this limit, which means inlining everything.

The default value of this option is "5".

func WithMaxInlineILSize

func WithMaxInlineILSize(size int) Option

WithMaxInlineILSize sets the maximum IL instruction count before not inlining.

Increasing of this option makes the compiler inline more aggressively, which lead to more memory consumptions but better runtime performance, and vice versa.

Set this option to "0" disables this limit, which means unlimited inlining IL buffer.

The default value of this option is "50000".

func WithMaxPretouchDepth

func WithMaxPretouchDepth(depth int) Option

WithMaxPretouchDepth controls how deep the compiler goes to compile indirectly referenced types.

Larger depth means more types will be pre-compiled when pretouching, which lead to longer compilation time, but lower runtime JIT latency, and vice versa. You might want to tune this value to strike a balance between compilation time and runtime performance.

The default value "0" means unlimited, which basically turns Frugal into an AOT compiler.

This option is only available when performing pretouch, otherwise it is ignored and do not have any effect.