fastpb

package module
v0.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 31, 2023 License: Apache-2.0 Imports: 4 Imported by: 119

README

Fastpb

A faster Protobuf serializer & deserializer.

Attention

  • only proto3 is supported now.
  • known-types(any, api, duration...) is not supported now.

Install

go install github.com/cloudwego/fastpb/protoc-gen-fastpb@latest

Usage

Refer to examples, use in two steps:

  1. use fastpb to generate code. (refer here)
  2. use fastpb API to marshal/unmarshal. (refer here)
Step 1: Generate Code

Using the command line tool to generate code:

protoc --go_out=. \
  --fastpb_out=. \
  ${your_idl}.proto

or

protoc --go_opt=paths=source_relative --go_out=. \
  --fastpb_opt=paths=source_relative --fastpb_out=. \
   ${your_idl}.proto
Step 2: Codec Message

Encoding and Decoding must use fastpb's API, shown as demo:

package examples

import (
	"github.com/cloudwego/fastpb"
)

// Marshal .
func Marshal(msg fastpb.Writer) []byte {
	// TODO: buffer can be reused.
	buf := make([]byte, msg.Size())

	msg.FastWrite(buf)
	return buf
}

// Unmarshal .
func Unmarshal(buf []byte, msg fastpb.Reader) error {
	_, err := fastpb.ReadMessage(buf, int8(fastpb.SkipTypeCheck), msg)
	return err
}

Performance

goos: linux
goarch: amd64
pkg: github.com/cloudwego/fastpb/benchmark
cpu: Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz

Benchmarks have compared golang/protobuf (referred to as _golang) and fastpb here.

Marshal
Benchmark_marshal_number_golang-48 375.2 ns/op ~ 96 B/op ~ 1 allocs/op
Benchmark_marshal_number_fastpb-48 145.7 ns/op -61.17% 0 B/op -100.00% 0 allocs/op
Benchmark_marshal_string_golang-48 1010 ns/op ~ 2304 B/op ~ 1 allocs/op
Benchmark_marshal_string_fastpb-48 58.57 ns/op -94.20% 0 B/op -100.00% 0 allocs/op
Benchmark_marshal_list_golang-48 8788 ns/op ~ 18432 B/op ~ 1 allocs/op
Benchmark_marshal_list_fastpb-48 3430 ns/op -60.97% 0 B/op -100.00% 0 allocs/op
Benchmark_marshal_map_golang-48 43497 ns/op ~ 21680 B/op ~ 393 allocs/op
Benchmark_marshal_map_fastpb-48 5951 ns/op -86.32% 0 B/op -100.00% 0 allocs/op
Unmarshal
Benchmark_unmarshal_number_golang-48 497.1 ns/op ~ 144 B/op 1 allocs/op
Benchmark_unmarshal_number_fastpb-48 431.6 ns/op -13.18% 144 B/op 1 allocs/op
Benchmark_unmarshal_string_golang-48 939.7 ns/op ~ 2128 B/op 3 allocs/op
Benchmark_unmarshal_string_fastpb-48 668.4 ns/op -28.87% 2128 B/op 3 allocs/op
Benchmark_unmarshal_list_golang-48 12527 ns/op ~ 20296 B/op 99 allocs/op
Benchmark_unmarshal_list_fastpb-48 12593 ns/op +0.53% 20296 B/op 99 allocs/op
Benchmark_unmarshal_map_golang-48 49868 ns/op ~ 24226 B/op 426 allocs/op
Benchmark_unmarshal_map_fastpb-48 21213 ns/op -57.46% 21467 B/op 61 allocs/op

Documentation

Index

Constants

View Source 
const (
	Name    = "Fastpb"
	Version = "v0.0.2"
	Home    = "https://github.com/cloudwego/fastpb"
)
View Source 
const (
	MapEntry_Key_FieldNumber   = 1  // protoreflect.FieldNumber
	MapEntry_Value_FieldNumber = 2  // protoreflect.FieldNumber
	SkipTagNumber              = -1 // protowire.Number
	SkipTypeCheck              = -1 // protowire.Type
)

Generic field names and numbers for synthetic map entry messages.

Variables

View Source 
var Impl impl

Impl implements Protocol.

Functions

func AppendBytes 

func AppendBytes(b, v []byte) (n int)

AppendBytes appends v to b as a length-prefixed bytes value.

func AppendFixed32 

func AppendFixed32(b []byte, v uint32) int

AppendFixed32 appends v to b as a little-endian uint32.

func AppendFixed64 

func AppendFixed64(b []byte, v uint64) int

AppendFixed64 appends v to b as a little-endian uint64.

func AppendString 

func AppendString(b []byte, v string) (n int)

AppendString appends v to b as a length-prefixed bytes value.

func AppendTag 

func AppendTag(b []byte, num protowire.Number, typ protowire.Type) int

AppendTag encodes num and typ as a varint-encoded tag and appends it to b.

func AppendVarint 

func AppendVarint(buf []byte, v uint64) int

AppendVarint appends v to b as a varint-encoded uint64.

func ConsumeBytes added in v0.0.2 

func ConsumeBytes(b []byte) (v []byte, total int)

ConsumeBytes parses b as a length-prefixed bytes value, reporting its length. This returns a negative length upon an error (see ParseError).

func ConsumeTag added in v0.0.2 

func ConsumeTag(b []byte) (protowire.Number, protowire.Type, int)

ConsumeTag parses b as a varint-encoded tag, reporting its length. This returns a negative length upon an error (see ParseError).

func ConsumeVarint added in v0.0.2 

func ConsumeVarint(b []byte) (v uint64, n int)

ConsumeVarint parses b as a varint-encoded uint64, reporting its length. This returns a negative length upon an error (see ParseError).

func EnforceUTF8 

func EnforceUTF8() bool

EnforceUTF8 todo: use as a switch.

func ReadBool 

func ReadBool(buf []byte, _type int8) (value bool, n int, err error)

ReadBool .

func ReadBytes 

func ReadBytes(buf []byte, _type int8) (value []byte, n int, err error)

ReadBytes .

func ReadDouble 

func ReadDouble(buf []byte, _type int8) (value float64, n int, err error)

ReadDouble .

func ReadFixed32 

func ReadFixed32(buf []byte, _type int8) (value uint32, n int, err error)

ReadFixed32 .

func ReadFixed64 

func ReadFixed64(buf []byte, _type int8) (value uint64, n int, err error)

ReadFixed64 .

func ReadFloat 

func ReadFloat(buf []byte, _type int8) (value float32, n int, err error)

ReadFloat .

func ReadInt32 

func ReadInt32(buf []byte, _type int8) (value int32, n int, err error)

ReadInt32 .

func ReadInt64 

func ReadInt64(buf []byte, _type int8) (value int64, n int, err error)

ReadInt64 .

func ReadList 

func ReadList(buf []byte, _type int8, single Unmarshal) (n int, err error)

ReadList .

func ReadMapEntry 

func ReadMapEntry(buf []byte, _type int8, umk, umv Unmarshal) (int, error)

ReadMapEntry .

func ReadMessage 

func ReadMessage(buf []byte, _type int8, reader Reader) (offset int, err error)

ReadMessage .

func ReadSfixed32 

func ReadSfixed32(buf []byte, _type int8) (value int32, n int, err error)

ReadSfixed32 .

func ReadSfixed64 

func ReadSfixed64(buf []byte, _type int8) (value int64, n int, err error)

ReadSfixed64 .

func ReadSint32 

func ReadSint32(buf []byte, _type int8) (value int32, n int, err error)

ReadSint32 .

func ReadSint64 

func ReadSint64(buf []byte, _type int8) (value int64, n int, err error)

ReadSint64 .

func ReadString 

func ReadString(buf []byte, _type int8) (value string, n int, err error)

ReadString .

func ReadUint32 

func ReadUint32(buf []byte, _type int8) (value uint32, n int, err error)

ReadUint32 .

func ReadUint64 

func ReadUint64(buf []byte, _type int8) (value uint64, n int, err error)

ReadUint64 .

func SizeBool 

func SizeBool(number int32, value bool) (n int)

SizeBool .

func SizeBytes 

func SizeBytes(number int32, value []byte) (n int)

SizeBytes .

func SizeDouble 

func SizeDouble(number int32, value float64) (n int)

SizeDouble .

func SizeFixed32 

func SizeFixed32(number int32, value uint32) (n int)

SizeFixed32 .

func SizeFixed64 

func SizeFixed64(number int32, value uint64) (n int)

SizeFixed64 .

func SizeFloat 

func SizeFloat(number int32, value float32) (n int)

SizeFloat .

func SizeInt32 

func SizeInt32(number, value int32) (n int)

SizeInt32 .

func SizeInt64 

func SizeInt64(number int32, value int64) (n int)

SizeInt64 .

func SizeListPacked 

func SizeListPacked(number int32, length int, single EntrySize) (n int)

SizeListPacked .

func SizeMapEntry 

func SizeMapEntry(number int32, entry EntrySize) (n int)

SizeMapEntry .

func SizeMessage 

func SizeMessage(number int32, sizer Sizer) (n int)

SizeMessage .

func SizeSfixed32 

func SizeSfixed32(number, value int32) (n int)

SizeSfixed32 .

func SizeSfixed64 

func SizeSfixed64(number int32, value int64) (n int)

SizeSfixed64 .

func SizeSint32 

func SizeSint32(number, value int32) (n int)

SizeSint32 .

func SizeSint64 

func SizeSint64(number int32, value int64) (n int)

SizeSint64 .

func SizeString 

func SizeString(number int32, value string) (n int)

SizeString .

func SizeUint32 

func SizeUint32(number int32, value uint32) (n int)

SizeUint32 .

func SizeUint64 

func SizeUint64(number int32, value uint64) (n int)

SizeUint64 .

func Skip 

func Skip(buf []byte, _type int8, number int32) (n int, err error)

Skip .

func WriteBool 

func WriteBool(buf []byte, number int32, value bool) (n int)

WriteBool .

func WriteBytes 

func WriteBytes(buf []byte, number int32, value []byte) (n int)

WriteBytes .

func WriteDouble 

func WriteDouble(buf []byte, number int32, value float64) (n int)

WriteDouble .

func WriteFixed32 

func WriteFixed32(buf []byte, number int32, value uint32) (n int)

WriteFixed32 .

func WriteFixed64 

func WriteFixed64(buf []byte, number int32, value uint64) (n int)

WriteFixed64 .

func WriteFloat 

func WriteFloat(buf []byte, number int32, value float32) (n int)

WriteFloat .

func WriteInt32 

func WriteInt32(buf []byte, number, value int32) (n int)

WriteInt32 .

func WriteInt64 

func WriteInt64(buf []byte, number int32, value int64) (n int)

WriteInt64 .

func WriteListPacked 

func WriteListPacked(buf []byte, number int32, length int, single Marshal) (n int)

WriteListPacked .

func WriteMapEntry 

func WriteMapEntry(buf []byte, number int32, entry Marshal) (n int)

WriteMapEntry .

func WriteMessage 

func WriteMessage(buf []byte, number int32, writer Writer) (n int)

WriteMessage .

func WriteSfixed32 

func WriteSfixed32(buf []byte, number, value int32) (n int)

WriteSfixed32 .

func WriteSfixed64 

func WriteSfixed64(buf []byte, number int32, value int64) (n int)

WriteSfixed64 .

func WriteSint32 

func WriteSint32(buf []byte, number, value int32) (n int)

WriteSint32 .

func WriteSint64 

func WriteSint64(buf []byte, number int32, value int64) (n int)

WriteSint64 .

func WriteString 

func WriteString(buf []byte, number int32, value string) (n int)

WriteString .

func WriteUint32 

func WriteUint32(buf []byte, number int32, value uint32) (n int)

WriteUint32 .

func WriteUint64 

func WriteUint64(buf []byte, number int32, value uint64) (n int)

WriteUint64 .

Types

type EntrySize 

type EntrySize func(numTagOrKey, numIdxOrVal int32) int

EntrySize .

type Marshal 

type Marshal func(buf []byte, numTagOrKey, numIdxOrVal int32) int

Marshal .

type Protocol 

type Protocol interface {
	WriteMessage(buf []byte, number int32, writer Writer) (n int)
	WriteListPacked(buf []byte, number int32, length int, single Marshal) (n int)
	WriteMapEntry(buf []byte, number int32, entry Marshal) (n int)
	WriteBool(buf []byte, number int32, value bool) (n int)
	WriteInt32(buf []byte, number, value int32) (n int)
	WriteInt64(buf []byte, number int32, value int64) (n int)
	WriteUint32(buf []byte, number int32, value uint32) (n int)
	WriteUint64(buf []byte, number int32, value uint64) (n int)
	WriteSint32(buf []byte, number, value int32) (n int)
	WriteSint64(buf []byte, number int32, value int64) (n int)
	WriteFloat(buf []byte, number int32, value float32) (n int)
	WriteDouble(buf []byte, number int32, value float64) (n int)
	WriteFixed32(buf []byte, number int32, value uint32) (n int)
	WriteFixed64(buf []byte, number int32, value uint64) (n int)
	WriteSfixed32(buf []byte, number, value int32) (n int)
	WriteSfixed64(buf []byte, number int32, value int64) (n int)
	WriteString(buf []byte, number int32, value string) (n int)
	WriteBytes(buf []byte, number int32, value []byte) (n int)
	ReadMessage(buf []byte, _type int8, reader Reader) (offset int, err error)
	ReadList(buf []byte, _type int8, single Unmarshal) (n int, err error)
	ReadMapEntry(buf []byte, _type int8, umk, umv Unmarshal) (int, error)
	ReadBool(buf []byte, _type int8) (value bool, n int, err error)
	ReadInt32(buf []byte, _type int8) (value int32, n int, err error)
	ReadInt64(buf []byte, _type int8) (value int64, n int, err error)
	ReadUint32(buf []byte, _type int8) (value uint32, n int, err error)
	ReadUint64(buf []byte, _type int8) (value uint64, n int, err error)
	ReadSint32(buf []byte, _type int8) (value int32, n int, err error)
	ReadSint64(buf []byte, _type int8) (value int64, n int, err error)
	ReadFloat(buf []byte, _type int8) (value float32, n int, err error)
	ReadDouble(buf []byte, _type int8) (value float64, n int, err error)
	ReadFixed32(buf []byte, _type int8) (value uint32, n int, err error)
	ReadFixed64(buf []byte, _type int8) (value uint64, n int, err error)
	ReadSfixed32(buf []byte, _type int8) (value int32, n int, err error)
	ReadSfixed64(buf []byte, _type int8) (value int64, n int, err error)
	ReadString(buf []byte, _type int8) (value string, n int, err error)
	ReadBytes(buf []byte, _type int8) (value []byte, n int, err error)
	SizeBool(number int32, value bool) (n int)
	SizeInt32(number, value int32) (n int)
	SizeInt64(number int32, value int64) (n int)
	SizeUint32(number int32, value uint32) (n int)
	SizeUint64(number int32, value uint64) (n int)
	SizeSint32(number, value int32) (n int)
	SizeSint64(number int32, value int64) (n int)
	SizeFloat(number int32, value float32) (n int)
	SizeDouble(number int32, value float64) (n int)
	SizeFixed32(number int32, value uint32) (n int)
	SizeFixed64(number int32, value uint64) (n int)
	SizeSfixed32(number, value int32) (n int)
	SizeSfixed64(number int32, value int64) (n int)
	SizeString(number int32, value string) (n int)
	SizeBytes(number int32, value []byte) (n int)
	SizeMessage(number int32, sizer Sizer) (n int)
	SizeMapEntry(number int32, entry EntrySize) (n int)
	SizeListPacked(number int32, length int, single EntrySize) (n int)
	Skip(buf []byte, _type int8, number int32) (n int, err error)
}

Protocol .

type Reader 

type Reader interface {
	FastRead(buf []byte, _type int8, number int32) (n int, err error)
}

Reader is designed for generating code.

type Sizer 

type Sizer interface {
	Size() (n int)
}

Sizer is designed for generating code.

type Unmarshal 

type Unmarshal func(buf []byte, _type int8) (n int, err error)

Unmarshal .

type Writer 

type Writer interface {
	Sizer
	FastWrite(buf []byte) (n int)
}

Writer is designed for generating code.

Source Files

View all Source files

Directories

Path Synopsis
fastpb_gen
gogo_gen
fastpb_gen
fastpb_gen/nested
fastpb_gen/pet
fastpb_gen/user
generator

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.