csproto

package module
v0.29.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 29, 2024 License: MIT Imports: 20 Imported by: 1

README

csproto - CrowdStrike's Protocol Buffers library

GoDoc

csproto is a Go module that provides a library for working with Protocol Buffers messages along with a protoc plug-in for generating optimized marshaling and unmarshaling code for those messages.

Like many other companies, CrowdStrike extensively uses Protocol Buffers as an efficient wire format for communicating between disparate processes and services. Protocol Buffers' compatibility guarantees and smaller, more efficient binary encoding made it a natural fit for the problems we needed to solve.

As our data volume continued to grow, CrowdStrike started to run into performance limitations in Google's Protobuf library and transitioned to using Gogo Protobuf in an attempt to overcome the issues. This adjustment proved successful and "Use Gogo Protobuf" became the de facto guidance within our development teams.

Fast forward to 2020 when the maintainers of that library announced that they are looking for people to take over. Unfortunately, this also coincided with Google releasing V2 of their Protobuf API. As new and/or improved functionality was introduced in Google's library, the lack of active maintenance on Gogo inevitably led to incompatibilities.

This created a problem for CrowdStrike. We needed to update our system to no longer depend on Gogo Protobuf but we had a lot of direct dependencies on that code spread throughout our codebase. The solution we arrived at is this library. It provides the "core" pieces of the Protocol Buffers API as used by consumers without having those consumers directly depend on any particular runtime implementation.

If you want to dive right in, there's a handy migration guide. Otherwise, keep reading for all of the technical details.

Disclaimer: csproto is an open source project, not a CrowdStrike product. As such, it carries no formal support, expressed or implied. The project is licensed under the MIT open source license.

Supporting Types Across Runtime Implementations

As part of their V2 API, Google also introduced significant changes to the protoc code generation plug-in, protoc-gen-go. One effect of this change was that code generated using the new plug-in uses the new API internally. An unfortunate side effect is that those types are no longer compatible with Gogo's API.

One technical limitation of Protocol Buffers is that deserializing a message requires knowledge of the actual message type because the encoded field values only contain the integer field tag. Due to this limitation, both Google and Gogo use reflection to read the struct tags on the generated Go types and to dynamically assign field values when unmarshaling Protobuf messages.

This dependence on reflection has created a scenario where passing a type generated by the new plug-in to Gogo's implementation of csproto.Unmarshal() results in failures. Specifically, there are several new fields in the generated code and the reflection-based logic in Gogo's library doesn't know how to treat them. Additionally, several fields that are used by the V1 API, and consequently Gogo's library, are no longer generated.

A Minimal Protocol Buffers API

After a bit of digging, we came up with what we consider the smallest API necessary to support reading and writing Protocol Buffers messages that does not expose any dependence on the runtime implementations.

  • Size(msg interface{}) int
    • Calculate the size, in bytes, required to hold the binary representation of msg
  • Marshal(msg interface{}) ([]byte, error)
    • Convert the contents of msg to the binary representation and return it
  • Unmarshal(p []byte, msg interface{}) error
    • Populate msg with the contents of the binary message in p
  • HasExtension(msg interface{}, ext interface{}) bool
    • Determine if msg contains a proto2 extension field
  • ClearExtension(msg interface{}, ext interface{})
    • Clears a proto2 extension field from msg
  • GetExtension(msg interface{}, ext interface{}) (interface{}, error)
    • Return the value of a proto2 extension field from msg
  • SetExtension(msg interface{}, ext interface{}, val interface{}) error
    • Assign the value of a proto2 extension field on msg

There isn't any common interface shared between Google's two runtimes and Gogo's runtime so our library had to use the empty interface for all message and extension definition parameters.

With this minimal API, services and libraries are able to create and consume Protobuf-encoded messages without being tightly coupled to any specific runtime. Being able to do this was essential for CrowdStrike because it is simply impossible to update everything at once to change which runtime library is in use. Instead, we gradually updated all of our libraries and services to use this new runtime-independent API so that each of our development teams is able to change out their runtime and code generation dependencies independently.

Don't Recreate Everything

Our intent is not to fully recreate the Protocol Buffers runtime. Instead, csproto is built to determine which existing runtime is the "correct" one for a given message and to delegate to that implementation.

We take advantage of the fact that the Go types can't change at runtime to minimize the impact of this indirection. The underlying type of the msg parameter is inspected to determine which of the 3 supported runtimes (Google V1, Gogo, and Google V2) is correct and we store that value in a lookup dictionary so that any given type only has to be inspected once.

Even with this optimization, calling reflect.TypeOf() on the message and performing the lookup has a cost, over 8% in some scenarios! At CrowdStrike's volume even that difference can add up to a non-trivial impact across the system so we needed to find a way to at least break even but, ideally, to better the performance.

The benchmarks below for proto2 marshaling were generated on a 2019 MacBook Pro:

$ go test -run='^$' -bench=. -benchmem
goos: darwin
goarch: amd64
pkg: github.com/CrowdStrike/csproto/example/proto2
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkEncodeGogo-12                   1741834               667.4 ns/op           216 B/op          4 allocs/op
BenchmarkCustomEncodeGogo-12             1785268               669.2 ns/op           216 B/op          4 allocs/op
BenchmarkEncodeGoogleV1-12               1326734               921.7 ns/op           176 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV1-12         1315390               933.5 ns/op           176 B/op          1 allocs/op
BenchmarkEncodeGoogleV2-12               1329092               906.9 ns/op           176 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV2-12         1306638               923.3 ns/op           176 B/op          1 allocs/op

And for proto3:

$ go test -run='^$' -bench=. -benchmem
goos: darwin
goarch: amd64
pkg: github.com/CrowdStrike/csproto/example/proto3
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkEncodeGogo-12                   3008721               394.1 ns/op            88 B/op          2 allocs/op
BenchmarkCustomEncodeGogo-12             2900726               400.1 ns/op            88 B/op          2 allocs/op
BenchmarkEncodeGoogleV1-12               3109386               388.2 ns/op            80 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV1-12         2990907               392.8 ns/op            80 B/op          1 allocs/op
BenchmarkEncodeGoogleV2-12               3290887               367.7 ns/op            80 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV2-12         3003828               398.3 ns/op            80 B/op          1 allocs/op

The table below shows the approximate cost of the indirection across the various combinations of Protobuf runtimes:

Cost proto2 proto3
Gogo +0.27% +1.52%
Google V1 +1.28% +1.18%
Google V2 +1.81% +8.32%

Optimized Protobuf Marshaling and Unmarshaling

The marshaling and unmarshaling implemented by both Google and Gogo necessarily relies on runtime reflection. Both implementations dynamically query the set of fields on the message type and read the associated Protobuf struct tags. This information is then used to match up the field tag and wire type in the encoded data to the corresponding field on the message and to assign the field values. This solution is generic and can be applied to any/all messages without any changes to the implementation but it is necessary slower because it has to inspect each message much more deeply.

Another common source of performance bottlenecks is repeated small allocations. It is, in most cases, far more efficient to allocate one buffer large enough to hold all of the data you need than to incrementally allocate many smaller buffers.

Before moving on, credit must be given to the Vitess team for their vtprotobuf project which they covered in this blog from June of 2021. That project already implements these strategies and more, only with some constraints that didn't work for us. Specifically, vtprotobuf is only compatible with things that are already using Google's V2 API. Given that the inception of this project for CrowdStrike was due to our dependency on Gogo Protobuf we weren't able to make use of their work. We also make significant usage of proto2 extensions, which may or may not be supported by the Vitess tooling.

Protocol Buffers Binary Codec

The first step to improving Protobuf serialization is to implement a binary encoder and decoder that avoids the issues noted in the last section. Additionally, the Protocol Buffer encoding spec has a much smaller surface area than the set of all valid Protobuf messages.

Encoder

The Encoder type wraps a pre-allocated byte slice and sequentially writes encoded field values to it. It is up to the caller to ensure that the provided buffer is large enough to hold the full encoded value. As each encoded field is prefixed by the integer field tag and Protobuf wire type, Encoder's API is provided as a set of EncodeXxx(tag int, val T) methods, one for each supported type of value.

This snippet encodes a boolean true value with a field tag of 1:

// Protobuf binary encoding will require 2 bytes, 1 for the tag/wire type and 1 for the value
buf := make([]byte, 2)
enc := csproto.NewEncoder(buf)
enc.EncodeBool(1, true)
// buf now contains {0x8, 0x1}

Encoding a full message is similar, but using csproto.Size() to calculate the required buffer size.

msg := SomeMessage{
    Name: csproto.String("example"),
    Value: csproto.Int32(42),
    // assign additional fields
}
siz := csproto.Size(msg)
buf := make([]byte, siz)
enc := csproto.NewEncoder(buf)
// encode each field sequentially
enc.EncodeString(1, msg.Name)
enc.EncodeInt32(2, msg.Value)
// ...
Decoder

Like Encoder, the Decoder type wraps a byte slice and sequentially reads field values from it. The Protobuf encoding does not require fields to be in tag order, or present at all for that matter, so decoding a message requires a for loop combined with a switch statement.

func decodeExample(p []byte) (SomeMessage, error) {
    var (
        msg SomeMessage
        s string
        i32 int32
    )
    dec := csproto.NewDecoder(p)
    for dec.More() {
        tag, wireType, err := dec.DecodeTag()
        if err != nil {
            return SomeMessage{}, err
        }
        switch tag {
        case 1: // Name
            if wireType != csproto.WireTypeLengthDelimited {
                return SomeMessage{}, fmt.Errorf("invalid wire type %s, expected %s", wireType, csproto.WireTypeLengthDelimited)
            }
            s, err = dec.DecodeString()
            if err != nil {
                return SomeMessage{}, fmt.Errorf("unable to decode string: %w", err)
            }
            msg.Name = csproto.String(s)
        case 2: // Value
            if wireType != csproto.WireTypeVarint {
                return SomeMessage{}, fmt.Errorf("invalid wire type %s, expected %s", wireType, csproto.WireTypeVarint)
            }
            i32, err = dec.DecodeInt32()
            if err != nil {
                return SomeMessage{}, fmt.Errorf("unable to decode int32: %w", err)
            }
            msg.Value = csproto.Int32(i32)
        default: // unknown/unrecognized field, skip it
            _, _ = dec.Skip(tag, wireType)
        }
    }
}

Notes:

  • The cases in the switch statement use csproto.String() and csproto.Int32() to grab pointers to copies of the decoded values.
  • The example above simply throws away unknown fields which you shouldn't do in practice.
Safe vs Fast

By default, Decoder.DecodeString() will make a full copy of the decoded string. This is the safest, most stable practice but it does come with a small cost in both time and allocations. For scenarios where maximum performance is more desirable, Decoder supports a "fast" mode that uses unsafe to return the bytes of wrapped buffer directly, saving the type conversion and allocation to create a new string value.

...
dec := csproto.NewDecoder(p)
dec.SetMode(proto.DecoderModeFast)
...
s, err := dec.DecodeString()
...

Representative benchmarks from a 2019 MacBook Pro

...> go test -run='^$' -bench=DecodeString -benchmem ./proto
goos: darwin
goarch: amd64
pkg: github.com/CrowdStrike/csproto
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkSafeDecodeString-12            37183212                27.33 ns/op           16 B/op          1 allocs/op
BenchmarkFastDecodeString-12            127437440                9.211 ns/op           0 B/op          0 allocs/op

The trade off for the increased performance is that the behavior is undefined if the wrapped buffer is modified after decoding the field values from it.

Opting In

Now that we have a custom, optimized codec available, we need a way to seamlessly integrate it into the developer workflow. We do that by defining several new interfaces which our API functions will look for when marshaling or unmarshaling messages.

We define 4 single-method interfaces as integration points:

  • csproto.Sizer
    • Size() int: calculates the size, in bytes, needed to hold the encoded contents of the message
      • csproto.Size() will call this method if the message satisfies the interface
  • csproto.Marshaler
    • Marshal() ([]byte, error): returns the binary encoding of the message
      • csproto.Marshal() will call this method if the message satisfies the interface
  • csproto.MarshalerTo
    • MarshalTo([]byte) error: encodes the message into the provided buffer
      • csproto.Marshal() will call this method, after allocating a sufficiently sized buffer, if the messaage satisfies the interface
  • csproto.Unmarshaler
    • Unmarshal([]byte) error: decodes the provided data into the message
      • csproto.Unmarshal() will call this method if the message satisfies the interface

With this in place developers have all of the parts needed to create a fully optimized implementation of Protocol Buffer marshaling and unmarshaling. We can make things even better, though, by capitalizing on the fact that the Protobuf IDL that developers have already written has all of the information we need to generate those optimized implementations.

The protoc plug-in

The final piece of the puzzle is protoc-gen-fastmarshal, a protoc compiler plug-in that reads the Protobuf file descriptor and emits implementations of the Size, Marshal, MarshalTo, and Unmarshal methods for each message defined in the .proto file.

Given this example message

message Example {
  string name   = 1;
  int32  result = 2;
}

the generated code would be roughly as follows

// Size returns the size, in bytes, required to store the contents of m in Protocol Buffers
// binary format.
func (m *Example) Size() int {
  if m == nil {
    return 0
  }
  var (
    sz, l int
  )
  // Name
  if m.Name != nil {
    // key + len + bytes
    l = len(*m.Name)
    sz += csproto.SizeOfVarint(uint64(1)) + csproto.SizeOfVarint(uint64(l)) + l
  }
  // Result
  if m.Result != nil {
    // key + varint
    sz += csproto.SizeOfVarint(uint64(2)) + csproto.SizeOfVarint(uint64(*m.Result))
  }
  // unknown/unrecognized fields
  sz += len(m.unknownFields)
  return sz
}
// Marshal allocates a buffer, writes the contents of m to it using Protocol Buffers binary
// format, then returns the the buffer.
func (m *Example) Marshal() ([]byte, error) {
  sz := m.Size()
  buf := make([]byte, sz)
  err := m.MarshalTo(buf)
  return buf, err
}
// MarshalTo writes the contents of m into dest using Protocol Buffers binary format.
func (m *Example) MarshalTo(dest []byte) error {
  var (
    buf []byte
    err error
  )
  enc := csproto.NewEncoder(dest)
  if m.Name != nil {
    enc.EncodeString(1, *m.Name)
  }
  if m.Result != nil {
    enc.EncodeInt32(2, *m.Result)
  }
  if len(m.unknownFields) > 0 {
    enc.EncodeRaw(m.unknownFields)
  }
  return nil
}
// Unmarshal decodes the Protocol Buffers binary format message in p and populates m with the
// result.
func (m *Example) Unmarshal(p []byte) error {
  if len(p) == 0 {
    return fmt.Errorf("cannot unmarshal from empty buffer")
  }
  var (
    tag int
    wt  csproto.WireType
    err error
  )
  dec := pbtools.NewDecoder(p)
  for dec.More() {
    tag, wt, err = dec.DecodeTag()
    if err != nil {
      return err
    }
    switch tag {
    case 1: // Name
      if wt != csproto.WireTypeLengthDelimited {
        return fmt.Errorf("invalid message data, expected wire type 2 for tag 1, got %v", wt)
      }
      if v, err := dec.DecodeString();  err != nil {
        return fmt.Errorf("unable to decode string value for tag 1: %w", err)
      } else {
        m.Name = csproto.String(v)
      }
    case 2: // Result
      if wt != csproto.WireTypeVarint {
        return fmt.Errorf("invalid message data, expected wire type 0 for tag 2, got %v", wt)
      }
      if v, err := dec.DecodeInt32(); err != nil {
        return fmt.Errorf("unable to decode int32 value for tag 2: %w", err)
      } else {
        m.Result = csproto.Int32(v)
      }
    default: // unrecognized/unknown field
      if skipped, err := dec.Skip(tag, wt); err != nil {
        return fmt.Errorf("invalid operation skipping tag %v: %w", tag, err)
      } else {
        m.unknownFields = append(m.unknownFields, skipped)
      }
    }
  }
  return nil
}
Final Benchmarks

After invoking protoc-gen-fastmarshal, the final benchmarks for our examples are:

$ go test -run='^$' -bench=. -benchmem ./proto2 ./proto3
goos: darwin
goarch: amd64
pkg: github.com/CrowdStrike/csproto/example/proto2
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkEncodeGogo-12                   1932699               597.6 ns/op           352 B/op          2 allocs/op
BenchmarkCustomEncodeGogo-12             2458599               482.3 ns/op           176 B/op          1 allocs/op
BenchmarkDecodeGogo-12                    622585              1887 ns/op            1376 B/op         34 allocs/op
BenchmarkCustomDecodeGogo-12              798523              1390 ns/op            1144 B/op         27 allocs/op
BenchmarkEncodeGoogleV1-12               1298185               925.5 ns/op           176 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV1-12         2701975               432.4 ns/op           176 B/op          1 allocs/op
BenchmarkDecodeGoogleV1-12                616106              1662 ns/op            1176 B/op         28 allocs/op
BenchmarkCustomDecodeGoogleV1-12          776244              1471 ns/op            1160 B/op         26 allocs/op
BenchmarkEncodeGoogleV2-12               1331971               911.3 ns/op           176 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV2-12         2817786               426.1 ns/op           176 B/op          1 allocs/op
BenchmarkDecodeGoogleV2-12                671048              1739 ns/op            1176 B/op         28 allocs/op
BenchmarkCustomDecodeGoogleV2-12          755186              1530 ns/op            1160 B/op         26 allocs/op
PASS
ok      github.com/CrowdStrike/csproto/example/proto2   12.247s

goos: darwin
goarch: amd64
pkg: github.com/CrowdStrike/csproto/example/proto3
cpu: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
BenchmarkEncodeGogo-12                   3479755               341.0 ns/op           208 B/op          3 allocs/op
BenchmarkCustomEncodeGogo-12             4824855               248.1 ns/op           112 B/op          2 allocs/op
BenchmarkDecodeGogo-12                   1328734               909.9 ns/op           424 B/op         16 allocs/op
BenchmarkCustomDecodeGogo-12             1604020               753.5 ns/op           408 B/op         15 allocs/op
BenchmarkEncodeGoogleV1-12               2599558               450.6 ns/op            96 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV1-12         3452514               348.4 ns/op           112 B/op          2 allocs/op
BenchmarkDecodeGoogleV1-12                962179              1076 ns/op             440 B/op         16 allocs/op
BenchmarkCustomDecodeGoogleV1-12         1337054               904.2 ns/op           424 B/op         15 allocs/op
BenchmarkEncodeGoogleV2-12               2741904               433.4 ns/op            96 B/op          1 allocs/op
BenchmarkCustomEncodeGoogleV2-12         3337425               356.1 ns/op           112 B/op          2 allocs/op
BenchmarkDecodeGoogleV2-12               1000000              1077 ns/op             440 B/op         16 allocs/op
BenchmarkCustomDecodeGoogleV2-12         1327365               913.4 ns/op           424 B/op         15 allocs/op
PASS
ok      github.com/CrowdStrike/csproto/example/proto3   9.186s

As you can see in the table below, the optimized code is faster across the board.

Cost proto2 (encode) proto3 (encode) proto2 (decode) proto3 (decode)
Gogo -19.3% -27.2% -26.3% -17.2%
Google V1 -53.3% -22.7% -11.5% -16.0%
Google V2 -53.3% -17.8% -12.0% -15.2%
A Warning About proto2 Extension Fields

Unfortunately, the news is not all good. This library is limited to the public APIs of the three underlying runtimes when working with proto2 extension fields and those APIs take advantage of unexported features to gain efficiency. Since they are unexported, those efficiency gains are unavailable to the code in csproto and the code generated by protoc-gen-fastmarshal that calls it.

One significant difference is in Unmarshal(). All three runtimes have internal code that delays decoding the bytes of proto2 extension fields until GetExtension() is called. Since csproto does not have access to this code, the unmarshaling code generated by protoc-gen-fastmarshal has to actually decode the extension field then explicitly call SetExtension() with the result. In some cases we've seen this be as much as 30% slower than calling the underlying proto.Unmarshal() directly. You can see this in action by removing the calls to GetExtension() from the proto2 Gogo benchmarks and observing the change in the resulting measurements.

Because of these issues, we advise that projects which rely heavily on proto2 extension fields SHOULD NOT use protoc-gen-fastmarshal to generate custom marshal/unmarshal code.

gRPC

To use with gRPC you will need to register csproto as the encoder. NOTE: If messages do not implement Marshaler or Unmarshaler then an error will be returned. An example is below.

For more information, see the gRPC documentation.

import (
  "github.com/CrowdStrike/csproto"
  "google.golang.org/grpc/encoding"
  _ "google.golang.org/grpc/encoding/proto"
)

func init() {
  encoding.RegisterCodec(csproto.GrpcCodec{})
}

Documentation

Index

Constants

View Source 
const MaxTagValue = 536870911

MaxTagValue is the largest supported protobuf field tag, which is 2^29 - 1 (or 536,870,911)

Variables

View Source 
var (
	// ErrInvalidFieldTag is returned by the decoder when it fails to read a varint-encoded field tag/wire type value.
	ErrInvalidFieldTag = errors.New("unable to read protobuf field tag")
	// ErrInvalidVarintData is returned by the decoder when it fails to read a varint-encoded value.
	ErrInvalidVarintData = errors.New("unable to read protobuf varint value")
	// ErrValueOverflow is returned by DecodeUInt32() or DecodeInt32() when the decoded value is too large for a 32-bit value.
	ErrValueOverflow = errors.New("value overflow trying to read protobuf varint value")
	// ErrInvalidZigZagData is returned by the decoder when it fails to read a zigzag-encoded value.
	ErrInvalidZigZagData = errors.New("unable to read protobuf zigzag value")
	// ErrInvalidFixed32Data is returned by the decoder when it fails to read a fixed-size 32-bit value.
	ErrInvalidFixed32Data = errors.New("unable to read protobuf fixed 32-bit value")
	// ErrInvalidFixed64Data is returned by the decoder when it fails to read a fixed-size 64-bit value.
	ErrInvalidFixed64Data = errors.New("unable to read protobuf fixed 64-bit value")
	// ErrInvalidPackedData is returned by the decoder when it fails to read a packed repeated value.
	ErrInvalidPackedData = errors.New("unable to read protobuf packed value")
)
View Source 
var (
	// ErrMarshaler is returned the Marshal() function when a message is passed in that does not
	// match any of the supported behaviors
	ErrMarshaler = errors.New("message does not implement csproto.Marshaler")
	// ErrUnmarshaler is returned the Unmarshal() function when a message is passed in that does not
	// match any of the supported behaviors
	ErrUnmarshaler = errors.New("message does not implement csproto.Unmarshaler")
)

Functions

func Bool 

func Bool(v bool) *bool

Bool returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func ClearAllExtensions added in v0.8.0 

func ClearAllExtensions(msg interface{})

ClearAllExtensions removes all proto2 extension fields from msg.

This operation can be very inefficient, especially for Google V2 messages, so it is generally better to explicitly clear individual extension fields.

func ClearExtension added in v0.3.0 

func ClearExtension(msg interface{}, ext interface{})

ClearExtension removes a proto2 extension field from msg, if it exists, delegating to the appropriate underlying Protobuf API based on the concrete type of msg.

This function panics if the provded parameters are invalid, rather than returning an error, to be consistent with the signature of the ClearExtension() functions in the underlying Protobuf runtimes.

func Clone added in v0.13.0 

func Clone(m interface{}) interface{}

Clone returns a deep copy of m, delegating to the appropriate underlying Protobuf API based on the concrete type of m. Since the underlying runtimes return different types, this function returns interface{} and the caller will need to type-assert back to the concrete type of m.

If m is not one of the supported message types, this function returns nil.

func DecodeFixed32 added in v0.20.0 

func DecodeFixed32(p []byte) (v uint32, n int, err error)

DecodeFixed32 reads a Protobuf fixed32 or float (4 byte, little endian) value from p and returns the value and the number of bytes consumed.

func DecodeFixed64 added in v0.20.0 

func DecodeFixed64(p []byte) (v uint64, n int, err error)

DecodeFixed64 reads a Protobuf fixed64 or double (8 byte, little endian) value from p and returns the value and the number of bytes consumed.

func DecodeVarint added in v0.20.0 

func DecodeVarint(p []byte) (v uint64, n int, err error)

DecodeVarint reads a base-128 varint encoded integer from p and returns the value and the number of bytes that were consumed.

func DecodeZigZag32 added in v0.20.0 

func DecodeZigZag32(p []byte) (v int32, n int, err error)

DecodeZigZag32 reads a base-128 zig zag encoded 32-bit integer from p and returns the value and the number of bytes that were consumed.

func DecodeZigZag64 added in v0.20.0 

func DecodeZigZag64(p []byte) (v int64, n int, err error)

DecodeZigZag64 reads a base-128 zig zag encoded 64-bit integer from p and returns the value and the number of bytes that were consumed.

func EncodeFixed32 added in v0.12.0 

func EncodeFixed32(dest []byte, v uint32) int

EncodeFixed32 encodes v into dest using the Protobuf fixed 32-bit encoding, which is just the 4 bytes of the value in little-endian format, and returns the number of bytes written

func EncodeFixed64 added in v0.12.0 

func EncodeFixed64(dest []byte, v uint64) int

EncodeFixed64 encodes v into dest using the Protobuf fixed 64-bit encoding, which is just the 8 bytes of the value in little-endian format, and returns the number of bytes written

func EncodeTag 

func EncodeTag(dest []byte, tag int, wireType WireType) int

EncodeTag combines tag and wireType then encodes the result into dest using the Protobuf varint format and returns the number of bytes written.

func EncodeVarint 

func EncodeVarint(dest []byte, v uint64) int

EncodeVarint encodes v into dest using the Protobuf base-128 varint format and returns the number of bytes written

func EncodeZigZag32 

func EncodeZigZag32(dest []byte, v int32) int

EncodeZigZag32 encodes v into dest using the Protobuf zig/zag encoding for more efficient encoding of negative numbers, and returns the number of bytes written.

func EncodeZigZag64 

func EncodeZigZag64(dest []byte, v int64) int

EncodeZigZag64 encodes v into dest using the Protobuf zig/zag encoding for more efficient encoding of negative numbers, and returns the number of bytes written.

func Equal added in v0.11.0 

func Equal(m1, m2 interface{}) bool

Equal returns true iff m1 and m2 are equal.

If m1 and m2 are not the same "type" (generated for the same runtime) then they are not equal. Otherwise, we delegate comparison to the appropriate underlying Protobuf API based on the concrete type of the messages

func ExtensionFieldNumber added in v0.15.0 

func ExtensionFieldNumber(ext any) (int, error)

ExtensionFieldNumber returns the integer field tag associated with the specified proto2 extension descriptor. If ext is not one of the three supported types, this function returns 0 and an error.

func Float32 

func Float32(v float32) *float32

Float32 returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func Float64 

func Float64(v float64) *float64

Float64 returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func GetExtension 

func GetExtension(msg interface{}, ext interface{}) (interface{}, error)

GetExtension returns a proto2 extension field from msg, delegating to the appropriate underlying Protobuf API based on the concrete type of msg.

func HasExtension 

func HasExtension(msg interface{}, ext interface{}) bool

HasExtension returns true if msg contains the specified proto2 extension field, delegating to the appropriate underlying Protobuf API based on the concrete type of msg.

func Int 

func Int(v int) *int32

Int returns a pointer to v as an int32 value (for use when assigning pointer fields on Protobuf message types)

func Int32 

func Int32(v int32) *int32

Int32 returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func Int64 

func Int64(v int64) *int64

Int64 returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func JSONMarshaler added in v0.2.0 

func JSONMarshaler(msg interface{}, opts ...JSONOption) json.Marshaler

JSONMarshaler returns an implementation of the json.Marshaler interface that formats msg to JSON using the specified options.

func JSONUnmarshaler added in v0.12.0 

func JSONUnmarshaler(msg interface{}, opts ...JSONOption) json.Unmarshaler

JSONUnmarshaler returns an implementation of the json.Unmarshaler interface that unmarshals a JSON data stream into msg using the specified options.

func Marshal 

func Marshal(msg interface{}) ([]byte, error)

Marshal marshals msg to binary Protobuf format, delegating to the appropriate underlying Protobuf API based on the concrete type of msg.

func MarshalText added in v0.9.0 

func MarshalText(msg interface{}) (string, error)

MarshalText converts the specified message to prototext string format

func PointerTo 

func PointerTo[T NativeTypes](v T) *T

PointerTo makes a copy of v and returns a pointer to that copy

func RangeExtensions added in v0.14.0 

func RangeExtensions(msg interface{}, fn func(value interface{}, name string, field int32) error) error

RangeExtensions iterates through all extension descriptors of a given proto message, calling fn on each iteration. It returns immediately on any error encountered. WARNING: RangeExtensions ranges over all registered extensions and therefore has a very high performance cost. Please consider using individual calls to GetExtension, if possible.

func Reset added in v0.18.0 

func Reset(m any)

Reset clears all fields of m.

If m is not a supported message type (either generated by google.golang.org/protobuf/cmd/protoc-gen-go or has a Reset() method) this function panics

func SetExtension 

func SetExtension(msg interface{}, ext interface{}, val interface{}) error

SetExtension sets a proto2 extension field in msg to the provided value, delegating to the appropriate underlying Protobuf API based on the concrete type of msg.

func Size 

func Size(msg interface{}) int

Size returns the encoded size of msg.

func SizeOfTagKey 

func SizeOfTagKey(k int) int

SizeOfTagKey returns the number of bytes required to hold the Protobuf varint encoding of k.

func SizeOfVarint 

func SizeOfVarint(v uint64) int

SizeOfVarint returns the number of bytes required to hold the Protobuf varint encoding of v.

func SizeOfZigZag 

func SizeOfZigZag(v uint64) int

SizeOfZigZag returns the number of bytes required to hold the zig zag encoding of v.

func String 

func String(v string) *string

String returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func Uint32 

func Uint32(v uint32) *uint32

Uint32 returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func Uint64 

func Uint64(v uint64) *uint64

Uint64 returns a pointer to v (for use when assigning pointer fields on Protobuf message types)

func Unmarshal 

func Unmarshal(data []byte, msg interface{}) error

Unmarshal decodes the specified Protobuf data into msg, delegating to the appropriate underlying Protobuf API based on the concrete type of msg.

Types

type Decoder 

type Decoder struct {
	// contains filtered or unexported fields
}

Decoder implements a binary Protobuf Decoder by sequentially reading from a provided []byte.

func NewDecoder 

func NewDecoder(p []byte) *Decoder

NewDecoder initializes a new Protobuf decoder to read the provided buffer.

func (*Decoder) DecodeBool 

func (d *Decoder) DecodeBool() (b bool, err error)

DecodeBool decodes a boolean value from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeBytes 

func (d *Decoder) DecodeBytes() ([]byte, error)

DecodeBytes decodes a length-delimited slice of bytes from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeFixed32 

func (d *Decoder) DecodeFixed32() (uint32, error)

DecodeFixed32 decodes a 4-byte integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeFixed64 

func (d *Decoder) DecodeFixed64() (uint64, error)

DecodeFixed64 decodes an 8-byte integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeFloat32 

func (d *Decoder) DecodeFloat32() (float32, error)

DecodeFloat32 decodes a 4-byte IEEE 754 floating point value from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeFloat64 

func (d *Decoder) DecodeFloat64() (float64, error)

DecodeFloat64 decodes an 8-byte IEEE 754 floating point value from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeInt32 

func (d *Decoder) DecodeInt32() (int32, error)

DecodeInt32 decodes a varint-encoded 32-bit integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeInt64 

func (d *Decoder) DecodeInt64() (int64, error)

DecodeInt64 decodes a varint-encoded 64-bit integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeNested 

func (d *Decoder) DecodeNested(m interface{}) error

DecodeNested decodes a nested Protobuf message from the stream into m. If m satisfies our csproto.Unmarshaler interface its Unmarshal() method will be called. Otherwise, this method delegates to Marshal().

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedBool 

func (d *Decoder) DecodePackedBool() ([]bool, error)

DecodePackedBool decodes a packed encoded list of boolean values from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedFixed32 

func (d *Decoder) DecodePackedFixed32() ([]uint32, error)

DecodePackedFixed32 decodes a packed encoded list of 32-bit fixed-width integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedFixed64 

func (d *Decoder) DecodePackedFixed64() ([]uint64, error)

DecodePackedFixed64 decodes a packed encoded list of 64-bit fixed-width integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedFloat32 

func (d *Decoder) DecodePackedFloat32() ([]float32, error)

DecodePackedFloat32 decodes a packed encoded list of 32-bit floating point numbers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedFloat64 

func (d *Decoder) DecodePackedFloat64() ([]float64, error)

DecodePackedFloat64 decodes a packed encoded list of 64-bit floating point numbers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedInt32 

func (d *Decoder) DecodePackedInt32() ([]int32, error)

DecodePackedInt32 decodes a packed encoded list of 32-bit integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedInt64 

func (d *Decoder) DecodePackedInt64() ([]int64, error)

DecodePackedInt64 decodes a packed encoded list of 64-bit integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedSint32 

func (d *Decoder) DecodePackedSint32() ([]int32, error)

DecodePackedSint32 decodes a packed encoded list of 32-bit signed integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedSint64 

func (d *Decoder) DecodePackedSint64() ([]int64, error)

DecodePackedSint64 decodes a packed encoded list of 64-bit signed integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedUint32 

func (d *Decoder) DecodePackedUint32() ([]uint32, error)

DecodePackedUint32 decodes a packed encoded list of unsigned 32-bit integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodePackedUint64 

func (d *Decoder) DecodePackedUint64() ([]uint64, error)

DecodePackedUint64 decodes a packed encoded list of unsigned 64-bit integers from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeSInt32 

func (d *Decoder) DecodeSInt32() (int32, error)

DecodeSInt32 decodes a zigzag-encoded 32-bit integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeSInt64 

func (d *Decoder) DecodeSInt64() (int64, error)

DecodeSInt64 decodes a zigzag-encoded 32-bit integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeString 

func (d *Decoder) DecodeString() (string, error)

DecodeString decodes a length-delimited string from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeTag 

func (d *Decoder) DecodeTag() (tag int, wireType WireType, err error)

DecodeTag decodes a field tag and Protobuf wire type from the stream and returns the values.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeUInt32 

func (d *Decoder) DecodeUInt32() (uint32, error)

DecodeUInt32 decodes a varint-encoded 32-bit unsigned integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) DecodeUInt64 

func (d *Decoder) DecodeUInt64() (uint64, error)

DecodeUInt64 decodes a varint-encoded 64-bit unsigned integer from the stream and returns the value.

io.ErrUnexpectedEOF is returned if the operation would read past the end of the data.

func (*Decoder) Mode added in v0.27.0 

func (d *Decoder) Mode() DecoderMode

Mode returns the current decoding mode, safe vs fastest.

func (*Decoder) More 

func (d *Decoder) More() bool

More indicates if there is more data to be read in the buffer.

func (*Decoder) Offset added in v0.23.0 

func (d *Decoder) Offset() int

Offset returns the current read offset

func (*Decoder) Reset 

func (d *Decoder) Reset()

Reset moves the read offset back to the beginning of the encoded data

func (*Decoder) Seek added in v0.27.0 

func (d *Decoder) Seek(offset int64, whence int) (int64, error)

Seek sets the position of the next read operation to [offset], interpreted according to [whence]: io.SeekStart means relative to the start of the data, io.SeekCurrent means relative to the current offset, and io.SeekEnd means relative to the end.

This low-level operation is provided to support advanced/custom usages of the decoder and it is up to the caller to ensure that the resulting offset will point to a valid location in the data stream.

func (*Decoder) SetMode 

func (d *Decoder) SetMode(m DecoderMode)

SetMode configures the decoding behavior, safe vs fastest.

func (*Decoder) Skip 

func (d *Decoder) Skip(tag int, wt WireType) ([]byte, error)

Skip skips over the encoded field value at the current offset, returning the raw bytes so that the caller can decide what to do with the data.

The tag and wire type are validated against the provided values and a DecoderSkipError error is returned if they do not match. This check is skipped when using "fast" mode.

io.ErrUnexpectedEOF is returned if the operation would advance past the end of the data.

type DecoderMode 

type DecoderMode int

DecoderMode defines the behavior of the decoder (safe vs fastest). 

const (
	// DecoderModeSafe instructs the decoder to only use safe operations when decoding values.
	DecoderModeSafe DecoderMode = iota
	// DecoderModeFast instructs the decoder to use unsafe operations to avoid allocations and copying data
	// for the fastest throughput.
	//
	// When using DecoderModeFast, the byte slice passed to the decoder must not be modified after
	// using the decoder to extract values.  The behavior is undefined if the slice is modified.
	DecoderModeFast
)

func (DecoderMode) String 

func (m DecoderMode) String() string

String returns a string representation of m, "safe" or "fast".

type DecoderSkipError 

type DecoderSkipError struct {
	ExpectedTag      int
	ExpectedWireType WireType
	ActualTag        int
	ActualWireType   WireType
}

DecoderSkipError defines an error returned by the decoder's Skip() method when the specified tag and wire type do not match the data in the stream at the current decoder offset.

func (*DecoderSkipError) Error 

func (e *DecoderSkipError) Error() string

Error satisfies the error interface

type Encoder 

type Encoder struct {
	// contains filtered or unexported fields
}

Encoder implements a binary Protobuf Encoder by sequentially writing to a wrapped []byte.

func NewEncoder 

func NewEncoder(p []byte) *Encoder

NewEncoder initializes a new Protobuf encoder to write to the specified buffer, which must be pre-allocated by the caller with sufficient space to hold the message(s) being written.

func (*Encoder) EncodeBool 

func (e *Encoder) EncodeBool(tag int, v bool)

EncodeBool writes a varint-encoded boolean value to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeBytes 

func (e *Encoder) EncodeBytes(tag int, v []byte)

EncodeBytes writes a length-delimited byte slice to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeFixed32 

func (e *Encoder) EncodeFixed32(tag int, v uint32)

EncodeFixed32 writes a 32-bit unsigned integer value to the buffer using 4 bytes in little endian format, preceded by the varint-encoded tag key.

func (*Encoder) EncodeFixed64 

func (e *Encoder) EncodeFixed64(tag int, v uint64)

EncodeFixed64 writes a 64-bit unsigned integer value to the buffer using 8 bytes in little endian format, preceded by the varint-encoded tag key.

func (*Encoder) EncodeFloat32 

func (e *Encoder) EncodeFloat32(tag int, v float32)

EncodeFloat32 writes a 32-bit IEEE 754 floating point value to the buffer using 4 bytes in little endian format, preceded by the varint-encoded tag key.

func (*Encoder) EncodeFloat64 

func (e *Encoder) EncodeFloat64(tag int, v float64)

EncodeFloat64 writes a 64-bit IEEE 754 floating point value to the buffer using 8 bytes in little endian format, preceded by the varint-encoded tag key.

func (*Encoder) EncodeInt32 

func (e *Encoder) EncodeInt32(tag int, v int32)

EncodeInt32 writes a varint-encoded 32-bit signed integer value to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeInt64 

func (e *Encoder) EncodeInt64(tag int, v int64)

EncodeInt64 writes a varint-encoded 64-bit signed integer value to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeMapEntryHeader 

func (e *Encoder) EncodeMapEntryHeader(tag int, size int)

EncodeMapEntryHeader writes a map entry header into the buffer, which consists of the specified tag with a wire type of WireTypeLengthDelimited followed by the varint encoded entry size.

func (*Encoder) EncodeNested 

func (e *Encoder) EncodeNested(tag int, m interface{}) error

EncodeNested writes a nested message to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedBool 

func (e *Encoder) EncodePackedBool(tag int, vs []bool)

EncodePackedBool writes a list of booleans to the buffer using packed encoding, preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedFixed32 

func (e *Encoder) EncodePackedFixed32(tag int, vs []uint32)

EncodePackedFixed32 writes a list of 32-bit fixed-width unsigned integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedFixed64 

func (e *Encoder) EncodePackedFixed64(tag int, vs []uint64)

EncodePackedFixed64 writes a list of 64-bit fixed-width unsigned integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedFloat32 

func (e *Encoder) EncodePackedFloat32(tag int, vs []float32)

EncodePackedFloat32 writes a list of 32-bit floating point numbers to the buffer using packed encoding, preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedFloat64 

func (e *Encoder) EncodePackedFloat64(tag int, vs []float64)

EncodePackedFloat64 writes a list of 64-bit floating point numbers to the buffer using packed encoding, preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedInt32 

func (e *Encoder) EncodePackedInt32(tag int, vs []int32)

EncodePackedInt32 writes a list of 32-bit integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

This operation is O(n^2) because we have to traverse the list of values to calculate the total encoded size and write that size *before* the actual encoded values.

func (*Encoder) EncodePackedInt64 

func (e *Encoder) EncodePackedInt64(tag int, vs []int64)

EncodePackedInt64 writes a list of 64-bit integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

This operation is O(n^2) because we have to traverse the list of values to calculate the total encoded size and write that size *before* the actual encoded values.

func (*Encoder) EncodePackedSFixed32 

func (e *Encoder) EncodePackedSFixed32(tag int, vs []int32)

EncodePackedSFixed32 writes a list of 32-bit fixed-width signed integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedSFixed64 

func (e *Encoder) EncodePackedSFixed64(tag int, vs []int64)

EncodePackedSFixed64 writes a list of 64-bit fixed-width signed integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

func (*Encoder) EncodePackedSInt32 

func (e *Encoder) EncodePackedSInt32(tag int, vs []int32)

EncodePackedSInt32 writes a list of 32-bit signed integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

This operation is O(n^2) because we have to traverse the list of values to calculate the total encoded size and write that size *before* the actual encoded values.

func (*Encoder) EncodePackedSInt64 

func (e *Encoder) EncodePackedSInt64(tag int, vs []int64)

EncodePackedSInt64 writes a list of 64-bit signed integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

This operation is O(n^2) because we have to traverse the list of values to calculate the total encoded size and write that size *before* the actual encoded values.

func (*Encoder) EncodePackedUInt32 

func (e *Encoder) EncodePackedUInt32(tag int, vs []uint32)

EncodePackedUInt32 writes a list of 32-bit unsigned integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

This operation is O(n^2) because we have to traverse the list of values to calculate the total encoded size and write that size *before* the actual encoded values.

func (*Encoder) EncodePackedUInt64 

func (e *Encoder) EncodePackedUInt64(tag int, vs []uint64)

EncodePackedUInt64 writes a list of 64-bit unsigned integers to the buffer using packed encoding, preceded by the varint-encoded tag key.

This operation is O(n^2) because we have to traverse the list of values to calculate the total encoded size and write that size *before* the actual encoded values.

func (*Encoder) EncodeRaw 

func (e *Encoder) EncodeRaw(d []byte)

EncodeRaw writes the raw bytes of d into the buffer at the current offset

func (*Encoder) EncodeSInt32 

func (e *Encoder) EncodeSInt32(tag int, v int32)

EncodeSInt32 writes a zigzag-encoded 32-bit signed integer value to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeSInt64 

func (e *Encoder) EncodeSInt64(tag int, v int64)

EncodeSInt64 writes a zigzag-encoded 64-bit signed integer value to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeString 

func (e *Encoder) EncodeString(tag int, s string)

EncodeString writes a length-delimited string value to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeUInt32 

func (e *Encoder) EncodeUInt32(tag int, v uint32)

EncodeUInt32 writes a varint-encoded 32-bit unsigned integer value to the buffer preceded by the varint-encoded tag key.

func (*Encoder) EncodeUInt64 

func (e *Encoder) EncodeUInt64(tag int, v uint64)

EncodeUInt64 writes a varint-encoded 64-bit unsigned integer value to the buffer preceded by the varint-encoded tag key.

type GrpcCodec 

type GrpcCodec struct{}

GrpcCodec implements gRPC encoding.Codec. See: https://pkg.go.dev/google.golang.org/grpc/encoding#Codec

func (GrpcCodec) Marshal 

func (GrpcCodec) Marshal(v interface{}) ([]byte, error)

Marshal returns the wire format of v.

func (GrpcCodec) Name 

func (GrpcCodec) Name() string

Name returns the name of the Codec implementation.

func (GrpcCodec) Unmarshal 

func (GrpcCodec) Unmarshal(data []byte, v interface{}) error

Unmarshal parses the wire format into v.

type JSONOption added in v0.2.0 

type JSONOption func(*jsonOptions)

JSONOption defines a function that sets a specific JSON formatting option

func JSONAllowPartialMessages added in v0.12.0 

func JSONAllowPartialMessages(allow bool) JSONOption

JSONAllowPartialMessages returns a JSON option that configured JSON unmarshaling to not return an error if unmarshaling data results in required fields not being set on the message.

Note: only applies to Google V2 (google.golang.org/protobuf) messages that are using proto2 syntax.

func JSONAllowUnknownFields added in v0.12.0 

func JSONAllowUnknownFields(allow bool) JSONOption

JSONAllowUnknownFields returns a JSON option that configures JSON unmarshaling to skip unknown fields rather than return an error

func JSONIncludeZeroValues added in v0.2.0 

func JSONIncludeZeroValues(emitZeroValues bool) JSONOption

JSONIncludeZeroValues returns a JSON option that enables or disables including zero-valued fields in the JSON output.

func JSONIndent added in v0.2.0 

func JSONIndent(indent string) JSONOption

JSONIndent returns a JSONOption that configures the JSON indentation.

Passing an empty string disables indentation. If not empty, indent must consist of only spaces or tab characters.

func JSONUseEnumNumbers added in v0.2.0 

func JSONUseEnumNumbers(useNumbers bool) JSONOption

JSONUseEnumNumbers returns a JSON option that enables or disables outputting integer values rather than the enum names for enum fields.

type Marshaler 

type Marshaler interface {
	// Marshal allocates a buffer large enough to hold this message, writes the contents of the
	// message into it, then returns the buffer.
	Marshal() ([]byte, error)
}

Marshaler defines a message that can marshal itself into binary Protobuf format.

type MarshalerTo 

type MarshalerTo interface {
	// MarshalTo writes the contents of this message into dest using the binary Protobuf encoding.
	//
	// It is up to the caller to ensure that the buffer is large enough to hold the serialized message
	// data.
	MarshalTo(dest []byte) error
}

MarshalerTo defines a message that can marshal itself into a provided buffer in binary Protobuf format.

type MessageType 

type MessageType int

MessageType defines the types of Protobuf message implementations this API supports. 

const (
	// MessageTypeUnknown indicates that we don't know which type of Protobuf message a type is.
	MessageTypeUnknown MessageType = iota
	// MessageTypeGogo indicates that a type was generated using Gogo Protobuf.
	MessageTypeGogo
	// MessageTypeGoogleV1 indicates that a type was generated using v1 of Google's Protobuf tools.
	MessageTypeGoogleV1
	// MessageTypeGoogle indicates that a type was generated using v2 of Google's Protobuf tools.
	MessageTypeGoogle
)

func MsgType 

func MsgType(msg interface{}) MessageType

MsgType accepts a protobuf message and returns the corresponding MessageType value.

type NativeTypes 

type NativeTypes interface {
	bool | int32 | int64 | uint32 | uint64 | float32 | float64 | string
}

NativeTypes defines a generic constraint for the native Go types that need to be converted to pointers when storing values in generated Protobuf message structs.

Unsized integers (type int) also need to be converted to a pointer, but Protobuf doesn't support unsized integers. Use csproto.Int(v int) *int32 instead.

type ProtoV1Marshaler 

type ProtoV1Marshaler interface {
	ProtoV1Sizer
	XXX_Marshal(b []byte, deterministic bool) ([]byte, error)
}

ProtoV1Marshaler defines the interface for a type that provides custom Protobuf V1 marshaling logic.

type ProtoV1Sizer 

type ProtoV1Sizer interface {
	XXX_Size() int
}

ProtoV1Sizer defines the interface for a type that provides custom Protobuf V1 sizing logic.

type ProtoV1Unmarshaler 

type ProtoV1Unmarshaler interface {
	XXX_Unmarshal([]byte) error
}

ProtoV1Unmarshaler defines the interface a type that provides custom Protobuf V1 unmarshaling.

type Sizer 

type Sizer interface {
	// Size returns the size, in bytes, required to serialize a message into binary Protobuf format.
	Size() int
}

Sizer defines a message that can pre-calculate the required size for storing the binary Protobuf encoding of its contents.

type Unmarshaler 

type Unmarshaler interface {
	// Unmarshal decodes the binary Protobuf data into this message.
	Unmarshal([]byte) error
}

Unmarshaler defines a message that can unmarshal binary Protobuf data into itself.

type WireType 

type WireType int

WireType defines the supported values for Protobuf wire encodings. 

const (
	// WireTypeVarint denotes a value that is encoded using base128 varint encoding.
	WireTypeVarint WireType = 0
	// WireTypeFixed64 denotes a value that is encoded using 8 bytes.
	WireTypeFixed64 WireType = 1
	// WireTypeLengthDelimited denotes a value that is encoded as a sequence of bytes preceded by a
	// varint-encoded length.
	WireTypeLengthDelimited WireType = 2
	// WireTypeFixed32 denotes a value that is encoded using 4 bytes.
	WireTypeFixed32 WireType = 5
)

func (WireType) String 

func (wt WireType) String() string

String returns a string representation of wt.

Source Files

View all Source files

Directories

Path Synopsis
cmd
protoc-gen-fastmarshal
protodump

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.