proto

package
v1.35.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 7, 2024 License: BSD-3-Clause Imports: 16 Imported by: 27,570

Documentation

Overview

Package proto provides functions operating on protocol buffer messages.

For documentation on protocol buffers in general, see: https://protobuf.dev.

For a tutorial on using protocol buffers with Go, see: https://protobuf.dev/getting-started/gotutorial.

For a guide to generated Go protocol buffer code, see: https://protobuf.dev/reference/go/go-generated.

Binary serialization

This package contains functions to convert to and from the wire format, an efficient binary serialization of protocol buffers.

  • Size reports the size of a message in the wire format.

  • Marshal converts a message to the wire format. The MarshalOptions type provides more control over wire marshaling.

  • Unmarshal converts a message from the wire format. The UnmarshalOptions type provides more control over wire unmarshaling.

Basic message operations

Optional scalar constructors

The API for some generated messages represents optional scalar fields as pointers to a value. For example, an optional string field has the Go type *string.

Generated enum types usually have an Enum method which performs the same operation.

Optional scalar fields are only supported in proto2.

Extension accessors

Extension fields are only supported in proto2.

This module contains additional packages for more specialized use cases. Consult the individual package documentation for details.

Index

Examples

Constants

This section is empty.

Variables

View Source
var Error error

Error matches all errors produced by packages in the protobuf module according to errors.Is.

Example usage:

if errors.Is(err, proto.Error) { ... }

Functions

func Bool

func Bool(v bool) *bool

Bool stores v in a new bool value and returns a pointer to it.

func CheckInitialized

func CheckInitialized(m Message) error

CheckInitialized returns an error if any required fields in m are not set.

func ClearExtension

func ClearExtension(m Message, xt protoreflect.ExtensionType)

ClearExtension clears an extension field such that subsequent HasExtension calls return false. It panics if m is invalid or if xt does not extend m.

func Equal

func Equal(x, y Message) bool

Equal reports whether two messages are equal, by recursively comparing the fields of the message.

  • Bytes fields are equal if they contain identical bytes. Empty bytes (regardless of nil-ness) are considered equal.

  • Floating-point fields are equal if they contain the same value. Unlike the == operator, a NaN is equal to another NaN.

  • Other scalar fields are equal if they contain the same value.

  • Message fields are equal if they have the same set of populated known and extension field values, and the same set of unknown fields values.

  • Lists are equal if they are the same length and each corresponding element is equal.

  • Maps are equal if they have the same set of keys and the corresponding value for each key is equal.

An invalid message is not equal to a valid message. An invalid message is only equal to another invalid message of the same type. An invalid message often corresponds to a nil pointer of the concrete message type. For example, (*pb.M)(nil) is not equal to &pb.M{}. If two valid messages marshal to the same bytes under deterministic serialization, then Equal is guaranteed to report true.

func Float32

func Float32(v float32) *float32

Float32 stores v in a new float32 value and returns a pointer to it.

func Float64

func Float64(v float64) *float64

Float64 stores v in a new float64 value and returns a pointer to it.

func GetExtension

func GetExtension(m Message, xt protoreflect.ExtensionType) any

GetExtension retrieves the value for an extension field. If the field is unpopulated, it returns the default value for scalars and an immutable, empty value for lists or messages. It panics if xt does not extend m.

The type of the value is dependent on the field type of the extension. For extensions generated by protoc-gen-go, the Go type is as follows:

╔═══════════════════╤═════════════════════════╗
║ Go type           │ Protobuf kind           ║
╠═══════════════════╪═════════════════════════╣
║ bool              │ bool                    ║
║ int32             │ int32, sint32, sfixed32 ║
║ int64             │ int64, sint64, sfixed64 ║
║ uint32            │ uint32, fixed32         ║
║ uint64            │ uint64, fixed64         ║
║ float32           │ float                   ║
║ float64           │ double                  ║
║ string            │ string                  ║
║ []byte            │ bytes                   ║
║ protoreflect.Enum │ enum                    ║
║ proto.Message     │ message, group          ║
╚═══════════════════╧═════════════════════════╝

The protoreflect.Enum and proto.Message types are the concrete Go type associated with the named enum or message. Repeated fields are represented using a Go slice of the base element type.

If a generated extension descriptor variable is directly passed to GetExtension, then the call should be followed immediately by a type assertion to the expected output value. For example:

mm := proto.GetExtension(m, foopb.E_MyExtension).(*foopb.MyMessage)

This pattern enables static analysis tools to verify that the asserted type matches the Go type associated with the extension field and also enables a possible future migration to a type-safe extension API.

Since singular messages are the most common extension type, the pattern of calling HasExtension followed by GetExtension may be simplified to:

if mm := proto.GetExtension(m, foopb.E_MyExtension).(*foopb.MyMessage); mm != nil {
    ... // make use of mm
}

The mm variable is non-nil if and only if HasExtension reports true.

func HasExtension

func HasExtension(m Message, xt protoreflect.ExtensionType) bool

HasExtension reports whether an extension field is populated. It returns false if m is invalid or if xt does not extend m.

func Int32

func Int32(v int32) *int32

Int32 stores v in a new int32 value and returns a pointer to it.

func Int64

func Int64(v int64) *int64

Int64 stores v in a new int64 value and returns a pointer to it.

func Marshal

func Marshal(m Message) ([]byte, error)

Marshal returns the wire-format encoding of m.

This is the most common entry point for encoding a Protobuf message.

See the MarshalOptions type if you need more control.

Example

This example illustrates how to marshal (encode) a Protobuf message struct literal into wire-format encoding.

This example hard-codes a duration of 125ns for the illustration of struct fields, but note that you do not need to fill the fields of well-known types like duration.proto yourself. To convert a time.Duration, use google.golang.org/protobuf/types/known/durationpb.New.

package main

import (
	"fmt"

	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/types/known/durationpb"
)

func main() {
	b, err := proto.Marshal(&durationpb.Duration{
		Nanos: 125,
	})
	if err != nil {
		panic(err)
	}

	fmt.Printf("125ns encoded into %d bytes of Protobuf wire format:\n% x\n", len(b), b)

	// You can use protoscope to explore the wire format:
	// https://github.com/protocolbuffers/protoscope
	//
	// echo -n '10 7d' | xxd -r -ps | protoscope
	// 2: 125

}
Output:

125ns encoded into 2 bytes of Protobuf wire format:
10 7d

func Merge

func Merge(dst, src Message)

Merge merges src into dst, which must be a message with the same descriptor.

Populated scalar fields in src are copied to dst, while populated singular messages in src are merged into dst by recursively calling Merge. The elements of every list field in src is appended to the corresponded list fields in dst. The entries of every map field in src is copied into the corresponding map field in dst, possibly replacing existing entries. The unknown fields of src are appended to the unknown fields of dst.

It is semantically equivalent to unmarshaling the encoded form of src into dst with the [UnmarshalOptions.Merge] option specified.

func MessageName added in v1.26.0

func MessageName(m Message) protoreflect.FullName

MessageName returns the full name of m. If m is nil, it returns an empty string.

func RangeExtensions added in v1.22.0

func RangeExtensions(m Message, f func(protoreflect.ExtensionType, any) bool)

RangeExtensions iterates over every populated extension field in m in an undefined order, calling f for each extension type and value encountered. It returns immediately if f returns false. While iterating, mutating operations may only be performed on the current extension field.

func Reset

func Reset(m Message)

Reset clears every field in the message. The resulting message shares no observable memory with its previous state other than the memory for the message itself.

func SetExtension

func SetExtension(m Message, xt protoreflect.ExtensionType, v any)

SetExtension stores the value of an extension field. It panics if m is invalid, xt does not extend m, or if type of v is invalid for the specified extension field.

The type of the value is dependent on the field type of the extension. For extensions generated by protoc-gen-go, the Go type is as follows:

╔═══════════════════╤═════════════════════════╗
║ Go type           │ Protobuf kind           ║
╠═══════════════════╪═════════════════════════╣
║ bool              │ bool                    ║
║ int32             │ int32, sint32, sfixed32 ║
║ int64             │ int64, sint64, sfixed64 ║
║ uint32            │ uint32, fixed32         ║
║ uint64            │ uint64, fixed64         ║
║ float32           │ float                   ║
║ float64           │ double                  ║
║ string            │ string                  ║
║ []byte            │ bytes                   ║
║ protoreflect.Enum │ enum                    ║
║ proto.Message     │ message, group          ║
╚═══════════════════╧═════════════════════════╝

The protoreflect.Enum and proto.Message types are the concrete Go type associated with the named enum or message. Repeated fields are represented using a Go slice of the base element type.

If a generated extension descriptor variable is directly passed to SetExtension (e.g., foopb.E_MyExtension), then the value should be a concrete type that matches the expected Go type for the extension descriptor so that static analysis tools can verify type correctness. This also enables a possible future migration to a type-safe extension API.

func Size

func Size(m Message) int

Size returns the size in bytes of the wire-format encoding of m.

Example

Checking if Size returns 0 is an easy way to recognize empty messages:

package main

import (
	"google.golang.org/protobuf/proto"
)

func main() {
	var m proto.Message
	if proto.Size(m) == 0 {
		// No fields set (or, in proto3, all fields matching the default);
		// skip processing this message, or return an error, or similar.
	}
}
Output:

func String

func String(v string) *string

String stores v in a new string value and returns a pointer to it.

func Uint32

func Uint32(v uint32) *uint32

Uint32 stores v in a new uint32 value and returns a pointer to it.

func Uint64

func Uint64(v uint64) *uint64

Uint64 stores v in a new uint64 value and returns a pointer to it.

func Unmarshal

func Unmarshal(b []byte, m Message) error

Unmarshal parses the wire-format message in b and places the result in m. The provided message must be mutable (e.g., a non-nil pointer to a message).

See the UnmarshalOptions type if you need more control.

Example

This example illustrates how to unmarshal (decode) wire format encoding into a Protobuf message.

package main

import (
	"fmt"

	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/types/known/durationpb"
)

func main() {
	// This is the wire format encoding produced by the Marshal example.
	// Typically you would read from the network, from disk, etc.
	b := []byte{0x10, 0x7d}

	var dur durationpb.Duration
	if err := proto.Unmarshal(b, &dur); err != nil {
		panic(err)
	}

	fmt.Printf("Protobuf wire format decoded to duration %v\n", dur.AsDuration())

}
Output:

Protobuf wire format decoded to duration 125ns

Types

type MarshalOptions

type MarshalOptions struct {
	pragma.NoUnkeyedLiterals

	// AllowPartial allows messages that have missing required fields to marshal
	// without returning an error. If AllowPartial is false (the default),
	// Marshal will return an error if there are any missing required fields.
	AllowPartial bool

	// Deterministic controls whether the same message will always be
	// serialized to the same bytes within the same binary.
	//
	// Setting this option guarantees that repeated serialization of
	// the same message will return the same bytes, and that different
	// processes of the same binary (which may be executing on different
	// machines) will serialize equal messages to the same bytes.
	// It has no effect on the resulting size of the encoded message compared
	// to a non-deterministic marshal.
	//
	// Note that the deterministic serialization is NOT canonical across
	// languages. It is not guaranteed to remain stable over time. It is
	// unstable across different builds with schema changes due to unknown
	// fields. Users who need canonical serialization (e.g., persistent
	// storage in a canonical form, fingerprinting, etc.) must define
	// their own canonicalization specification and implement their own
	// serializer rather than relying on this API.
	//
	// If deterministic serialization is requested, map entries will be
	// sorted by keys in lexographical order. This is an implementation
	// detail and subject to change.
	Deterministic bool

	// UseCachedSize indicates that the result of a previous Size call
	// may be reused.
	//
	// Setting this option asserts that:
	//
	// 1. Size has previously been called on this message with identical
	// options (except for UseCachedSize itself).
	//
	// 2. The message and all its submessages have not changed in any
	// way since the Size call.
	//
	// If either of these invariants is violated,
	// the results are undefined and may include panics or corrupted output.
	//
	// Implementations MAY take this option into account to provide
	// better performance, but there is no guarantee that they will do so.
	// There is absolutely no guarantee that Size followed by Marshal with
	// UseCachedSize set will perform equivalently to Marshal alone.
	UseCachedSize bool
}

MarshalOptions configures the marshaler.

Example usage:

b, err := MarshalOptions{Deterministic: true}.Marshal(m)

func (MarshalOptions) Marshal

func (o MarshalOptions) Marshal(m Message) ([]byte, error)

Marshal returns the wire-format encoding of m.

func (MarshalOptions) MarshalAppend

func (o MarshalOptions) MarshalAppend(b []byte, m Message) ([]byte, error)

MarshalAppend appends the wire-format encoding of m to b, returning the result.

This is a less common entry point than Marshal, which is only needed if you need to supply your own buffers for performance reasons.

Example (SameBuffer)

This example illustrates how to marshal (encode) many Protobuf messages into wire-format encoding, using the same buffer.

MarshalAppend will grow the buffer as needed, so over time it will grow large enough to not need further allocations.

If unbounded growth of the buffer is undesirable in your application, you can use MarshalOptions.Size to determine a buffer size that is guaranteed to be large enough for marshaling without allocations.

package main

import (
	"google.golang.org/protobuf/proto"
)

func main() {
	var m proto.Message

	opts := proto.MarshalOptions{
		// set e.g. Deterministic: true, if needed
	}

	var buf []byte
	for i := 0; i < 100000; i++ {
		var err error
		buf, err = opts.MarshalAppend(buf[:0], m)
		if err != nil {
			panic(err)
		}
		// cap(buf) will grow to hold the largest m.

		// write buf to disk, network, etc.
	}
}
Output:

func (MarshalOptions) MarshalState

MarshalState returns the wire-format encoding of a message.

This method permits fine-grained control over the marshaler. Most users should use Marshal instead.

func (MarshalOptions) Size

func (o MarshalOptions) Size(m Message) int

Size returns the size in bytes of the wire-format encoding of m.

type Message

type Message = protoreflect.ProtoMessage

Message is the top-level interface that all messages must implement. It provides access to a reflective view of a message. Any implementation of this interface may be used with all functions in the protobuf module that accept a Message, except where otherwise specified.

This is the v2 interface definition for protobuf messages. The v1 interface definition is github.com/golang/protobuf/proto.Message.

func Clone

func Clone(m Message) Message

Clone returns a deep copy of m. If the top-level message is invalid, it returns an invalid message as well.

type UnmarshalOptions

type UnmarshalOptions struct {
	pragma.NoUnkeyedLiterals

	// Merge merges the input into the destination message.
	// The default behavior is to always reset the message before unmarshaling,
	// unless Merge is specified.
	Merge bool

	// AllowPartial accepts input for messages that will result in missing
	// required fields. If AllowPartial is false (the default), Unmarshal will
	// return an error if there are any missing required fields.
	AllowPartial bool

	// If DiscardUnknown is set, unknown fields are ignored.
	DiscardUnknown bool

	// Resolver is used for looking up types when unmarshaling extension fields.
	// If nil, this defaults to using protoregistry.GlobalTypes.
	Resolver interface {
		FindExtensionByName(field protoreflect.FullName) (protoreflect.ExtensionType, error)
		FindExtensionByNumber(message protoreflect.FullName, field protoreflect.FieldNumber) (protoreflect.ExtensionType, error)
	}

	// RecursionLimit limits how deeply messages may be nested.
	// If zero, a default limit is applied.
	RecursionLimit int
}

UnmarshalOptions configures the unmarshaler.

Example usage:

err := UnmarshalOptions{DiscardUnknown: true}.Unmarshal(b, m)

func (UnmarshalOptions) Unmarshal

func (o UnmarshalOptions) Unmarshal(b []byte, m Message) error

Unmarshal parses the wire-format message in b and places the result in m. The provided message must be mutable (e.g., a non-nil pointer to a message).

func (UnmarshalOptions) UnmarshalState

UnmarshalState parses a wire-format message and places the result in m.

This method permits fine-grained control over the unmarshaler. Most users should use Unmarshal instead.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL