proto

package
v1.36.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Dec 16, 2024 License: BSD-3-Clause Imports: 16 Imported by: 27,682

Documentation

Overview

Package proto provides functions operating on protocol buffer messages.

For documentation on protocol buffers in general, see: https://protobuf.dev.

For a tutorial on using protocol buffers with Go, see: https://protobuf.dev/getting-started/gotutorial.

For a guide to generated Go protocol buffer code, see: https://protobuf.dev/reference/go/go-generated.

Binary serialization

This package contains functions to convert to and from the wire format, an efficient binary serialization of protocol buffers.

  • Size reports the size of a message in the wire format.

  • Marshal converts a message to the wire format. The MarshalOptions type provides more control over wire marshaling.

  • Unmarshal converts a message from the wire format. The UnmarshalOptions type provides more control over wire unmarshaling.

Basic message operations

Optional scalar constructors

The API for some generated messages represents optional scalar fields as pointers to a value. For example, an optional string field has the Go type *string.

Generated enum types usually have an Enum method which performs the same operation.

Optional scalar fields are only supported in proto2.

Extension accessors

Extension fields are only supported in proto2.

This module contains additional packages for more specialized use cases. Consult the individual package documentation for details.

Index

Examples

Constants

This section is empty.

Variables

View Source 
var Error error

Error matches all errors produced by packages in the protobuf module according to errors.Is.

Example usage: 

if errors.Is(err, proto.Error) { ... }

Functions

func Bool 

func Bool(v bool) *bool

Bool stores v in a new bool value and returns a pointer to it.

func CheckInitialized 

func CheckInitialized(m Message) error

CheckInitialized returns an error if any required fields in m are not set.

func ClearExtension 

func ClearExtension(m Message, xt protoreflect.ExtensionType)

ClearExtension clears an extension field such that subsequent HasExtension calls return false. It panics if m is invalid or if xt does not extend m.

func Equal 

func Equal(x, y Message) bool

Equal reports whether two messages are equal, by recursively comparing the fields of the message.

  • Bytes fields are equal if they contain identical bytes. Empty bytes (regardless of nil-ness) are considered equal.

  • Floating-point fields are equal if they contain the same value. Unlike the == operator, a NaN is equal to another NaN.

  • Other scalar fields are equal if they contain the same value.

  • Message fields are equal if they have the same set of populated known and extension field values, and the same set of unknown fields values.

  • Lists are equal if they are the same length and each corresponding element is equal.

  • Maps are equal if they have the same set of keys and the corresponding value for each key is equal.

An invalid message is not equal to a valid message. An invalid message is only equal to another invalid message of the same type. An invalid message often corresponds to a nil pointer of the concrete message type. For example, (*pb.M)(nil) is not equal to &pb.M{}. If two valid messages marshal to the same bytes under deterministic serialization, then Equal is guaranteed to report true.

func Float32 

func Float32(v float32) *float32

Float32 stores v in a new float32 value and returns a pointer to it.

func Float64 

func Float64(v float64) *float64

Float64 stores v in a new float64 value and returns a pointer to it.

func GetExtension 

func GetExtension(m Message, xt protoreflect.ExtensionType) any

GetExtension retrieves the value for an extension field. If the field is unpopulated, it returns the default value for scalars and an immutable, empty value for lists or messages. It panics if xt does not extend m.

The type of the value is dependent on the field type of the extension. For extensions generated by protoc-gen-go, the Go type is as follows: 

╔═══════════════════╤═════════════════════════╗
║ Go type           │ Protobuf kind           ║
╠═══════════════════╪═════════════════════════╣
║ bool              │ bool                    ║
║ int32             │ int32, sint32, sfixed32 ║
║ int64             │ int64, sint64, sfixed64 ║
║ uint32            │ uint32, fixed32         ║
║ uint64            │ uint64, fixed64         ║
║ float32           │ float                   ║
║ float64           │ double                  ║
║ string            │ string                  ║
║ []byte            │ bytes                   ║
║ protoreflect.Enum │ enum                    ║
║ proto.Message     │ message, group          ║
╚═══════════════════╧═════════════════════════╝

The protoreflect.Enum and proto.Message types are the concrete Go type associated with the named enum or message. Repeated fields are represented using a Go slice of the base element type.

If a generated extension descriptor variable is directly passed to GetExtension, then the call should be followed immediately by a type assertion to the expected output value. For example: 

mm := proto.GetExtension(m, foopb.E_MyExtension).(*foopb.MyMessage)

This pattern enables static analysis tools to verify that the asserted type matches the Go type associated with the extension field and also enables a possible future migration to a type-safe extension API.

Since singular messages are the most common extension type, the pattern of calling HasExtension followed by GetExtension may be simplified to: 

if mm := proto.GetExtension(m, foopb.E_MyExtension).(*foopb.MyMessage); mm != nil {
    ... // make use of mm
}

The mm variable is non-nil if and only if HasExtension reports true.

func HasExtension 

func HasExtension(m Message, xt protoreflect.ExtensionType) bool

HasExtension reports whether an extension field is populated. It returns false if m is invalid or if xt does not extend m.

func Int32 

func Int32(v int32) *int32

Int32 stores v in a new int32 value and returns a pointer to it.

func Int64 

func Int64(v int64) *int64

Int64 stores v in a new int64 value and returns a pointer to it.

func Marshal 

func Marshal(m Message) ([]byte, error)

Marshal returns the wire-format encoding of m.

This is the most common entry point for encoding a Protobuf message.

See the MarshalOptions type if you need more control.

Example

This example illustrates how to marshal (encode) a Protobuf message struct literal into wire-format encoding.

This example hard-codes a duration of 125ns for the illustration of struct fields, but note that you do not need to fill the fields of well-known types like duration.proto yourself. To convert a time.Duration, use google.golang.org/protobuf/types/known/durationpb.New. 

package main

import (
	"fmt"

	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/types/known/durationpb"
)

func main() {
	b, err := proto.Marshal(&durationpb.Duration{
		Nanos: 125,
	})
	if err != nil {
		panic(err)
	}

	fmt.Printf("125ns encoded into %d bytes of Protobuf wire format:\n% x\n", len(b), b)

	// You can use protoscope to explore the wire format:
	// https://github.com/protocolbuffers/protoscope
	//
	// echo -n '10 7d' | xxd -r -ps | protoscope
	// 2: 125

}

Output:

125ns encoded into 2 bytes of Protobuf wire format:
10 7d

func Merge 

func Merge(dst, src Message)

Merge merges src into dst, which must be a message with the same descriptor.

Populated scalar fields in src are copied to dst, while populated singular messages in src are merged into dst by recursively calling Merge. The elements of every list field in src is appended to the corresponded list fields in dst. The entries of every map field in src is copied into the corresponding map field in dst, possibly replacing existing entries. The unknown fields of src are appended to the unknown fields of dst.

It is semantically equivalent to unmarshaling the encoded form of src into dst with the [UnmarshalOptions.Merge] option specified.

func MessageName added in v1.26.0 

func MessageName(m Message) protoreflect.FullName

MessageName returns the full name of m. If m is nil, it returns an empty string.

func RangeExtensions added in v1.22.0 

func RangeExtensions(m Message, f func(protoreflect.ExtensionType, any) bool)

RangeExtensions iterates over every populated extension field in m in an undefined order, calling f for each extension type and value encountered. It returns immediately if f returns false. While iterating, mutating operations may only be performed on the current extension field.

func Reset 

func Reset(m Message)

Reset clears every field in the message. The resulting message shares no observable memory with its previous state other than the memory for the message itself.

func SetExtension 

func SetExtension(m Message, xt protoreflect.ExtensionType, v any)

SetExtension stores the value of an extension field. It panics if m is invalid, xt does not extend m, or if type of v is invalid for the specified extension field.

The type of the value is dependent on the field type of the extension. For extensions generated by protoc-gen-go, the Go type is as follows: 

╔═══════════════════╤═════════════════════════╗
║ Go type           │ Protobuf kind           ║
╠═══════════════════╪═════════════════════════╣
║ bool              │ bool                    ║
║ int32             │ int32, sint32, sfixed32 ║
║ int64             │ int64, sint64, sfixed64 ║
║ uint32            │ uint32, fixed32         ║
║ uint64            │ uint64, fixed64         ║
║ float32           │ float                   ║
║ float64           │ double                  ║
║ string            │ string                  ║
║ []byte            │ bytes                   ║
║ protoreflect.Enum │ enum                    ║
║ proto.Message     │ message, group          ║
╚═══════════════════╧═════════════════════════╝

The protoreflect.Enum and proto.Message types are the concrete Go type associated with the named enum or message. Repeated fields are represented using a Go slice of the base element type.

If a generated extension descriptor variable is directly passed to SetExtension (e.g., foopb.E_MyExtension), then the value should be a concrete type that matches the expected Go type for the extension descriptor so that static analysis tools can verify type correctness. This also enables a possible future migration to a type-safe extension API.

func Size 

func Size(m Message) int

Size returns the size in bytes of the wire-format encoding of m.

Note that Size might return more bytes than Marshal will write in the case of lazily decoded messages that arrive in non-minimal wire format: see https://protobuf.dev/reference/go/size/ for more details.

Example

Checking if Size returns 0 is an easy way to recognize empty messages: 

package main

import (
	"google.golang.org/protobuf/proto"
)

func main() {
	var m proto.Message
	if proto.Size(m) == 0 {
		// No fields set (or, in proto3, all fields matching the default);
		// skip processing this message, or return an error, or similar.
	}
}

Output:

func String 

func String(v string) *string

String stores v in a new string value and returns a pointer to it.

func Uint32 

func Uint32(v uint32) *uint32

Uint32 stores v in a new uint32 value and returns a pointer to it.

func Uint64 

func Uint64(v uint64) *uint64

Uint64 stores v in a new uint64 value and returns a pointer to it.

func Unmarshal 

func Unmarshal(b []byte, m Message) error

Unmarshal parses the wire-format message in b and places the result in m. The provided message must be mutable (e.g., a non-nil pointer to a message).

See the UnmarshalOptions type if you need more control.

Example

This example illustrates how to unmarshal (decode) wire format encoding into a Protobuf message. 

package main

import (
	"fmt"

	"google.golang.org/protobuf/proto"
	"google.golang.org/protobuf/types/known/durationpb"
)

func main() {
	// This is the wire format encoding produced by the Marshal example.
	// Typically you would read from the network, from disk, etc.
	b := []byte{0x10, 0x7d}

	var dur durationpb.Duration
	if err := proto.Unmarshal(b, &dur); err != nil {
		panic(err)
	}

	fmt.Printf("Protobuf wire format decoded to duration %v\n", dur.AsDuration())

}

Output:

Protobuf wire format decoded to duration 125ns

func ValueOrDefault added in v1.36.0 

func ValueOrDefault[T interface {
	*P
	Message
}, P any](val T) T

ValueOrDefault returns the protobuf message val if val is not nil, otherwise it returns a pointer to an empty val message.

This function allows for translating code from the old Open Struct API to the new Opaque API.

The old Open Struct API represented oneof fields with a wrapper struct: 

var signedImg *accountpb.SignedImage
profile := &accountpb.Profile{
	// The Avatar oneof will be set, with an empty SignedImage.
	Avatar: &accountpb.Profile_SignedImage{signedImg},
}

The new Opaque API treats oneof fields like regular fields, there are no more wrapper structs: 

var signedImg *accountpb.SignedImage
profile := &accountpb.Profile{}
profile.SetSignedImage(signedImg)

For convenience, the Opaque API also offers Builders, which allow for a direct translation of struct initialization. However, because Builders use nilness to represent field presence (but there is no non-nil wrapper struct anymore), Builders cannot distinguish between an unset oneof and a set oneof with nil message. The above code would need to be translated with help of the ValueOrDefault function to retain the same behavior: 

var signedImg *accountpb.SignedImage
return &accountpb.Profile_builder{
	SignedImage: proto.ValueOrDefault(signedImg),
}.Build()

func ValueOrDefaultBytes added in v1.36.0 

func ValueOrDefaultBytes(val []byte) []byte

ValueOrDefaultBytes is like ValueOrDefault but for working with fields of type []byte.

func ValueOrNil added in v1.36.0 

func ValueOrNil[T any](has bool, getter func() T) *T

ValueOrNil returns nil if has is false, or a pointer to a new variable containing the value returned by the specified getter.

This function is similar to the wrappers (proto.Int32(), proto.String(), etc.), but is generic (works for any field type) and works with the hasser and getter of a field, as opposed to a value.

This is convenient when populating builder fields.

Example: 

hop := attr.GetDirectHop()
injectedRoute := ripb.InjectedRoute_builder{
  Prefixes: route.GetPrefixes(),
  NextHop:  proto.ValueOrNil(hop.HasAddress(), hop.GetAddress),
}

Types

type MarshalOptions 

type MarshalOptions struct {
	pragma.NoUnkeyedLiterals

	// AllowPartial allows messages that have missing required fields to marshal
	// without returning an error. If AllowPartial is false (the default),
	// Marshal will return an error if there are any missing required fields.
	AllowPartial bool

	// Deterministic controls whether the same message will always be
	// serialized to the same bytes within the same binary.
	//
	// Setting this option guarantees that repeated serialization of
	// the same message will return the same bytes, and that different
	// processes of the same binary (which may be executing on different
	// machines) will serialize equal messages to the same bytes.
	// It has no effect on the resulting size of the encoded message compared
	// to a non-deterministic marshal.
	//
	// Note that the deterministic serialization is NOT canonical across
	// languages. It is not guaranteed to remain stable over time. It is
	// unstable across different builds with schema changes due to unknown
	// fields. Users who need canonical serialization (e.g., persistent
	// storage in a canonical form, fingerprinting, etc.) must define
	// their own canonicalization specification and implement their own
	// serializer rather than relying on this API.
	//
	// If deterministic serialization is requested, map entries will be
	// sorted by keys in lexographical order. This is an implementation
	// detail and subject to change.
	Deterministic bool

	// UseCachedSize indicates that the result of a previous Size call
	// may be reused.
	//
	// Setting this option asserts that:
	//
	// 1. Size has previously been called on this message with identical
	// options (except for UseCachedSize itself).
	//
	// 2. The message and all its submessages have not changed in any
	// way since the Size call. For lazily decoded messages, accessing
	// a message results in decoding the message, which is a change.
	//
	// If either of these invariants is violated,
	// the results are undefined and may include panics or corrupted output.
	//
	// Implementations MAY take this option into account to provide
	// better performance, but there is no guarantee that they will do so.
	// There is absolutely no guarantee that Size followed by Marshal with
	// UseCachedSize set will perform equivalently to Marshal alone.
	UseCachedSize bool
}

MarshalOptions configures the marshaler.

Example usage: 

b, err := MarshalOptions{Deterministic: true}.Marshal(m)

func (MarshalOptions) Marshal 

func (o MarshalOptions) Marshal(m Message) ([]byte, error)

Marshal returns the wire-format encoding of m.

func (MarshalOptions) MarshalAppend 

func (o MarshalOptions) MarshalAppend(b []byte, m Message) ([]byte, error)

MarshalAppend appends the wire-format encoding of m to b, returning the result.

This is a less common entry point than Marshal, which is only needed if you need to supply your own buffers for performance reasons.

Example (SameBuffer)

This example illustrates how to marshal (encode) many Protobuf messages into wire-format encoding, using the same buffer.

MarshalAppend will grow the buffer as needed, so over time it will grow large enough to not need further allocations.

If unbounded growth of the buffer is undesirable in your application, you can use MarshalOptions.Size to determine a buffer size that is guaranteed to be large enough for marshaling without allocations. 

package main

import (
	"google.golang.org/protobuf/proto"
)

func main() {
	var m proto.Message

	opts := proto.MarshalOptions{
		// set e.g. Deterministic: true, if needed
	}

	var buf []byte
	for i := 0; i < 100000; i++ {
		var err error
		buf, err = opts.MarshalAppend(buf[:0], m)
		if err != nil {
			panic(err)
		}
		// cap(buf) will grow to hold the largest m.

		// write buf to disk, network, etc.
	}
}

Output:

func (MarshalOptions) MarshalState 

func (o MarshalOptions) MarshalState(in protoiface.MarshalInput) (protoiface.MarshalOutput, error)

MarshalState returns the wire-format encoding of a message.

This method permits fine-grained control over the marshaler. Most users should use Marshal instead.

func (MarshalOptions) Size 

func (o MarshalOptions) Size(m Message) int

Size returns the size in bytes of the wire-format encoding of m.

Note that Size might return more bytes than Marshal will write in the case of lazily decoded messages that arrive in non-minimal wire format: see https://protobuf.dev/reference/go/size/ for more details.

type Message 

type Message = protoreflect.ProtoMessage

Message is the top-level interface that all messages must implement. It provides access to a reflective view of a message. Any implementation of this interface may be used with all functions in the protobuf module that accept a Message, except where otherwise specified.

This is the v2 interface definition for protobuf messages. The v1 interface definition is github.com/golang/protobuf/proto.Message.

func Clone 

func Clone(m Message) Message

Clone returns a deep copy of m. If the top-level message is invalid, it returns an invalid message as well.

type UnmarshalOptions 

type UnmarshalOptions struct {
	pragma.NoUnkeyedLiterals

	// Merge merges the input into the destination message.
	// The default behavior is to always reset the message before unmarshaling,
	// unless Merge is specified.
	Merge bool

	// AllowPartial accepts input for messages that will result in missing
	// required fields. If AllowPartial is false (the default), Unmarshal will
	// return an error if there are any missing required fields.
	AllowPartial bool

	// If DiscardUnknown is set, unknown fields are ignored.
	DiscardUnknown bool

	// Resolver is used for looking up types when unmarshaling extension fields.
	// If nil, this defaults to using protoregistry.GlobalTypes.
	Resolver interface {
		FindExtensionByName(field protoreflect.FullName) (protoreflect.ExtensionType, error)
		FindExtensionByNumber(message protoreflect.FullName, field protoreflect.FieldNumber) (protoreflect.ExtensionType, error)
	}

	// RecursionLimit limits how deeply messages may be nested.
	// If zero, a default limit is applied.
	RecursionLimit int

	//
	// NoLazyDecoding turns off lazy decoding, which otherwise is enabled by
	// default. Lazy decoding only affects submessages (annotated with [lazy =
	// true] in the .proto file) within messages that use the Opaque API.
	NoLazyDecoding bool
}

UnmarshalOptions configures the unmarshaler.

Example usage: 

err := UnmarshalOptions{DiscardUnknown: true}.Unmarshal(b, m)

func (UnmarshalOptions) Unmarshal 

func (o UnmarshalOptions) Unmarshal(b []byte, m Message) error

Unmarshal parses the wire-format message in b and places the result in m. The provided message must be mutable (e.g., a non-nil pointer to a message).

func (UnmarshalOptions) UnmarshalState 

func (o UnmarshalOptions) UnmarshalState(in protoiface.UnmarshalInput) (protoiface.UnmarshalOutput, error)

UnmarshalState parses a wire-format message and places the result in m.

This method permits fine-grained control over the unmarshaler. Most users should use Unmarshal instead.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.