proto

package
v1.0.0-beta1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 11, 2016 License: BSD-3-Clause, Apache-2.0 Imports: 17 Imported by: 0

Documentation

Overview

Package proto converts data structures to and from the wire format of protocol buffers. It works in concert with the Go source code generated for .proto files by the protocol compiler.

A summary of the properties of the protocol buffer interface for a protocol buffer variable v:

  • Names are turned from camel_case to CamelCase for export.
  • There are no methods on v to set fields; just treat them as structure fields.
  • There are getters that return a field's value if set, and return the field's default value if unset. The getters work even if the receiver is a nil message.
  • The zero value for a struct is its correct initialization state. All desired fields must be set before marshaling.
  • A Reset() method will restore a protobuf struct to its zero state.
  • Non-repeated fields are pointers to the values; nil means unset. That is, optional or required field int32 f becomes F *int32.
  • Repeated fields are slices.
  • Helper functions are available to aid the setting of fields. msg.Foo = proto.String("hello") // set field
  • Constants are defined to hold the default values of all fields that have them. They have the form Default_StructName_FieldName. Because the getter methods handle defaulted values, direct use of these constants should be rare.
  • Enums are given type names and maps from names to values. Enum values are prefixed by the enclosing message's name, or by the enum's type name if it is a top-level enum. Enum types have a String method, and a Enum method to assist in message construction.
  • Nested messages, groups and enums have type names prefixed with the name of the surrounding message type.
  • Extensions are given descriptor names that start with E_, followed by an underscore-delimited list of the nested messages that contain it (if any) followed by the CamelCased name of the extension field itself. HasExtension, ClearExtension, GetExtension and SetExtension are functions for manipulating extensions.
  • Marshal and Unmarshal are functions to encode and decode the wire format.

The simplest way to describe this is to see an example. Given file test.proto, containing 

package example;

enum FOO { X = 17; }

message Test {
  required string label = 1;
  optional int32 type = 2 [default=77];
  repeated int64 reps = 3;
  optional group OptionalGroup = 4 {
    required string RequiredField = 5;
  }
}

The resulting file, test.pb.go, is: 

package example

import proto "github.com/golang/protobuf/proto"
import math "math"

type FOO int32
const (
	FOO_X FOO = 17
)
var FOO_name = map[int32]string{
	17: "X",
}
var FOO_value = map[string]int32{
	"X": 17,
}

func (x FOO) Enum() *FOO {
	p := new(FOO)
	*p = x
	return p
}
func (x FOO) String() string {
	return proto.EnumName(FOO_name, int32(x))
}
func (x *FOO) UnmarshalJSON(data []byte) error {
	value, err := proto.UnmarshalJSONEnum(FOO_value, data)
	if err != nil {
		return err
	}
	*x = FOO(value)
	return nil
}

type Test struct {
	Label            *string             `protobuf:"bytes,1,req,name=label" json:"label,omitempty"`
	Type             *int32              `protobuf:"varint,2,opt,name=type,def=77" json:"type,omitempty"`
	Reps             []int64             `protobuf:"varint,3,rep,name=reps" json:"reps,omitempty"`
	Optionalgroup    *Test_OptionalGroup `protobuf:"group,4,opt,name=OptionalGroup" json:"optionalgroup,omitempty"`
	XXX_unrecognized []byte              `json:"-"`
}
func (m *Test) Reset()         { *m = Test{} }
func (m *Test) String() string { return proto.CompactTextString(m) }
func (*Test) ProtoMessage()    {}
const Default_Test_Type int32 = 77

func (m *Test) GetLabel() string {
	if m != nil && m.Label != nil {
		return *m.Label
	}
	return ""
}

func (m *Test) GetType() int32 {
	if m != nil && m.Type != nil {
		return *m.Type
	}
	return Default_Test_Type
}

func (m *Test) GetOptionalgroup() *Test_OptionalGroup {
	if m != nil {
		return m.Optionalgroup
	}
	return nil
}

type Test_OptionalGroup struct {
	RequiredField *string `protobuf:"bytes,5,req" json:"RequiredField,omitempty"`
}
func (m *Test_OptionalGroup) Reset()         { *m = Test_OptionalGroup{} }
func (m *Test_OptionalGroup) String() string { return proto.CompactTextString(m) }

func (m *Test_OptionalGroup) GetRequiredField() string {
	if m != nil && m.RequiredField != nil {
		return *m.RequiredField
	}
	return ""
}

func init() {
	proto.RegisterEnum("example.FOO", FOO_name, FOO_value)
}

To create and play with a Test object: 

package main

import (
	"log"

	"github.com/golang/protobuf/proto"
	pb "./example.pb"
)

func main() {
	test := &pb.Test{
		Label: proto.String("hello"),
		Type:  proto.Int32(17),
		Optionalgroup: &pb.Test_OptionalGroup{
			RequiredField: proto.String("good bye"),
		},
	}
	data, err := proto.Marshal(test)
	if err != nil {
		log.Fatal("marshaling error: ", err)
	}
	newTest := &pb.Test{}
	err = proto.Unmarshal(data, newTest)
	if err != nil {
		log.Fatal("unmarshaling error: ", err)
	}
	// Now test and newTest contain the same data.
	if test.GetLabel() != newTest.GetLabel() {
		log.Fatalf("data mismatch %q != %q", test.GetLabel(), newTest.GetLabel())
	}
	// etc.
}

Index

Constants

View Source 
const (
	WireVarint     = 0
	WireFixed64    = 1
	WireBytes      = 2
	WireStartGroup = 3
	WireEndGroup   = 4
	WireFixed32    = 5
)

Constants that identify the encoding of a value on the wire.

Variables

View Source 
var ErrMissingExtension = errors.New("proto: missing extension")

ErrMissingExtension is the error returned by GetExtension if the named extension is not in the message.

View Source 
var (

	// ErrNil is the error returned if Marshal is called with nil.
	ErrNil = errors.New("proto: Marshal called with nil")
)
View Source 
var ErrNoMessageTypeId = errors.New("proto does not have a message type ID")

ErrNoMessageTypeId occurs when a protocol buffer does not have a message type ID. A message type ID is required for storing a protocol buffer in a message set.

Functions

func Bool 

func Bool(v bool) *bool

Bool is a helper routine that allocates a new bool value to store v and returns a pointer to it.

func ClearExtension 

func ClearExtension(pb extendableProto, extension *ExtensionDesc)

ClearExtension removes the given extension from pb.

func CompactText 

func CompactText(w io.Writer, pb Message) error

CompactText writes a given protocol buffer in compact text format (one line).

func CompactTextString 

func CompactTextString(pb Message) string

CompactTextString is the same as CompactText, but returns the string directly.

func DecodeVarint 

func DecodeVarint(buf []byte) (x uint64, n int)

DecodeVarint reads a varint-encoded integer from the slice. It returns the integer and the number of bytes consumed, or zero if there is not enough. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func EncodeVarint 

func EncodeVarint(x uint64) []byte

EncodeVarint returns the varint encoding of x. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types. Not used by the package itself, but helpful to clients wishing to use the same encoding.

func EnumName 

func EnumName(m map[int32]string, v int32) string

EnumName is a helper function to simplify printing protocol buffer enums by name. Given an enum map and a value, it returns a useful string.

func Equal 

func Equal(a, b Message) bool

Equal returns true iff protocol buffers a and b are equal. The arguments must both be pointers to protocol buffer structs.

Equality is defined in this way:

  • Two messages are equal iff they are the same type, corresponding fields are equal, unknown field sets are equal, and extensions sets are equal.
  • Two set scalar fields are equal iff their values are equal. If the fields are of a floating-point type, remember that NaN != x for all x, including NaN.
  • Two repeated fields are equal iff their lengths are the same, and their corresponding elements are equal (a "bytes" field, although represented by []byte, is not a repeated field)
  • Two unset fields are equal.
  • Two unknown field sets are equal if their current encoded state is equal.
  • Two extension sets are equal iff they have corresponding elements that are pairwise equal.
  • Every other combination of things are not equal.

The return value is undefined if a and b are not protocol buffers.

func Float32 

func Float32(v float32) *float32

Float32 is a helper routine that allocates a new float32 value to store v and returns a pointer to it.

func Float64 

func Float64(v float64) *float64

Float64 is a helper routine that allocates a new float64 value to store v and returns a pointer to it.

func GetExtension 

func GetExtension(pb extendableProto, extension *ExtensionDesc) (interface{}, error)

GetExtension parses and returns the given extension of pb. If the extension is not present it returns ErrMissingExtension.

func GetExtensions 

func GetExtensions(pb Message, es []*ExtensionDesc) (extensions []interface{}, err error)

GetExtensions returns a slice of the extensions present in pb that are also listed in es. The returned slice has the same length as es; missing extensions will appear as nil elements.

func HasExtension 

func HasExtension(pb extendableProto, extension *ExtensionDesc) bool

HasExtension returns whether the given extension is present in pb.

func Int 

func Int(v int) *int32

Int is a helper routine that allocates a new int32 value to store v and returns a pointer to it, but unlike Int32 its argument value is an int.

func Int32 

func Int32(v int32) *int32

Int32 is a helper routine that allocates a new int32 value to store v and returns a pointer to it.

func Int64 

func Int64(v int64) *int64

Int64 is a helper routine that allocates a new int64 value to store v and returns a pointer to it.

func Marshal 

func Marshal(pb Message) ([]byte, error)

Marshal takes the protocol buffer and encodes it into the wire format, returning the data.

func MarshalMessageSet 

func MarshalMessageSet(m map[int32]Extension) ([]byte, error)

MarshalMessageSet encodes the extension map represented by m in the message set wire format. It is called by generated Marshal methods on protocol buffer messages with the message_set_wire_format option.

func MarshalMessageSetJSON 

func MarshalMessageSetJSON(m map[int32]Extension) ([]byte, error)

MarshalMessageSetJSON encodes the extension map represented by m in JSON format. It is called by generated MarshalJSON methods on protocol buffer messages with the message_set_wire_format option.

func MarshalText 

func MarshalText(w io.Writer, pb Message) error

MarshalText writes a given protocol buffer in text format. The only errors returned are from w.

func MarshalTextString 

func MarshalTextString(pb Message) string

MarshalTextString is the same as MarshalText, but returns the string directly.

func Merge 

func Merge(dst, src Message)

Merge merges src into dst. Required and optional fields that are set in src will be set to that value in dst. Elements of repeated fields will be appended. Merge panics if src and dst are not the same type, or if dst is nil.

func RegisterEnum 

func RegisterEnum(typeName string, unusedNameMap map[int32]string, valueMap map[string]int32)

RegisterEnum is called from the generated code to install the enum descriptor maps into the global table to aid parsing text format protocol buffers.

func RegisterExtension 

func RegisterExtension(desc *ExtensionDesc)

RegisterExtension is called from the generated code.

func RegisterMessageSetType 

func RegisterMessageSetType(m Message, fieldNum int32, name string)

RegisterMessageSetType is called from the generated code.

func RegisteredExtensions 

func RegisteredExtensions(pb Message) map[int32]*ExtensionDesc

RegisteredExtensions returns a map of the registered extensions of a protocol buffer struct, indexed by the extension number. The argument pb should be a nil pointer to the struct type.

func SetDefaults 

func SetDefaults(pb Message)

SetDefaults sets unset protocol buffer fields to their default values. It only modifies fields that are both unset and have defined defaults. It recursively sets default values in any non-nil sub-messages.

func SetExtension 

func SetExtension(pb extendableProto, extension *ExtensionDesc, value interface{}) error

SetExtension sets the specified extension of pb to the specified value.

func SetRawExtension 

func SetRawExtension(base extendableProto, id int32, b []byte)

SetRawExtension is for testing only.

func Size 

func Size(pb Message) (n int)

Size returns the encoded size of a protocol buffer.

func String 

func String(v string) *string

String is a helper routine that allocates a new string value to store v and returns a pointer to it.

func Uint32 

func Uint32(v uint32) *uint32

Uint32 is a helper routine that allocates a new uint32 value to store v and returns a pointer to it.

func Uint64 

func Uint64(v uint64) *uint64

Uint64 is a helper routine that allocates a new uint64 value to store v and returns a pointer to it.

func Unmarshal 

func Unmarshal(buf []byte, pb Message) error

Unmarshal parses the protocol buffer representation in buf and places the decoded result in pb. If the struct underlying pb does not match the data in buf, the results can be unpredictable.

Unmarshal resets pb before starting to unmarshal, so any existing data in pb is always removed. Use UnmarshalMerge to preserve and append to existing data.

func UnmarshalJSONEnum 

func UnmarshalJSONEnum(m map[string]int32, data []byte, enumName string) (int32, error)

UnmarshalJSONEnum is a helper function to simplify recovering enum int values from their JSON-encoded representation. Given a map from the enum's symbolic names to its int values, and a byte buffer containing the JSON-encoded value, it returns an int32 that can be cast to the enum type by the caller.

The function can deal with both JSON representations, numeric and symbolic.

func UnmarshalMerge 

func UnmarshalMerge(buf []byte, pb Message) error

UnmarshalMerge parses the protocol buffer representation in buf and writes the decoded result to pb. If the struct underlying pb does not match the data in buf, the results can be unpredictable.

UnmarshalMerge merges into existing data in pb. Most code should use Unmarshal instead.

func UnmarshalMessageSet 

func UnmarshalMessageSet(buf []byte, m map[int32]Extension) error

UnmarshalMessageSet decodes the extension map encoded in buf in the message set wire format. It is called by generated Unmarshal methods on protocol buffer messages with the message_set_wire_format option.

func UnmarshalMessageSetJSON 

func UnmarshalMessageSetJSON(buf []byte, m map[int32]Extension) error

UnmarshalMessageSetJSON decodes the extension map encoded in buf in JSON format. It is called by generated UnmarshalJSON methods on protocol buffer messages with the message_set_wire_format option.

func UnmarshalText 

func UnmarshalText(s string, pb Message) error

UnmarshalText reads a protocol buffer in Text format. UnmarshalText resets pb before starting to unmarshal, so any existing data in pb is always removed. If a required field is not set and no other error occurs, UnmarshalText returns *RequiredNotSetError.

Types

type Buffer 

type Buffer struct {
	// contains filtered or unexported fields
}

A Buffer is a buffer manager for marshaling and unmarshaling protocol buffers. It may be reused between invocations to reduce memory usage. It is not necessary to use a Buffer; the global functions Marshal and Unmarshal create a temporary Buffer and are fine for most applications.

func NewBuffer 

func NewBuffer(e []byte) *Buffer

NewBuffer allocates a new Buffer and initializes its internal data to the contents of the argument slice.

func (*Buffer) Bytes 

func (p *Buffer) Bytes() []byte

Bytes returns the contents of the Buffer.

func (*Buffer) DebugPrint 

func (o *Buffer) DebugPrint(s string, b []byte)

DebugPrint dumps the encoded data in b in a debugging format with a header including the string s. Used in testing but made available for general debugging.

func (*Buffer) DecodeFixed32 

func (p *Buffer) DecodeFixed32() (x uint64, err error)

DecodeFixed32 reads a 32-bit integer from the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) DecodeFixed64 

func (p *Buffer) DecodeFixed64() (x uint64, err error)

DecodeFixed64 reads a 64-bit integer from the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) DecodeRawBytes 

func (p *Buffer) DecodeRawBytes(alloc bool) (buf []byte, err error)

DecodeRawBytes reads a count-delimited byte buffer from the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages.

func (*Buffer) DecodeStringBytes 

func (p *Buffer) DecodeStringBytes() (s string, err error)

DecodeStringBytes reads an encoded string from the Buffer. This is the format used for the proto2 string type.

func (*Buffer) DecodeVarint 

func (p *Buffer) DecodeVarint() (x uint64, err error)

DecodeVarint reads a varint-encoded integer from the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func (*Buffer) DecodeZigzag32 

func (p *Buffer) DecodeZigzag32() (x uint64, err error)

DecodeZigzag32 reads a zigzag-encoded 32-bit integer from the Buffer. This is the format used for the sint32 protocol buffer type.

func (*Buffer) DecodeZigzag64 

func (p *Buffer) DecodeZigzag64() (x uint64, err error)

DecodeZigzag64 reads a zigzag-encoded 64-bit integer from the Buffer. This is the format used for the sint64 protocol buffer type.

func (*Buffer) EncodeFixed32 

func (p *Buffer) EncodeFixed32(x uint64) error

EncodeFixed32 writes a 32-bit integer to the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) EncodeFixed64 

func (p *Buffer) EncodeFixed64(x uint64) error

EncodeFixed64 writes a 64-bit integer to the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) EncodeRawBytes 

func (p *Buffer) EncodeRawBytes(b []byte) error

EncodeRawBytes writes a count-delimited byte buffer to the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages.

func (*Buffer) EncodeStringBytes 

func (p *Buffer) EncodeStringBytes(s string) error

EncodeStringBytes writes an encoded string to the Buffer. This is the format used for the proto2 string type.

func (*Buffer) EncodeVarint 

func (p *Buffer) EncodeVarint(x uint64) error

EncodeVarint writes a varint-encoded integer to the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func (*Buffer) EncodeZigzag32 

func (p *Buffer) EncodeZigzag32(x uint64) error

EncodeZigzag32 writes a zigzag-encoded 32-bit integer to the Buffer. This is the format used for the sint32 protocol buffer type.

func (*Buffer) EncodeZigzag64 

func (p *Buffer) EncodeZigzag64(x uint64) error

EncodeZigzag64 writes a zigzag-encoded 64-bit integer to the Buffer. This is the format used for the sint64 protocol buffer type.

func (*Buffer) Marshal 

func (p *Buffer) Marshal(pb Message) error

Marshal takes the protocol buffer and encodes it into the wire format, writing the result to the Buffer.

func (*Buffer) Reset 

func (p *Buffer) Reset()

Reset resets the Buffer, ready for marshaling a new protocol buffer.

func (*Buffer) SetBuf 

func (p *Buffer) SetBuf(s []byte)

SetBuf replaces the internal buffer with the slice, ready for unmarshaling the contents of the slice.

func (*Buffer) Unmarshal 

func (p *Buffer) Unmarshal(pb Message) error

Unmarshal parses the protocol buffer representation in the Buffer and places the decoded result in pb. If the struct underlying pb does not match the data in the buffer, the results can be unpredictable.

type Extension 

type Extension struct {
	// contains filtered or unexported fields
}

Extension represents an extension in a message.

type ExtensionDesc 

type ExtensionDesc struct {
	ExtendedType  Message     // nil pointer to the type that is being extended
	ExtensionType interface{} // nil pointer to the extension type
	Field         int32       // field number
	Name          string      // fully-qualified name of extension, for text formatting
	Tag           string      // protobuf tag style
}

ExtensionDesc represents an extension specification. Used in generated code from the protocol compiler.

type ExtensionRange 

type ExtensionRange struct {
	Start, End int32 // both inclusive
}

ExtensionRange represents a range of message extensions for a protocol buffer. Used in code generated by the protocol compiler.

type Marshaler 

type Marshaler interface {
	Marshal() ([]byte, error)
}

Marshaler is the interface representing objects that can marshal themselves.

type Message 

type Message interface {
	Reset()
	String() string
	ProtoMessage()
}

Message is implemented by generated protocol buffer messages.

func Clone 

func Clone(pb Message) Message

Clone returns a deep copy of a protocol buffer.

type MessageSet 

type MessageSet struct {
	Item             []*_MessageSet_Item `protobuf:"group,1,rep"`
	XXX_unrecognized []byte
}

func (*MessageSet) Has 

func (ms *MessageSet) Has(pb Message) bool

func (*MessageSet) Marshal 

func (ms *MessageSet) Marshal(pb Message) error

func (*MessageSet) ProtoMessage 

func (*MessageSet) ProtoMessage()

func (*MessageSet) Reset 

func (ms *MessageSet) Reset()

func (*MessageSet) String 

func (ms *MessageSet) String() string

func (*MessageSet) Unmarshal 

func (ms *MessageSet) Unmarshal(pb Message) error

type ParseError 

type ParseError struct {
	Message string
	Line    int // 1-based line number
	Offset  int // 0-based byte offset from start of input
}

func (*ParseError) Error 

func (p *ParseError) Error() string

type Properties 

type Properties struct {
	Name     string // name of the field, for error messages
	OrigName string // original name before protocol compiler (always set)
	Wire     string
	WireType int
	Tag      int
	Required bool
	Optional bool
	Repeated bool
	Packed   bool   // relevant for repeated primitives only
	Enum     string // set for enum types only

	Default    string // default value
	HasDefault bool   // whether an explicit default was provided
	// contains filtered or unexported fields
}

Properties represents the protocol-specific behavior of a single struct field.

func (*Properties) Init 

func (p *Properties) Init(typ reflect.Type, name, tag string, f *reflect.StructField)

Init populates the properties from a protocol buffer struct tag.

func (*Properties) Parse 

func (p *Properties) Parse(s string)

Parse populates p by parsing a string in the protobuf struct field tag style.

func (*Properties) String 

func (p *Properties) String() string

String formats the properties in the protobuf struct field tag style.

type RequiredNotSetError 

type RequiredNotSetError struct {
	// contains filtered or unexported fields
}

RequiredNotSetError is the error returned if Marshal is called with a protocol buffer struct whose required fields have not all been initialized. It is also the error returned if Unmarshal is called with an encoded protocol buffer that does not include all the required fields.

When printed, RequiredNotSetError reports the first unset required field in a message. If the field cannot be precisely determined, it is reported as "{Unknown}".

func (*RequiredNotSetError) Error 

func (e *RequiredNotSetError) Error() string

type Stats 

type Stats struct {
	Emalloc uint64 // mallocs in encode
	Dmalloc uint64 // mallocs in decode
	Encode  uint64 // number of encodes
	Decode  uint64 // number of decodes
	Chit    uint64 // number of cache hits
	Cmiss   uint64 // number of cache misses
	Size    uint64 // number of sizes
}

Stats records allocation details about the protocol buffer encoders and decoders. Useful for tuning the library itself.

func GetStats 

func GetStats() Stats

GetStats returns a copy of the global Stats structure.

type StructProperties 

type StructProperties struct {
	Prop []*Properties // properties for each field
	// contains filtered or unexported fields
}

StructProperties represents properties for all the fields of a struct. decoderTags and decoderOrigNames should only be used by the decoder.

func GetProperties 

func GetProperties(t reflect.Type) *StructProperties

GetProperties returns the list of properties for the type represented by t. t must represent a generated struct type of a protocol message.

func (*StructProperties) Len 

func (sp *StructProperties) Len() int

func (*StructProperties) Less 

func (sp *StructProperties) Less(i, j int) bool

func (*StructProperties) Swap 

func (sp *StructProperties) Swap(i, j int)

type Unmarshaler 

type Unmarshaler interface {
	Unmarshal([]byte) error
}

Unmarshaler is the interface representing objects that can unmarshal themselves. The method should reset the receiver before decoding starts. The argument points to data that may be overwritten, so implementations should not keep references to the buffer.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.