codec

Documentation

Overview

Package codec contains a reader/write type that assists with encoding and decoding protobuf's binary representation.

The code in this package began as a fork of proto.Buffer but provides additional API to make it more useful to code that needs to dynamically process or produce the protobuf binary format.

Index

• Variables
• func DecodeLengthDelimitedField(fd *desc.FieldDescriptor, bytes []byte, mf MessageFactory) (interface{}, error)
• func DecodeScalarField(fd *desc.FieldDescriptor, v uint64) (interface{}, error)
• func DecodeZigZag32(v uint64) int32
• func DecodeZigZag64(v uint64) int64
• func EncodeZigZag32(v int32) uint64
• func EncodeZigZag64(v int64) uint64
• type Buffer
  • func NewBuffer(buf []byte) *Buffer
  • func (cb *Buffer) Bytes() []byte
  • func (cb *Buffer) DecodeFieldValue(fieldFinder func(int32) *desc.FieldDescriptor, fact MessageFactory) (*desc.FieldDescriptor, interface{}, error)
  • func (cb *Buffer) DecodeFixed32() (x uint64, err error)
  • func (cb *Buffer) DecodeFixed64() (x uint64, err error)
  • func (cb *Buffer) DecodeRawBytes(alloc bool) (buf []byte, err error)
  • func (cb *Buffer) DecodeTagAndWireType() (tag int32, wireType int8, err error)
  • func (cb *Buffer) DecodeVarint() (uint64, error)
  • func (cb *Buffer) EOF() bool
  • func (cb *Buffer) EncodeDelimitedMessage(pm proto.Message) error
  • func (cb *Buffer) EncodeFieldValue(fd *desc.FieldDescriptor, val interface{}) error
  • func (cb *Buffer) EncodeFixed32(x uint64) error
  • func (cb *Buffer) EncodeFixed64(x uint64) error
  • func (cb *Buffer) EncodeMessage(pm proto.Message) error
  • func (cb *Buffer) EncodeRawBytes(b []byte) error
  • func (cb *Buffer) EncodeTagAndWireType(tag int32, wireType int8) error
  • func (cb *Buffer) EncodeVarint(x uint64) error
  • func (cb *Buffer) IsDeterministic() bool
  • func (cb *Buffer) Len() int
  • func (cb *Buffer) Read(dest []byte) (int, error)
  • func (cb *Buffer) ReadGroup(alloc bool) ([]byte, error)
  • func (cb *Buffer) Reset()
  • func (cb *Buffer) SetDeterministic(deterministic bool)
  • func (cb *Buffer) Skip(count int) error
  • func (cb *Buffer) SkipField(wireType int8) error
  • func (cb *Buffer) SkipGroup() error
  • func (cb *Buffer) String() string
  • func (cb *Buffer) Write(data []byte) (int, error)
• type MessageFactory
• type UnknownField

Variables

var ErrBadWireType = codec.ErrBadWireType

ErrBadWireType is returned when decoding a wire-type from a buffer that is not valid.

var ErrOverflow = codec.ErrOverflow

ErrOverflow is returned when an integer is too large to be represented.

var ErrWireTypeEndGroup = errors.New("unexpected wire type: end group")

ErrWireTypeEndGroup is returned from DecodeFieldValue if the tag and wire-type it reads indicates an end-group marker.

Functions

func DecodeLengthDelimitedField

func DecodeLengthDelimitedField(fd *desc.FieldDescriptor, bytes []byte, mf MessageFactory) (interface{}, error)

DecodeLengthDelimitedField extracts a properly-typed value from bytes. The returned value's type will usually be []byte, string, or, for nested messages, the type returned from the given message factory. However, since repeated scalar fields can be length-delimited, when they used packed encoding, it can also return an []interface{}, where each element is a scalar value. Furthermore, it could return a scalar type, not in a slice, if the given field descriptor is not repeated. This is to support cases where a field is changed from optional to repeated. New code may emit a packed repeated representation, but old code still expects a single scalar value. In this case, if the actual data in bytes contains multiple values, only the last value is returned.

func DecodeScalarField

func DecodeScalarField(fd *desc.FieldDescriptor, v uint64) (interface{}, error)

DecodeScalarField extracts a properly-typed value from v. The returned value's type depends on the given field descriptor type. It will be the same type as generated structs use for the field descriptor's type. Enum types will return an int32. If the given field type uses length-delimited encoding (nested messages, bytes, and strings), an error is returned.

func DecodeZigZag32

func DecodeZigZag32(v uint64) int32

DecodeZigZag32 decodes a signed 32-bit integer from the given zig-zag encoded value.

func DecodeZigZag64

func DecodeZigZag64(v uint64) int64

DecodeZigZag64 decodes a signed 64-bit integer from the given zig-zag encoded value.

func EncodeZigZag32

func EncodeZigZag32(v int32) uint64

EncodeZigZag32 does zig-zag encoding to convert the given signed 32-bit integer into a form that can be expressed efficiently as a varint, even for negative values.

func EncodeZigZag64

func EncodeZigZag64(v int64) uint64

EncodeZigZag64 does zig-zag encoding to convert the given signed 64-bit integer into a form that can be expressed efficiently as a varint, even for negative values.

Types

type Buffer

type Buffer codec.Buffer

Buffer is a reader and a writer that wraps a slice of bytes and also provides API for decoding and encoding the protobuf binary format.

Its operation is similar to that of a bytes.Buffer: writing pushes data to the end of the buffer while reading pops data from the head of the buffer. So the same buffer can be used to both read and write.

func NewBuffer

func NewBuffer(buf []byte) *Buffer

NewBuffer creates a new buffer with the given slice of bytes as the buffer's initial contents.

func (*Buffer) Bytes

func (cb *Buffer) Bytes() []byte

Bytes returns the slice of bytes remaining in the buffer. Note that this does not perform a copy: if the contents of the returned slice are modified, the modifications will be visible to subsequent reads via the buffer.

func (*Buffer) DecodeFieldValue

func (cb *Buffer) DecodeFieldValue(fieldFinder func(int32) *desc.FieldDescriptor, fact MessageFactory) (*desc.FieldDescriptor, interface{}, error)

DecodeFieldValue will read a field value from the buffer and return its value and the corresponding field descriptor. The given function is used to lookup a field descriptor by tag number. The given factory is used to instantiate a message if the field value is (or contains) a message value.

On error, the field descriptor and value are typically nil. However, if the error returned is ErrWireTypeEndGroup, the returned value will indicate any tag number encoded in the end-group marker.

If the field descriptor returned is nil, that means that the given function returned nil. This is expected to happen for unrecognized tag numbers. In that case, no error is returned, and the value will be an UnknownField.

func (*Buffer) DecodeFixed32

func (cb *Buffer) DecodeFixed32() (x uint64, err error)

DecodeFixed32 reads a 32-bit integer from the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) DecodeFixed64

func (cb *Buffer) DecodeFixed64() (x uint64, err error)

DecodeFixed64 reads a 64-bit integer from the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) DecodeRawBytes

func (cb *Buffer) DecodeRawBytes(alloc bool) (buf []byte, err error)

DecodeRawBytes reads a count-delimited byte buffer from the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages.

func (*Buffer) DecodeTagAndWireType

func (cb *Buffer) DecodeTagAndWireType() (tag int32, wireType int8, err error)

DecodeTagAndWireType decodes a field tag and wire type from input. This reads a varint and then extracts the two fields from the varint value read.

func (*Buffer) DecodeVarint

func (cb *Buffer) DecodeVarint() (uint64, error)

DecodeVarint reads a varint-encoded integer from the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func (*Buffer) EOF

func (cb *Buffer) EOF() bool

EOF returns true if there are no more bytes remaining to read.

func (*Buffer) EncodeDelimitedMessage

func (cb *Buffer) EncodeDelimitedMessage(pm proto.Message) error

EncodeDelimitedMessage writes the given message to the buffer with a varint-encoded length prefix (the delimiter).

func (*Buffer) EncodeFieldValue

func (cb *Buffer) EncodeFieldValue(fd *desc.FieldDescriptor, val interface{}) error

func (*Buffer) EncodeFixed32

func (cb *Buffer) EncodeFixed32(x uint64) error

EncodeFixed32 writes a 32-bit integer to the Buffer. This is the format for the fixed32, sfixed32, and float protocol buffer types.

func (*Buffer) EncodeFixed64

func (cb *Buffer) EncodeFixed64(x uint64) error

EncodeFixed64 writes a 64-bit integer to the Buffer. This is the format for the fixed64, sfixed64, and double protocol buffer types.

func (*Buffer) EncodeMessage

func (cb *Buffer) EncodeMessage(pm proto.Message) error

EncodeMessage writes the given message to the buffer.

func (*Buffer) EncodeRawBytes

func (cb *Buffer) EncodeRawBytes(b []byte) error

EncodeRawBytes writes a count-delimited byte buffer to the Buffer. This is the format used for the bytes protocol buffer type and for embedded messages.

func (*Buffer) EncodeTagAndWireType

func (cb *Buffer) EncodeTagAndWireType(tag int32, wireType int8) error

EncodeTagAndWireType encodes the given field tag and wire type to the buffer. This combines the two values and then writes them as a varint.

func (*Buffer) EncodeVarint

func (cb *Buffer) EncodeVarint(x uint64) error

EncodeVarint writes a varint-encoded integer to the Buffer. This is the format for the int32, int64, uint32, uint64, bool, and enum protocol buffer types.

func (*Buffer) IsDeterministic

func (cb *Buffer) IsDeterministic() bool

IsDeterministic returns whether or not this buffer is configured to encode messages deterministically.

func (*Buffer) Len

func (cb *Buffer) Len() int

Len returns the remaining number of bytes in the buffer.

func (*Buffer) Read

func (cb *Buffer) Read(dest []byte) (int, error)

Read implements the io.Reader interface. If there are no bytes remaining in the buffer, it will return 0, io.EOF. Otherwise, it reads max(len(dest), cb.Len()) bytes from input and copies them into dest. It returns the number of bytes copied and a nil error in this case.

func (*Buffer) ReadGroup

func (cb *Buffer) ReadGroup(alloc bool) ([]byte, error)

ReadGroup reads the input until a "group end" tag is found and returns the data up to that point. Subsequent reads from the buffer will read data after the group end tag. If alloc is true, the data is copied to a new slice before being returned. Otherwise, the returned slice is a view into the buffer's underlying byte slice.

This function correctly handles nested groups: if a "group start" tag is found, then that group's end tag will be included in the returned data.

func (*Buffer) Reset

func (cb *Buffer) Reset()

Reset resets this buffer back to empty. Any subsequent writes/encodes to the buffer will allocate a new backing slice of bytes.

func (*Buffer) SetDeterministic

func (cb *Buffer) SetDeterministic(deterministic bool)

SetDeterministic sets this buffer to encode messages deterministically. This is useful for tests. But the overhead is non-zero, so it should not likely be used outside of tests. When true, map fields in a message must have their keys sorted before serialization to ensure deterministic output. Otherwise, values in a map field will be serialized in map iteration order.

func (*Buffer) Skip

func (cb *Buffer) Skip(count int) error

Skip attempts to skip the given number of bytes in the input. If the input has fewer bytes than the given count, io.ErrUnexpectedEOF is returned and the buffer is unchanged. Otherwise, the given number of bytes are skipped and nil is returned.

func (*Buffer) SkipField

func (cb *Buffer) SkipField(wireType int8) error

SkipField attempts to skip the value of a field with the given wire type. When consuming a protobuf-encoded stream, it can be called immediately after DecodeTagAndWireType to discard the subsequent data for the field.

func (*Buffer) SkipGroup

func (cb *Buffer) SkipGroup() error

SkipGroup is like ReadGroup, except that it discards the data and just advances the buffer to point to the input right *after* the "group end" tag.

func (*Buffer) String

func (cb *Buffer) String() string

String returns the remaining bytes in the buffer as a string.

func (*Buffer) Write

func (cb *Buffer) Write(data []byte) (int, error)

Write implements the io.Writer interface. It always returns len(data), nil.

type MessageFactory

type MessageFactory interface {
	NewMessage(md *desc.MessageDescriptor) proto.Message
}

MessageFactory is used to instantiate messages when DecodeFieldValue needs to decode a message value.

Also see MessageFactory in "github.com/jhump/protoreflect/dynamic", which implements this interface.

type UnknownField

type UnknownField struct {
	// The tag number for the unrecognized field.
	Tag int32

	// Encoding indicates how the unknown field was encoded on the wire. If it
	// is proto.WireBytes or proto.WireGroupStart then Contents will be set to
	// the raw bytes. If it is proto.WireTypeFixed32 then the data is in the least
	// significant 32 bits of Value. Otherwise, the data is in all 64 bits of
	// Value.
	Encoding int8
	Contents []byte
	Value    uint64
}

UnknownField represents a field that was parsed from the binary wire format for a message, but was not a recognized field number. Enough information is preserved so that re-serializing the message won't lose any of the unrecognized data.