pack

README

Proto-Pack format Version 1.1

File

name	type	description
magic	byte[9]	'p', 'r', 'o', 't', 'o', 'p', 'a', 'c', 'k'
header	Chunk<Header>	Header chunk
data	Chunk<Section>[]	File content

Chunk<T>

name	type	description
size	varint	Size of the chunk data
data	byte[size]	T data

Section

A section is either a type or a object:

Group object (tag < 0)

Encoding of an object instance. May be a child object of a previously encoded object.

name	type	description
tag	zig-zag varint	Type index -tag - 1
parent	varint	Relative parent group index + 1. 0 indicates no parent
data	byte[]	Proto data

Group object finalizer (tag = 0)

Indicates that the specified object has no more children. Once a group has been encoded, the global group counter is incremented. This counter is used to reference parent groups.

name	type	description
tag	zig-zag varint	0
object	varint	Relative group index that's being finalized

Type (tag = 1)

Declaration of a new type. The type is described using a DescriptorProto. Types are referenced by index of their declaration order in the stream.

name	type	description
tag	zig-zag varint	1
name	varstr	Proto type name
desc	DescriptorProto	Proto type descriptor

Object (tag >= 2)

Encoding of an object instance. May be a child object of a previously encoded object.

name	type	description
tag	zig-zag varint	Type index tag - 2
parent	varint	Relative parent group index + 1. 0 indicates no parent
data	byte[]	Proto data

Documentation

Overview

Package pack provides methods to deal with self describing files of proto data.

The file format consists of a magic marker, followed by a Header. After that is a repeated sequence of uvarint length, tag and matching encoded message pair. Some section tags will also be followed by a string. The tag 0 is special, and marks a type entry, the body will be a descriptor.DescriptorProto.

Index

• Constants
• Variables
• func Read(ctx context.Context, from io.Reader, events Events) error
• type Dynamic
  • func (d Dynamic) Format(f fmt.State, r rune)
  • func (Dynamic) ProtoMessage()
  • func (d *Dynamic) Reset()
  • func (d Dynamic) String() string
  • func (d *Dynamic) Unmarshal(data []byte) error
• type ErrUnknownType
  • func (e ErrUnknownType) Error() string
• type ErrUnsupportedVersion
  • func (e ErrUnsupportedVersion) Error() string
• type Events
• type Writer
  • func NewWriter(to io.Writer) (*Writer, error)
  • func (w *Writer) BeginChildGroup(ctx context.Context, msg proto.Message, parentID uint64) (id uint64, err error)
  • func (w *Writer) BeginGroup(ctx context.Context, msg proto.Message) (id uint64, err error)
  • func (w *Writer) ChildObject(ctx context.Context, msg proto.Message, parentID uint64) error
  • func (w *Writer) EndGroup(ctx context.Context, id uint64) error
  • func (w *Writer) Object(ctx context.Context, msg proto.Message) error

Constants

const (
	// Magic is the file magic that prefixes all pack files.
	Magic = "protopack"

	// ErrIncorrectMagic is the error returned when the file header is not matched.
	ErrIncorrectMagic = fault.Const("Incorrect pack magic header")
)

Variables

var (
	// MinVersion is the current minimum supported version of pack files.
	MinVersion = &Version{1, 1}

	// MaxVersion is the current maximum supported version of pack files.
	MaxVersion = &Version{1, 1}
)

Functions

func Read

func Read(ctx context.Context, from io.Reader, events Events) error

Read reads the pack file from the supplied stream. This function will read the header from the stream, adjusting it's position. It may read extra bytes from the stream into an internal buffer.

Types

type Dynamic

type Dynamic struct {
	Desc   *descriptor.DescriptorProto
	Fields map[string]interface{}
	// contains filtered or unexported fields
}

Dynamic is a dynamic proto message.

func (Dynamic) Format

func (d Dynamic) Format(f fmt.State, r rune)

Format implements fmt.Formatter to print the full value chain.

func (Dynamic) ProtoMessage

func (Dynamic) ProtoMessage()

func (*Dynamic) Reset

func (d *Dynamic) Reset()

func (Dynamic) String

func (d Dynamic) String() string

func (*Dynamic) Unmarshal

func (d *Dynamic) Unmarshal(data []byte) error

type ErrUnknownType

type ErrUnknownType struct{ TypeName string }

ErrUnknownType is the error returned by Reader.Unmarshal() when it encounters an unknown proto type.

func (ErrUnknownType) Error

func (e ErrUnknownType) Error() string

type ErrUnsupportedVersion

type ErrUnsupportedVersion struct{ Version Version }

ErrUnsupportedVersion is the error returned when the header version is one this package cannot handle.

func (ErrUnsupportedVersion) Error

func (e ErrUnsupportedVersion) Error() string

type Events

type Events interface {
	// BeginGroup is called to start a new root group with the given identifier.
	BeginGroup(ctx context.Context, msg proto.Message, id uint64) error

	// BeginChildGroup is called to start a new group with the given identifier
	// as a child of the group with the parent identifier.
	BeginChildGroup(ctx context.Context, msg proto.Message, id, parentID uint64) error

	// EndGroup finalizes the group with the given identifier. It is illegal to
	// attempt to add children to the group after this is called.
	// EndGroup should be closed immediately after the last child has been added
	// to the group.
	EndGroup(ctx context.Context, id uint64) error

	// Object is called to declare an object outside of any group.
	Object(ctx context.Context, msg proto.Message) error

	// ChildObject is called to declare an object in the group with the given
	// identifier.
	ChildObject(ctx context.Context, msg proto.Message, parentID uint64) error
}

Events describes the events used to construct groups and objects that are stored in a proto-pack stream.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer is the type for a pack file writer. They should only be constructed by NewWriter.

func NewWriter

func NewWriter(to io.Writer) (*Writer, error)

NewWriter constructs and returns a new Writer that writes to the supplied output stream. This method will write the packfile magic and header to the underlying stream.

func (*Writer) BeginChildGroup

func (w *Writer) BeginChildGroup(ctx context.Context, msg proto.Message, parentID uint64) (id uint64, err error)

BeginChildGroup is called to start a new group as a child of the group with the parent identifier.

func (*Writer) BeginGroup

func (w *Writer) BeginGroup(ctx context.Context, msg proto.Message) (id uint64, err error)

BeginGroup is called to start a new root group.

func (*Writer) ChildObject

func (w *Writer) ChildObject(ctx context.Context, msg proto.Message, parentID uint64) error

ChildObject is called to declare an object in the group with the given identifier.

func (*Writer) EndGroup

func (w *Writer) EndGroup(ctx context.Context, id uint64) error

EndGroup finalizes the group with the given identifier. It is illegal to attempt to add children to the group after this is called. EndGroup should be closed immediately after the last child has been added to the group.

func (*Writer) Object

func (w *Writer) Object(ctx context.Context, msg proto.Message) error

Object is called to declare an object outside of any group.