Documentation ¶
Overview ¶
Package capn is a capnproto library for go
see http://kentonv.github.io/capnproto/
capnpc-go provides the compiler backend for capnp after installing to $PATH capnp files can be compiled with
capnp compile -ogo *.capnp
capnpc-go requires two annotations for all types. This is the package and import found in go.capnp. Package is needed to know what package to place at the head of the generated file and what go name to use when referring to the type from another package. Import should be the fully qualified import path and is used to generate import statement from other packages and to detect when two types are in the same package. Typically these are added as file annotations. For example:
using Go = import "github.com/glycerine/go-capnproto/go.capnp"; $Go.package("main"); $Go.import("github.com/glycerine/go-capnproto/example");
In capnproto, the unit of communication is a message. A message consists of one or more of segments to allow easier allocation, but ideally and typically you just make one segment per message.
Logically, a message organized in a tree of objects, with the root always being a struct (as opposed to a list or primitive).
Here is an example of writing a new message. We use the demo schema aircraft.capnp from the aircraftlib directory. You may wish to read the schema before reading this example.
<< Example moved to its own file: See the file, write_test.go >>
In summary, when you make a new message, you should first make new segment, and then create the root struct in that segment. Then add your non-child (contained) objects. This is because, as the spec says:
The first word of the first segment of the message is always a pointer pointing to the message's root struct.
All objects are values with pointer semantics that point into the data in a message or segment. Messages can be read/written from a stream uncompressed or using the capnproto compression.
In this library a *Segment is taken to refer to both a specific segment as well as the containing message. This is to reduce the number of types generic code needs to deal with and allows objects to be created in the same segment as their outer object (thus reducing the number of far pointers).
Most getters/setters in the library don't return an error. Instead a get that fails due to an invalid pointer, out of bounds, etc will return the default value. A invalid set will be noop'ed. If you really need to know whether a set succeeds then errors are provided by the lower level Object methods.
Since go doesn't have any templating, lists are created for the basic types and one level of named types. The list of basic types (e.g. List(UInt8), List(Text), etc) are all provided in this library. Lists of user named types are created with the user types (e.g. user struct Foo will create a Foo_List type). capnp schemas that use deeper lists (e.g. List(List(UInt8))) will use PointerList and the user will have to use the Object.ToList* functions to cast to the correct type.
For adding documentation comments to the generated code, there's the doc annotation. This annotation adds the comment to a struct, enum or field so that godoc will pick it up. For Example:
struct Zdate $Go.doc("Zdate represents an instance in time") { year @0 :Int16; month @1 :UInt8; day @2 :UInt8 ; }
Structs ¶
capnpc-go will generate the following for structs:
// Foo is a value with pointer semantics referencing the data in a // segment. Member functions are provided to get/set members in the // struct. Getters/setters of an outer struct will use values of type // Foo to set/get pointers. type Foo capn.Struct // NewFoo creates a new orphaned Foo struct. This can then be added to // a message by using a Set function which takes a Foo argument. func NewFoo(s *capn.Segment) Foo // NewRootFoo creates a new root of type Foo in the next unused space in the // provided segment. This is distinct from NewFoo as this always // creates a root tag. Typically the provided segment should be empty. // Remember that a message is a tree of objects with a single root, and // you usually have to create the root before any other object in a // segment. The only exception would be for a multi-segment message. func NewRootFoo(s *capn.Segment) Foo // ReadRootFoo reads the root tag at the beginning of the provided // segment and returns it as a Foo struct. func ReadRootFoo(s *capn.Segment) Foo // Foo_List is a value with pointer semantics. It is created for all // structs, and is used for List(Foo) in the capnp file. type Foo_List capn.List // NewFooList creates a new orphaned List(Foo). This can then be added // to a message by using a Set function which takes a Foo_List. sz // specifies the list size. Due to the list using memory directly in // the outgoing buffer (i.e. arena style memory management), the size // can not be changed after creation. func NewFooList(s *capn.Segment, sz int) Foo_List // Len returns the list length. For composite lists this is the number // of list elements. func (s Foo_List) Len() int // At returns a pointer to the i'th element. If i is an invalid index, // this will return a null Foo (all getters will return default // values, setters will fail). For a composite list the returned value // will be a list member. Setting another value to point to list // members forces a copy of the data. For pointer lists, the pointer // value will be auto-derefenced. func (s Foo_List) At(i int) Foo // ToArray converts the capnproto list into a go list. For large lists // this is inefficient as it has to read all elements. This can be // quite convenient especially for iterating as it lets you use a for // range clause: // for i, f := range mylist.ToArray() {} func (s Foo_List) ToArray() []Foo
Groups ¶
For each group a typedef is created with a different method set for just the groups fields:
struct Foo { group :Group { field @0 :Bool; } } type Foo capn.Struct type FooGroup Foo func (s Foo) Group() FooGroup func (s FooGroup) Field() bool
That way the following may be used to access a field in a group:
var f Foo value := f.Group().Field()
Note that Group accessors just cast the type and so have no overhead
func (s Foo) Group() FooGroup {return FooGroup(s)}
Unions ¶
Named unions are treated as a group with an inner unnamed union. Unnamed unions generate an enum Type_Which and a corresponding Which() function:
struct Foo { union { a @0 :Bool; b @1 :Bool; } } type Foo_Which uint16 const ( FOO_A Foo_Which = 0 FOO_B = 1 ) func (s Foo) A() bool func (s Foo) B() bool func (s Foo) SetA(v bool) func (s Foo) SetB(v bool) func (s Foo) Which() Foo_Which
Which() should be checked before using the getters, and the default case must always be handled.
Setters for single values will set the union discriminator as well as set the value.
For voids in unions, there is a void setter that just sets the discriminator. For example:
struct Foo { union { a @0 :Void; b @1 :Void; } } f.SetA() // Set that we are using A f.SetB() // Set that we are using B
For groups in unions, there is a group setter that just sets the discriminator. This must be called before the group getter can be used to set values. For example:
struct Foo { union { a :group { v :Bool } b :group { v :Bool } } } f.SetA() // Set that we are using group A f.A().SetV(true) // then we can use the group A getter to set the inner values
Enums ¶
capnpc-go generates enum values in all caps. For example in the capnp file:
enum ElementSize { empty @0; bit @1; byte @2; twoBytes @3; fourBytes @4; eightBytes @5; pointer @6; inlineComposite @7; }
In the generated capnp.go file:
type ElementSize uint16 const ( ELEMENTSIZE_EMPTY ElementSize = 0 ELEMENTSIZE_BIT = 1 ELEMENTSIZE_BYTE = 2 ELEMENTSIZE_TWOBYTES = 3 ELEMENTSIZE_FOURBYTES = 4 ELEMENTSIZE_EIGHTBYTES = 5 ELEMENTSIZE_POINTER = 6 ELEMENTSIZE_INLINECOMPOSITE = 7 )
In addition an enum.String() function is generated that will convert the constants to a string for debugging or logging purposes. By default, the enum name is used as the tag value, but the tags can be customized with a $Go.tag or $Go.notag annotation.
For example:
enum ElementSize { empty @0 $Go.tag("void"); bit @1 $Go.tag("1 bit"); byte @2 $Go.tag("8 bits"); inlineComposite @7 $Go.notag; }
In the generated go file:
func (c ElementSize) String() string { switch c { case ELEMENTSIZE_EMPTY: return "void" case ELEMENTSIZE_BIT: return "1 bit" case ELEMENTSIZE_BYTE: return "8 bits" default: return "" } }
Index ¶
- Constants
- Variables
- func A(val uint64) int
- func B(val uint64) int
- func CopyToFrom(dest, src Object) error
- func ListC(val uint64) int
- func ListCString(val uint64) string
- func ListD(val uint64) int
- func ReadFromMemoryZeroCopyNoAlloc(data []byte, multi *MultiBuffer) (bytesRead int64, err error)
- func StructC(val uint64) int
- func StructD(val uint64) int
- type BitList
- type Compressor
- type DataList
- type DecompParseState
- type Decompressor
- type Float32List
- type Float64List
- type Int16List
- type Int32List
- type Int64List
- type Int8List
- type Message
- type MultiBuffer
- type Object
- func (o Object) DupWithOff(off int) Object
- func (p Object) HasData() bool
- func (p Object) ToBitList() BitList
- func (p Object) ToData() []byte
- func (p Object) ToDataDefault(def []byte) []byte
- func (p Object) ToDataDefaultTrimLastByte(def []byte) []byte
- func (p Object) ToDataList() DataList
- func (p Object) ToDataTrimLastByte() []byte
- func (p Object) ToFloat32List() Float32List
- func (p Object) ToFloat64List() Float64List
- func (p Object) ToInt16List() Int16List
- func (p Object) ToInt32List() Int32List
- func (p Object) ToInt64List() Int64List
- func (p Object) ToInt8List() Int8List
- func (p Object) ToListDefault(s *Segment, tagoff int) Object
- func (p Object) ToObjectDefault(s *Segment, tagoff int) Object
- func (p Object) ToPointerList() PointerList
- func (p Object) ToStruct() Struct
- func (p Object) ToStructDefault(s *Segment, tagoff int) Struct
- func (p Object) ToText() string
- func (p Object) ToTextDefault(def string) string
- func (p Object) ToTextList() TextList
- func (p Object) ToUInt16List() UInt16List
- func (p Object) ToUInt32List() UInt32List
- func (p Object) ToUInt64List() UInt64List
- func (p Object) ToUInt8List() UInt8List
- func (p Object) ToVoidList() VoidList
- func (p Object) Type() ObjectType
- type ObjectType
- type PointerList
- type Segment
- func NewBuffer(data []byte) *Segment
- func NewMultiBuffer(data [][]byte) *Segment
- func ReadFromMemoryZeroCopy(data []byte) (seg *Segment, bytesRead int64, err error)
- func ReadFromPackedStream(r io.Reader, buf *bytes.Buffer) (*Segment, error)
- func ReadFromStream(r io.Reader, buf *bytes.Buffer) (*Segment, error)
- func (s *Segment) NewBitList(sz int) BitList
- func (s *Segment) NewCompositeList(datasz, ptrs, length int) PointerList
- func (s *Segment) NewData(v []byte) Object
- func (s *Segment) NewDataList(sz int) DataList
- func (s *Segment) NewFloat32List(sz int) Float32List
- func (s *Segment) NewFloat64List(sz int) Float64List
- func (s *Segment) NewInt16List(sz int) Int16List
- func (s *Segment) NewInt32List(sz int) Int32List
- func (s *Segment) NewInt64List(sz int) Int64List
- func (s *Segment) NewInt8List(sz int) Int8List
- func (s *Segment) NewPointerList(sz int) PointerList
- func (s *Segment) NewRoot() (PointerList, int, error)
- func (s *Segment) NewRootStruct(datasz, ptrs int) Struct
- func (s *Segment) NewStruct(datasz, ptrs int) Struct
- func (s *Segment) NewStructAR(datasz, ptrs int) Struct
- func (s *Segment) NewText(v string) Object
- func (s *Segment) NewTextList(sz int) TextList
- func (s *Segment) NewUInt16List(sz int) UInt16List
- func (s *Segment) NewUInt32List(sz int) UInt32List
- func (s *Segment) NewUInt64List(sz int) UInt64List
- func (s *Segment) NewUInt8List(sz int) UInt8List
- func (s *Segment) NewVoidList(sz int) VoidList
- func (s *Segment) Root(off int) Object
- func (s *Segment) WriteTo(w io.Writer) (int64, error)
- func (s *Segment) WriteToPacked(w io.Writer) (int64, error)
- type Struct
- func (p Struct) Get1(bitoff int) bool
- func (p Struct) Get16(off int) uint16
- func (p Struct) Get32(off int) uint32
- func (p Struct) Get64(off int) uint64
- func (p Struct) Get8(off int) uint8
- func (p Struct) GetObject(off int) Object
- func (p Struct) Set1(bitoff int, v bool)
- func (p Struct) Set16(off int, v uint16)
- func (p Struct) Set32(off int, v uint32)
- func (p Struct) Set64(off int, v uint64)
- func (p Struct) Set8(off int, v uint8)
- func (p Struct) SetObject(i int, src Object)
- type TextList
- type UInt16List
- type UInt32List
- type UInt64List
- type UInt8List
- type Void
- type VoidList
Examples ¶
Constants ¶
const ( S_NORMAL DecompParseState = 0 // The 1-3 states are for dealing with the 0xFF tag and the raw bytes that follow. // They tell us where to pick up if we are interrupted in the middle of anything // after the 0xFF tag, until we are done with the raw read. S_POSTFF = 1 S_READN = 2 S_RAW = 3 )
const CanonicalizableOn = true
const Customtype = uint64(0xfa10659ae02f2093)
const Doc = uint64(0xc58ad6bd519f935e)
const Import = uint64(0xe130b601260e44b5)
const JSON_enabled = true
If you want to omit the json support in the generated code, to save space, it can be disabled here.
const Name = uint64(0xc2b96012172f8df1)
const Notag = uint64(0xc8768679ec52e012)
const Package = uint64(0xbea97f1023792be0)
const Tag = uint64(0xa574b41924caefc7)
const VerboseCompress = false
const VerboseDecomp = false
externally available flag for compiling with debug info on/off
Variables ¶
var ( ErrOverlarge = errors.New("capn: overlarge struct/list") ErrOutOfBounds = errors.New("capn: write out of bounds") ErrCopyDepth = errors.New("capn: copy depth too large") ErrOverlap = errors.New("capn: overlapping data on copy") )
var ( ErrInvalidSegment = errors.New("capn: invalid segment id") ErrTooMuchData = errors.New("capn: too much data in stream") )
var ( MaxSegmentNumber = 1024 MaxTotalSize = 1024 * 1024 * 1024 )
var Zerohi32 uint64
used in orable30BitOffsetPart() and signedOffsetFromStructPointer()
Functions ¶
func CopyToFrom ¶
For manually copying between segments. Not typically needed.
func ListCString ¶
func ReadFromMemoryZeroCopyNoAlloc ¶
func ReadFromMemoryZeroCopyNoAlloc(data []byte, multi *MultiBuffer) (bytesRead int64, err error)
ReadFromMemoryZeroCopyNoAlloc: like ReadFromMemoryZeroCopy, but avoid all allocations so we get zero GC pressure.
This requires some strict but easy to meet pre-requisites:
PRE: the capnp bytes in data must come from only one segment. Else we panic. PRE: multi must point to an existing MultiBuffer that has exactly one Segment
that will be re-used and over-written. If in doubt, you can allocate a correct new one the first time by calling NewSingleSegmentMultiBuffer().
Types ¶
type Compressor ¶
type Compressor struct {
// contains filtered or unexported fields
}
func NewCompressor ¶
func NewCompressor(w io.Writer) *Compressor
type DecompParseState ¶
type DecompParseState uint8
type Decompressor ¶
type Decompressor struct {
// contains filtered or unexported fields
}
func NewDecompressor ¶
func NewDecompressor(r io.Reader) *Decompressor
type Float32List ¶
type Float32List Object
func (Float32List) At ¶
func (p Float32List) At(i int) float32
func (Float32List) Len ¶
func (p Float32List) Len() int
func (Float32List) Set ¶
func (p Float32List) Set(i int, v float32)
func (Float32List) ToArray ¶
func (p Float32List) ToArray() []float32
type Float64List ¶
type Float64List Object
func (Float64List) At ¶
func (p Float64List) At(i int) float64
func (Float64List) Len ¶
func (p Float64List) Len() int
func (Float64List) Set ¶
func (p Float64List) Set(i int, v float64)
func (Float64List) ToArray ¶
func (p Float64List) ToArray() []float64
type MultiBuffer ¶
type MultiBuffer struct {
Segments []*Segment
}
func NewSingleSegmentMultiBuffer ¶
func NewSingleSegmentMultiBuffer() *MultiBuffer
func (*MultiBuffer) NewSegment ¶
func (m *MultiBuffer) NewSegment(minsz int) (*Segment, error)
type Object ¶
type Object struct { Segment *Segment // contains filtered or unexported fields }
func (Object) DupWithOff ¶
func (Object) ToDataDefault ¶
func (Object) ToDataDefaultTrimLastByte ¶
func (Object) ToDataList ¶
func (Object) ToDataTrimLastByte ¶
func (Object) ToFloat32List ¶
func (p Object) ToFloat32List() Float32List
func (Object) ToFloat64List ¶
func (p Object) ToFloat64List() Float64List
func (Object) ToInt16List ¶
func (Object) ToInt32List ¶
func (Object) ToInt64List ¶
func (Object) ToInt8List ¶
func (Object) ToPointerList ¶
func (p Object) ToPointerList() PointerList
func (Object) ToTextDefault ¶
func (Object) ToTextList ¶
func (Object) ToUInt16List ¶
func (p Object) ToUInt16List() UInt16List
func (Object) ToUInt32List ¶
func (p Object) ToUInt32List() UInt32List
func (Object) ToUInt64List ¶
func (p Object) ToUInt64List() UInt64List
func (Object) ToUInt8List ¶
func (Object) ToVoidList ¶
func (Object) Type ¶
func (p Object) Type() ObjectType
type ObjectType ¶
type ObjectType uint8
const ( TypeNull ObjectType = iota TypeStruct TypeList TypePointerList TypeBitList )
func (ObjectType) String ¶
func (o ObjectType) String() string
type PointerList ¶
type PointerList Object
func (PointerList) At ¶
func (p PointerList) At(i int) Object
func (PointerList) Len ¶
func (p PointerList) Len() int
func (PointerList) ToArray ¶
func (p PointerList) ToArray() *[]Object
type Segment ¶
func NewBuffer ¶
NewBuffer creates an expanding single segment buffer. Creating new objects will expand the buffer. Data can be nil (or length 0 with some capacity) if creating a new session. If parsing an existing segment then data should be the segment contents and will not be copied.
func NewMultiBuffer ¶
NewMultiBuffer creates a new multi segment message. Creating new objects will try and reuse the buffers available, but will create new ones if there is insufficient capacity. When parsing an existing message data should be the list of segments. The data buffers will not be copied.
func ReadFromMemoryZeroCopy ¶
ReadFromMemoryZeroCopy: like ReadFromStream, but reads a non-packed serialized stream that already resides in memory in the argument data. The returned segment is the first segment read, which contains the root pointer. The returned bytesRead says how many bytes were consumed from data in making seg. The caller should advance the data slice by doing data = data[bytesRead:] between successive calls to ReadFromMemoryZeroCopy().
func ReadFromPackedStream ¶
ReadFromPackedStream reads a single message from the stream r in packed form returning the first segment. buf can be specified in order to reuse the buffer (or it is allocated each call if nil).
func ReadFromStream ¶
ReadFromStream reads a non-packed serialized stream from r. buf is used to buffer the read contents, can be nil, and is provided so that the buffer can be reused between messages. The returned segment is the first segment read, which contains the root pointer.
Warning about buf reuse: It is safer to just pass nil for buf. When making multiple calls to ReadFromStream() with the same buf argument, you may overwrite the data in a previously returned Segment. The re-use of buf is an optimization for when you are actually done with any previously returned Segment which may have data still alive in buf.
Example ¶
package main import ( "bytes" "encoding/hex" "fmt" capn "github.com/glycerine/go-capnproto" air "github.com/glycerine/go-capnproto/aircraftlib" ) func main() { s := capn.NewBuffer(nil) d := air.NewRootZdate(s) d.SetYear(2004) d.SetMonth(12) d.SetDay(7) buf := bytes.Buffer{} s.WriteTo(&buf) fmt.Println(hex.EncodeToString(buf.Bytes())) // Read s, err := capn.ReadFromStream(&buf, nil) if err != nil { fmt.Printf("read error %v\n", err) return } d = air.ReadRootZdate(s) fmt.Printf("year %d, month %d, day %d\n", d.Year(), d.Month(), d.Day()) }
Output:
func (*Segment) NewBitList ¶
func (*Segment) NewCompositeList ¶
func (s *Segment) NewCompositeList(datasz, ptrs, length int) PointerList
func (*Segment) NewDataList ¶
func (*Segment) NewFloat32List ¶
func (s *Segment) NewFloat32List(sz int) Float32List
func (*Segment) NewFloat64List ¶
func (s *Segment) NewFloat64List(sz int) Float64List
func (*Segment) NewInt16List ¶
func (*Segment) NewInt32List ¶
func (*Segment) NewInt64List ¶
func (*Segment) NewInt8List ¶
func (*Segment) NewPointerList ¶
func (s *Segment) NewPointerList(sz int) PointerList
func (*Segment) NewRootStruct ¶
func (*Segment) NewStructAR ¶
NewStructAR (AutoRoot): experimental Root setting: assumes the struct is the root iff it is the first allocation in a segment.
func (*Segment) NewTextList ¶
func (*Segment) NewUInt16List ¶
func (s *Segment) NewUInt16List(sz int) UInt16List
func (*Segment) NewUInt32List ¶
func (s *Segment) NewUInt32List(sz int) UInt32List
func (*Segment) NewUInt64List ¶
func (s *Segment) NewUInt64List(sz int) UInt64List
func (*Segment) NewUInt8List ¶
func (*Segment) NewVoidList ¶
func (s *Segment) NewVoidList(sz int) VoidList { return VoidList{typ: TypeList, length: sz, datasz: 0} }
type UInt16List ¶
type UInt16List Object
func (UInt16List) At ¶
func (p UInt16List) At(i int) uint16
func (UInt16List) Len ¶
func (p UInt16List) Len() int
func (UInt16List) Set ¶
func (p UInt16List) Set(i int, v uint16)
func (UInt16List) ToArray ¶
func (p UInt16List) ToArray() []uint16
func (UInt16List) ToEnumArray ¶
func (p UInt16List) ToEnumArray() *[]uint16
type UInt32List ¶
type UInt32List Object
func (UInt32List) At ¶
func (p UInt32List) At(i int) uint32
func (UInt32List) Len ¶
func (p UInt32List) Len() int
func (UInt32List) Set ¶
func (p UInt32List) Set(i int, v uint32)
func (UInt32List) ToArray ¶
func (p UInt32List) ToArray() []uint32
type UInt64List ¶
type UInt64List Object
func (UInt64List) At ¶
func (p UInt64List) At(i int) uint64
func (UInt64List) Len ¶
func (p UInt64List) Len() int
func (UInt64List) Set ¶
func (p UInt64List) Set(i int, v uint64)
func (UInt64List) ToArray ¶
func (p UInt64List) ToArray() []uint64
type UInt8List ¶
type UInt8List Object