zserio

README

go-zserio

zserio code generation in Go.

Versioning

go-zserio is still in active development, and not completely stable yet. Until the first stable release all versions will have a major version number of 0. The minor version number will be updated for (API) breaking changes.

Usage

First you need to install the go-zserio command:

go install github.com/woven-planet/go-zserio/cmd/go-zserio@latest

With this command you can generate the Go interface files to read/write zserio:

go-zserio generate <zserio_directory> --rootpackage <root_package> --out <output_directory>

This command compiles the zserio files, and generates Go files to read and write zserio encoded data. zserio_directory is the directory where the zserio definitions are stored. Supported file extensions are .zs and .zserio. rootpackage specifies the root package name of the generated Go files. output_directory specifies the path where the Go files will be generated.

Limitations

• Implicit length arrays are deprecated, and are not supported.
• The ternary operator only works when used in functions.
• Cyclic imports, while not recommended, are supported by zserio, but not supported by go-zserio.
• Offsets and indexed offsets are not implemented yet.
• Constraints are not implemented yet.
• The bazel run //internal/parser:update command to rerun the antlr parser does not work on Windows. See https://github.com/woven-planet/go-zserio/issues/128 for more details.

Links

• zserio
• Woven by Toyota

Documentation

Overview

zserio is a serialization framework designed for environments where storage or bandwidth is the most critical constraint. The primary use case is the NDS.Live format for automobile map data.

This package implements the de(serialization) of the core zserio data types, as well as a code generator to create Go code from zserio schema definitions.

Installation

You can use the normal go install method to install go-zserio:

go install github.com/woven-planet/go-zserio/cmd/go-zserio@latest

Code generation

The most common, and strongly preferred, method to use zserio is to write a zserio schema, and then generate code to define the data structures and serialization code.

As an example let's look at a zserio schema for a contact list. In a minimal example we only define an Addresses structure to store address information.

package contacts;

// Address, so I know where your house lives
struct Address
{
    string street;
    optional uint8 number;
};

After saving this in schema/contacts.zs you can generate Go code for this schema with this command:

go-zserio generate --out . -r myprojects.home/zserio-example ./schema

This will create a number of code files in the "contacts" directory. The root package is set to "myprojects.home/zserio-example", and must be defined to import statements can be generated.

You can now import the generated code, and serialize Address records using zserio:

 package main

 import (
     "fmt"
     "log"

     zserio "github.com/woven-planet/go-zserio"
     "myproject.home/zserio-example/contacts"
 )

 //go:generate go run github.com/woven-planet/go-zserio/cmd/go-zserio generate --rootpackage myproject.home/zserio-example --out . ./schema

 func main() {
     address := contacts.Address{Street: "Mainstreet"}
     bytes, err := zserio.Marshal(&address)
     if err != nil {
         log.Fatalf("error serializing address: %w", err)
     }

     fmt.Print(bytes)
}

For subsequent code generation you can use:

go generate ./...

See also

https://zserio.org

Index

• func Marshal(m Marshaler) ([]byte, error)
• func Unmarshal(b []byte, m Unmarshaler) error
• type Aligner
• type IDeltaContext
• type Marshaler
• type PackableMarshaler
• type PackableUnmarshaler
• type PackableZserioType
• type PackingContextNode
  • func (context *PackingContextNode) AddChild(child *PackingContextNode)
  • func (context *PackingContextNode) BitSizeOfDescriptor() int
  • func (context *PackingContextNode) GetChildren() []*PackingContextNode
  • func (context *PackingContextNode) HasContext() bool
  • func (context *PackingContextNode) ReadDescriptor(r Reader) error
  • func (context *PackingContextNode) WriteDescriptor(w Writer) error
• type Reader
• type Unmarshaler
• type Writer
• type ZserioType

Functions

func Marshal

func Marshal(m Marshaler) ([]byte, error)

Marshal accepts a pointer to an instance of a zserio object and returns a byte array which represents a zserio stream. On error, it will return an empty slice.

func Unmarshal

func Unmarshal(b []byte, m Unmarshaler) error

Unmarshal accepts a zserio byte stream and a pointer to an instance of a zserio object type, which is going to be populate with values deserialized from the bytes and returns a deserialization error.

Types

type Aligner

type Aligner interface {
	Align(boundary uint8) (int64, error)
}

Aligner aligns the underlying strim to the byte boundary and returns the number of bits that have been read or written since the beginning.

type IDeltaContext

type IDeltaContext interface {
	ReadDescriptor(r Reader) error
	WriteDescriptor(w Writer) error
	BitSizeOfDescriptor() int
}

type Marshaler

type Marshaler interface {
	MarshalZserio(w Writer) error
	ZserioBitSize(bitPosition int) (int, error)
}

Marshaler is the interface implemented by types that can be serialized themselves to a zserio bitstream. The implementation is normally automatically generated by the zserio compiler.

type PackableMarshaler

type PackableMarshaler interface {
	MarshalZserioPacked(contextNode *PackingContextNode, w Writer) error
	ZserioInitPackingContext(contextNode *PackingContextNode) error
	ZserioInitializeOffsetsPacked(contextNode *PackingContextNode, bitPosition int) int
	ZserioBitSizePacked(contextNode *PackingContextNode, bitPosition int) (int, error)
}

type PackableUnmarshaler

type PackableUnmarshaler interface {
	ZserioCreatePackingContext(contextNode *PackingContextNode) error
	UnmarshalZserioPacked(contextNode *PackingContextNode, r Reader) error
}

type PackableZserioType

type PackableZserioType interface {
	ZserioType
	PackableMarshaler
	PackableUnmarshaler
}

type PackingContextNode

type PackingContextNode struct {

	// the delta context used in a packed context.
	Context IDeltaContext
	// contains filtered or unexported fields
}

PackingContextNode is a context for writing packed data.

func (*PackingContextNode) AddChild

func (context *PackingContextNode) AddChild(child *PackingContextNode)

AddChild adds a new child to a PackingContextNode.

func (*PackingContextNode) BitSizeOfDescriptor

func (context *PackingContextNode) BitSizeOfDescriptor() int

func (*PackingContextNode) GetChildren

func (context *PackingContextNode) GetChildren() []*PackingContextNode

GetChildren returns the child PackingContextNodes of a PackingContextNode.

func (*PackingContextNode) HasContext

func (context *PackingContextNode) HasContext() bool

HasContext reurns true if the packing context has a delta context.

func (*PackingContextNode) ReadDescriptor

func (context *PackingContextNode) ReadDescriptor(r Reader) error

ReadDescriptor returns the delta context of the packing context.

func (*PackingContextNode) WriteDescriptor

func (context *PackingContextNode) WriteDescriptor(w Writer) error

type Reader

type Reader interface {
	io.ByteReader
	io.Reader
	Aligner

	ReadBits(n uint8) (uint64, error)
	ReadBool() (bool, error)
}

Reader is used to read at the bit level and allows to align the stream to the byte boundary at any stage and get the number of bits read.

type Unmarshaler

type Unmarshaler interface {
	UnmarshalZserio(r Reader) error
}

Unmarshaler is the interface implemented by types that can be read from a zserio bitstream. The implementation is normally automatically generated by the zserio compiler.

type Writer

type Writer interface {
	io.ByteWriter
	io.Writer
	Aligner

	WriteBits(r uint64, n uint8) error
	WriteBitsUnsafe(r uint64, n uint8) error
	WriteBool(b bool) error
}

Writer is used to write at the bit level and allows to align the stream to the byte boundary at any stage and get the number of bits written.

type ZserioType

type ZserioType interface {
	Marshaler
	Unmarshaler
	Clone() ZserioType
}