protobuf

Documentation

Overview

Package protobuf defines functionality for parsing protocol buffer definitions and instances.

Proto definition mapping follows the guidelines of mapping Proto to JSON as discussed in https://developers.google.com/protocol-buffers/docs/proto3, and carries some of the mapping further when possible with CUE.

Package Paths

If a .proto file contains a go_package directive, it will be used as the destination package for the generated .cue files. A common use case is to generate the CUE in the same directory as the .proto definition. If a destination package is not within the current CUE module, it will be written relative to the pkg directory.

If a .proto file does not specify go_package, it will convert a proto package "google.parent.sub" to the import path "googleapis.com/google/parent/sub". It is safe to mix packages with and without a go_package within the same project.

Type Mappings

The following type mappings of definitions apply:

Proto type     CUE type/def     Comments
message        struct           Message fields become CUE fields, whereby
                                names are mapped to lowerCamelCase.
enum           e1 | e2 | ...    Where ex are strings. A separate mapping is
                                generated to obtain the numeric values.
map<K, V>      { <>: V }        All keys are converted to strings.
repeated V     [...V]           null is accepted as the empty list [].
bool           bool
string         string
bytes          bytes            A base64-encoded string when converted to JSON.
int32, fixed32 int32            An integer with bounds as defined by int32.
uint32         uint32           An integer with bounds as defined by uint32.
int64, fixed64 int64            An integer with bounds as defined by int64.
uint64         uint64           An integer with bounds as defined by uint64.
float          float32          A number with bounds as defined by float32.
double         float64          A number with bounds as defined by float64.
Struct         struct           See struct.proto.
Value          _                See struct.proto.
ListValue      [...]            See struct.proto.
NullValue      null             See struct.proto.
BoolValue      bool             See struct.proto.
StringValue    string           See struct.proto.
NumberValue    number           See struct.proto.
StringValue    string           See struct.proto.
Empty          close({})
Timestamp      time.Time        See struct.proto.
Duration       time.Duration    See struct.proto.

Annotations

Protobuf definitions can be annotated with CUE constraints that are included in the generated CUE:

(cue.val)     string        CUE expression defining a constraint for this
                            field. The string may refer to other fields
                            in a message definition using their JSON name.

(cue.opt)     FieldOptions
   required   bool          Defines the field is required. Use with
                            caution.

Index

• func Extract(filename string, src interface{}, c *Config) (f *ast.File, err error)
• type Config
• type Extractor
  • func NewExtractor(c *Config) *Extractor
  • func (b *Extractor) AddFile(filename string, src interface{}) error
  • func (b *Extractor) Err() error
  • func (b *Extractor) Files() (files []*ast.File, err error)
  • func (b *Extractor) Instances() (instances []*build.Instance, err error)

Examples

• Extract

Functions

func Extract

func Extract(filename string, src interface{}, c *Config) (f *ast.File, err error)

Extract parses a single proto file and returns its contents translated to a CUE file. If src is not nil, it will use this as the contents of the file. It may be a string, []byte or io.Reader. Otherwise Extract will open the given file name at the fully qualified path.

Extract assumes the proto file compiles with protoc and may not report an error if it does not. Imports are resolved using the paths defined in Config.

Example

package main

import (
	"fmt"
	"log"
	"os"
	"path/filepath"

	"cuelang.org/go/cue/format"
	"cuelang.org/go/encoding/protobuf"
)

func main() {
	cwd, _ := os.Getwd()
	var paths = []string{}
	paths = append(paths, cwd)
	paths = append(paths, filepath.Join(cwd, "testdata"))

	f, err := protobuf.Extract("examples/basic/basic.proto", nil, &protobuf.Config{
		Paths: paths,
	})

	if err != nil {
		log.Fatal(err, "")
	}

	b, _ := format.Node(f)
	fmt.Println(string(b))

}

Output:

// Package basic is just that: basic.
package basic

// This is my type.
#MyType: {
	stringValue?: string @protobuf(1,string,name=string_value) // just any 'ole string

	// A method must start with a capital letter.
	method?: [...string] @protobuf(2,string)
	method?: [...=~"^[A-Z]"]
	exampleMap?: {
		[string]: string
	} @protobuf(3,map[string]string,example_map)
}

Types

type Config

type Config struct {
	// Root specifies the root of the CUE project, which typically coincides
	// with, for example, a version control repository root or the Go module.
	// Any imports of proto files within the directory tree of this of this root
	// are considered to be "project files" and are generated at the
	// corresponding location with this hierarchy. Any other imports are
	// considered to be external. Files for such imports are rooted under the
	// $Root/pkg/, using the Go package path specified in the .proto file.
	Root string

	// Module is the Go package import path of the module root. It is the value
	// as after "module" in a cue.mod/modules.cue file, if a module file is
	// present.
	Module string // TODO: determine automatically if unspecified.

	// Paths defines the include directory in which to search for imports.
	Paths []string

	// PkgName specifies the package name for a generated CUE file. A value
	// will be derived from the Go package name if undefined.
	PkgName string

	// EnumMode defines whether enums should be set as integer values, instead
	// of strings.
	//
	//    json    value is a string, corresponding to the standard JSON mapping
	//            of Protobuf. The value is associated with a #enumValue
	//            to allow the json+pb interpretation to interpret integers
	//            as well.
	//
	//    int     value is an integer associated with an #enumValue definition
	//            The json+pb interpreter uses the definition names in the
	//            disjunction of the enum to interpret strings.
	//
	EnumMode string
}

Config specifies the environment into which to parse a proto definition file.

type Extractor

type Extractor struct {
	// contains filtered or unexported fields
}

An Extractor converts a collection of proto files, typically belonging to one repo or module, to CUE. It thereby observes the CUE package layout.

CUE observes the same package layout as Go and requires .proto files to have the go_package directive. Generated CUE files are put in the same directory as their corresponding .proto files if the .proto files are located in the specified Root (or current working directory if none is specified). All other imported files are assigned to the CUE pkg dir ($Root/pkg) according to their Go package import path.

func NewExtractor

func NewExtractor(c *Config) *Extractor

NewExtractor creates an Extractor. If the configuration contained any errors it will be observable by the Err method fo the Extractor. It is safe, however, to only check errors after building the output.

func (*Extractor) AddFile

func (b *Extractor) AddFile(filename string, src interface{}) error

AddFile adds a proto definition file to be converted into CUE by the builder. Relatives paths are always taken relative to the Root with which the b is configured.

AddFile assumes that the proto file compiles with protoc and may not report an error if it does not. Imports are resolved using the paths defined in Config.

func (*Extractor) Err

func (b *Extractor) Err() error

Err returns the errors accumulated during testing. The returned error may be of type errors.List.

func (*Extractor) Files

func (b *Extractor) Files() (files []*ast.File, err error)

Files returns a File for each proto file that was added or imported, recursively.

func (*Extractor) Instances

func (b *Extractor) Instances() (instances []*build.Instance, err error)

Instances creates a build.Instances for every package for which a proto file was added to the builder. This includes transitive dependencies. It does not write the generated files to disk.

The returned instances can be passed to cue.Build to generated the corresponding CUE instances.

All import paths are located within the specified Root, where external packages are located under $Root/pkg. Instances for builtin (like time) packages may be omitted, and if not will have no associated files.