generator

package
v0.0.0-...-18dd212 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 1, 2014 License: BSD-3-Clause Imports: 16 Imported by: 0

Documentation

Overview

The code generator for the plugin for the Google protocol buffer compiler.
It generates Go code from the protocol buffer description files read by the
main routine.

The marshalto generates a Marshal and MarshalTo method for each message. The `Marshal() ([]byte, error)` method results in the fact that the message implements the Marshaler interface. This allows proto.Marshal to be faster by calling the generated Marshal method rather than using reflect to Marshal the struct.

The generation of marshalling tests are enabled using one of the following extensions:

  • testgen
  • testgen_all

And benchmarks given it is enabled using one of the following extensions:

  • benchgen
  • benchgen_all

Let us look at:

code.google.com/p/gogoprotobuf/test/example/example.proto

Btw all the output can be seen at:

code.google.com/p/gogoprotobuf/test/example/*

The following message:

  message B {
	option (gogoproto.description) = true;
	optional string A = 1 [(gogoproto.embed) = true];
	repeated int64 G = 2 [(gogoproto.customtype) = "github.com/dropbox/goprotoc/test.Id"];
  }

the marshalto code will generate the following code:

func (m *B) Marshal() (data []byte, err error) {
	size := m.Size()
	data = make([]byte, size)
	n, err := m.MarshalToUsingCachedSize(data)
	if err != nil {
		return nil, err
	}
	return data[:n], nil
}

func (m *B) MarshalTo(data []byte) (n int, err error) {
	m.Size()
	return m.MarshalToUsingCachedSize(data)
}

func (m *B) MarshalToUsingCachedSize(data []byte) (n int, err error) {
	var i int
	_ = i
	var l int
	_ = l
	if m.xxx_IsASet {
		data[i] = 0xa
		i++
		i = encodeVarintCustom(data, i, uint64(len(m.a)))
		i += copy(data[i:], m.a)
	}
	if m.xxx_LenG > 0 {
		for idx := 0; idx < m.xxx_LenG; idx++ {
			num := m.g[idx]
			data[i] = 0x10
			i++
			i = encodeVarintCustom(data, i, uint64(num))
		}
	}
	if m.XXX_unrecognized != nil {
		i += copy(data[i:], m.XXX_unrecognized)
	}
	return i, nil
}

As shown above Marshal calculates the size of the not yet marshalled message and allocates the appropriate buffer. This is followed by calling the MarshalTo method which requires a preallocated buffer. The MarshalTo method allows a user to rather preallocated a reusable buffer.

The Size method is generated using the size plugin and the gogoproto.sizer, gogoproto.sizer_all extensions. The user can also using the generated Size method to check that his reusable buffer is still big enough.

The generated tests and benchmarks will keep you safe and show that this is really a significant speed improvement.

This file generates a Size method for each message. This is useful with the MarshalTo method generated by the marshalto

The size code also generates a test given it is enabled using one of the following extensions:

  • testgen
  • testgen_all

And a benchmark given it is enabled using one of the following extensions:

  • benchgen
  • benchgen_all

Let us look at:

code.google.com/p/gogoprotobuf/test/example/example.proto

Btw all the output can be seen at:

code.google.com/p/gogoprotobuf/test/example/*

The following message:

  message B {
	option (gogoproto.description) = true;
	optional string A = 1 [(gogoproto.embed) = true];
	repeated int64 G = 2 [(gogoproto.customtype) = "github.com/dropbox/goprotoc/test.Id"];
  }

will generate the following code:

  func (m *B) Size() (n int) {
	var l int
	_ = l
	if m.xxx_IsASet {
		l = len(m.a)
		n += 1 + l + sovCustom(uint64(l))
	}
	if m.xxx_LenG > 0 {
		for i := 0; i < m.xxx_LenG; i++ {
			e := m.g[i]
			n += 1 + sovCustom(uint64(e))
		}
	}
	if m.XXX_unrecognized != nil {
		n += len(m.XXX_unrecognized)
	}
	m.xxx_sizeCached = n
	return n
  }

and the following test code:

  func TestBSize(t *testing5.T) {
	popr := math_rand5.New(math_rand5.NewSource(time5.Now().UnixNano()))
	p := NewPopulatedB(popr, true)
	data, err := dropbox_gogoprotobuf_proto2.Marshal(p)
	if err != nil {
	  panic(err)
	}
	size := g.Size()
	if len(data) != size {
	  t.Fatalf("size %v != marshalled size %v", size, len(data))
	}
  }

  func BenchmarkBSize(b *testing5.B) {
	popr := math_rand5.New(math_rand5.NewSource(616))
	total := 0
	pops := make([]*B, 1000)
	for i := 0; i < 1000; i++ {
	  pops[i] = NewPopulatedB(popr, false)
	}
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
	  total += pops[i%1000].Size()
	}
	b.SetBytes(int64(total / b.N))
  }

The sovExample function is a size of varint function for the example.pb.go file.

The unmarshal code generates a Unmarshal method for each message. The `Unmarshal([]byte) error` method results in the fact that the message implements the Unmarshaler interface. The allows proto.Unmarshal to be faster by calling the generated Unmarshal method rather than using reflect.

The generation of unmarshalling tests are enabled using one of the following extensions:

  • testgen
  • testgen_all

And benchmarks given it is enabled using one of the following extensions:

  • benchgen
  • benchgen_all

Let us look at:

code.google.com/p/gogoprotobuf/test/example/example.proto

Btw all the output can be seen at:

code.google.com/p/gogoprotobuf/test/example/*

The following message:

  option (gogoproto.unmarshaler_all) = true;

  message B {
	option (gogoproto.description) = true;
	optional string A = 1 [(gogoproto.embed) = true];
	repeated int64 G = 2 [(gogoproto.customtype) = "github.com/dropbox/goprotoc/test.Id"];
  }

the unmarshal will generate the following code:

func (m *B) Unmarshal(data []byte) error {
	l := len(data)
	index := 0
	for index < l {
		var wire uint64
		for shift := uint(0); ; shift += 7 {
			if index >= l {
				return io.ErrUnexpectedEOF
			}
			b := data[index]
			index++
			wire |= (uint64(b) & 0x7F) << shift
			if b < 0x80 {
				break
			}
		}
		fieldNum := int32(wire >> 3)
		wireType := int(wire & 0x7)
		switch fieldNum {
		case 1:
			if wireType != 2 {
				return fmt.Errorf("proto: wrong wireType = %d for field a", wireType)
			}
			m.xxx_IsASet = true
			var stringLen uint64
			for shift := uint(0); ; shift += 7 {
				if index >= l {
					return io.ErrUnexpectedEOF
				}
				b := data[index]
				index++
				stringLen |= (uint64(b) & 0x7F) << shift
				if b < 0x80 {
					break
				}
			}
			postIndex := index + int(stringLen)
			if postIndex > l {
				return io.ErrUnexpectedEOF
			}
			m.a = string(data[index:postIndex])
			index = postIndex
		case 2:
			if wireType != 0 {
				return fmt.Errorf("proto: wrong wireType = %d for field g", wireType)
			}
			m.xxx_LenG += 1
			var v int64
			for shift := uint(0); ; shift += 7 {
				if index >= l {
					return io.ErrUnexpectedEOF
				}
				b := data[index]
				index++
				v |= (int64(b) & 0x7F) << shift
				if b < 0x80 {
					break
				}
			}
			m.g = append(m.g, github_com_dropbox_goprotoc_test.Id(v))
		default:
			var sizeOfWire int
			for {
				sizeOfWire++
				wire >>= 7
				if wire == 0 {
					break
				}
			}
			index -= sizeOfWire
			skippy, err := proto.Skip(data[index:])
			if err != nil {
				return err
			}
			if (index + skippy) > l {
				return io.ErrUnexpectedEOF
			}
			m.XXX_unrecognized = append(m.XXX_unrecognized, data[index:index+skippy]...)
			index += skippy
		}
	}
	return nil
}

Remember when using this code to call proto.Unmarshal. This will call m.Reset and invoke the generated Unmarshal method for you. If you call m.Unmarshal without m.Reset you could be merging protocol buffers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CamelCase

func CamelCase(s string) string

CamelCase returns the CamelCased name. If there is an interior underscore followed by a lower case letter, drop the underscore and convert the letter to upper case. There is a remote possibility of this rewrite causing a name collision, but it's so remote we're prepared to pretend it's nonexistent - since the C++ generator lowercases names, it's extremely unlikely to have two fields with different capitalizations. In short, _my_field_name_2 becomes XMyFieldName_2.

func CamelCaseSlice

func CamelCaseSlice(elem []string) string

CamelCaseSlice is like CamelCase, but the argument is a slice of strings to be joined with "_".

func EmbedFieldName

func EmbedFieldName(goTyp string) string

func FileName

func FileName(file *FileDescriptor) string

func GetCustomType

func GetCustomType(field *descriptor.FieldDescriptorProto) (packageName string, typ string, err error)

func GetDefaultValue

func GetDefaultValue(field *descriptor.FieldDescriptorProto) (value string)

func GoTypeToName

func GoTypeToName(goTyp string) string

func IsMessageType

func IsMessageType(field *descriptor.FieldDescriptorProto) bool

Is this field a message?

func IsRepeated

func IsRepeated(field *descriptor.FieldDescriptorProto) bool

Is this field repeated?

func MakePrivate

func MakePrivate(s string) string

Returns the private name starting with a lower case letter

func NeedsStar

func NeedsStar(field *descriptor.FieldDescriptorProto) bool

func NewPluginImports

func NewPluginImports(generator *Generator) *pluginImports

func RegisterPlugin

func RegisterPlugin(p Plugin)

RegisterPlugin installs a (second-order) plugin to be run when the Go output is generated. It is typically called during initialization.

func RegisterUniquePackageName

func RegisterUniquePackageName(pkg string, f *FileDescriptor) string

Create and remember a guaranteed unique package name for this file descriptor. Pkg is the candidate name. If f is nil, it's a builtin package like "proto" and has no file descriptor.

func SetterName

func SetterName(fieldName string) string

func SizerName

func SizerName(fieldName string) string

Types

type Descriptor

type Descriptor struct {
	*descriptor.DescriptorProto
	// contains filtered or unexported fields
}

Descriptor represents a protocol buffer message.

func (*Descriptor) File

func (c *Descriptor) File() *descriptor.FileDescriptorProto

func (*Descriptor) IsGroup

func (d *Descriptor) IsGroup() bool

func (*Descriptor) PackageName

func (c *Descriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*Descriptor) TypeName

func (d *Descriptor) TypeName() []string

TypeName returns the elements of the dotted type name. The package name is not part of this name.

type EnumDescriptor

type EnumDescriptor struct {
	*descriptor.EnumDescriptorProto
	// contains filtered or unexported fields
}

EnumDescriptor describes an enum. If it's at top level, its parent will be nil. Otherwise it will be the descriptor of the message in which it is defined.

func (*EnumDescriptor) File

func (c *EnumDescriptor) File() *descriptor.FileDescriptorProto

func (*EnumDescriptor) PackageName

func (c *EnumDescriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*EnumDescriptor) TypeName

func (e *EnumDescriptor) TypeName() (s []string)

TypeName returns the elements of the dotted type name. The package name is not part of this name.

type ExtensionDescriptor

type ExtensionDescriptor struct {
	*descriptor.FieldDescriptorProto
	// contains filtered or unexported fields
}

ExtensionDescriptor describes an extension. If it's at top level, its parent will be nil. Otherwise it will be the descriptor of the message in which it is defined.

func (*ExtensionDescriptor) DescName

func (e *ExtensionDescriptor) DescName() string

DescName returns the variable name used for the generated descriptor.

func (*ExtensionDescriptor) File

func (c *ExtensionDescriptor) File() *descriptor.FileDescriptorProto

func (*ExtensionDescriptor) PackageName

func (c *ExtensionDescriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*ExtensionDescriptor) TypeName

func (e *ExtensionDescriptor) TypeName() (s []string)

TypeName returns the elements of the dotted type name. The package name is not part of this name.

type FileDescriptor

type FileDescriptor struct {
	*descriptor.FileDescriptorProto
	// contains filtered or unexported fields
}

FileDescriptor describes an protocol buffer descriptor file (.proto). It includes slices of all the messages and enums defined within it. Those slices are constructed by WrapTypes.

func (*FileDescriptor) Enums

func (d *FileDescriptor) Enums() []*EnumDescriptor

func (*FileDescriptor) Messages

func (d *FileDescriptor) Messages() []*Descriptor

func (*FileDescriptor) PackageName

func (d *FileDescriptor) PackageName() string

PackageName is the package name we'll use in the generated code to refer to this file.

type Generator

type Generator struct {
	*bytes.Buffer

	Request  *plugin.CodeGeneratorRequest  // The input.
	Response *plugin.CodeGeneratorResponse // The output.

	Param             map[string]string // Command-line parameters.
	PackageImportPath string            // Go import path of the package we're generating code for
	ImportPrefix      string            // String to prefix to imported package file names.
	ImportMap         map[string]string // Mapping from import name to generated name

	Pkg map[string]string // The names under which we import support packages
	// contains filtered or unexported fields
}

Generator is the type whose methods generate the output, stored in the associated response structure.

func New

func New() *Generator

New creates a new generator and allocates the request and response protobufs.

func (*Generator) AllFiles

func (g *Generator) AllFiles() *descriptor.FileDescriptorSet

func (*Generator) BuildTypeNameMap

func (g *Generator) BuildTypeNameMap()

BuildTypeNameMap builds the map from fully qualified type names to objects. The key names for the map come from the input data, which puts a period at the beginning. It should be called after SetPackageNames and before GenerateAllFiles.

func (*Generator) CommandLineParameters

func (g *Generator) CommandLineParameters(parameter string)

CommandLineParameters breaks the comma-separated list of key=value pairs in the parameter (a member of the request protobuf) into a key/value map. It then sets file name mappings defined by those entries.

func (*Generator) DefaultPackageName

func (g *Generator) DefaultPackageName(obj Object) string

DefaultPackageName returns the package name printed for the object. If its file is in a different package, it returns the package name we're using for this file, plus ".". Otherwise it returns the empty string.

func (*Generator) Error

func (g *Generator) Error(err error, msgs ...string)

Error reports a problem, including an error, and exits the program.

func (*Generator) Fail

func (g *Generator) Fail(msgs ...string)

Fail reports a problem and exits the program.

func (*Generator) FileOf

FileOf return the FileDescriptor for this FileDescriptorProto.

func (*Generator) GenerateAllFiles

func (g *Generator) GenerateAllFiles()

GenerateAllFiles generates the output for all the files we're outputting.

func (*Generator) GeneratePlugin

func (g *Generator) GeneratePlugin(p Plugin)

func (*Generator) GetFieldName

func (g *Generator) GetFieldName(message *Descriptor, field *descriptor.FieldDescriptorProto) string

func (*Generator) GoBaseType

func (g *Generator) GoBaseType(field *descriptor.FieldDescriptorProto) (typ string, wire string)

GoBaseType returns a string representing the base type name, and the wire type

func (*Generator) GoType

func (g *Generator) GoType(message *Descriptor, field *descriptor.FieldDescriptorProto) (typ string, wire string)

GoType returns a string representing the type name, and the wire type

func (*Generator) In

func (g *Generator) In()

In Indents the output one tab stop.

func (*Generator) IsGroup

func (g *Generator) IsGroup(field *descriptor.FieldDescriptorProto) bool

func (*Generator) ObjectNamed

func (g *Generator) ObjectNamed(typeName string) Object

ObjectNamed, given a fully-qualified input type name as it appears in the input data, returns the descriptor for the message or enum with that name.

func (*Generator) Out

func (g *Generator) Out()

Out unindents the output one tab stop.

func (*Generator) P

func (g *Generator) P(str ...interface{})

P prints the arguments to the generated output. It handles strings and int32s, plus handling indirections because they may be *string, etc.

func (*Generator) PrintComments

func (g *Generator) PrintComments(path string)

PrintComments prints any comments from the source .proto file. The path is a comma-separated list of integers. See descriptor.proto for its format.

func (*Generator) RecordTypeUse

func (g *Generator) RecordTypeUse(t string)

func (*Generator) SetPackageNames

func (g *Generator) SetPackageNames()

SetPackageNames sets the package name for this run. The package name must agree across all files being generated. It also defines unique package names for all imported files.

func (*Generator) TypeName

func (g *Generator) TypeName(obj Object) string

TypeName is the printed name appropriate for an item. If the object is in the current file, TypeName drops the package name and underscores the rest. Otherwise the object is from another package; and the result is the underscored package name followed by the item name. The result always has an initial capital.

func (*Generator) TypeNameByObject

func (g *Generator) TypeNameByObject(typeName string) Object

func (*Generator) TypeNameWithPackage

func (g *Generator) TypeNameWithPackage(obj Object) string

TypeNameWithPackage is like TypeName, but always includes the package name even if the object is in our own package.

func (*Generator) WrapTypes

func (g *Generator) WrapTypes()

WrapTypes walks the incoming data, wrapping DescriptorProtos, EnumDescriptorProtos and FileDescriptorProtos into file-referenced objects within the Generator. It also creates the list of files to generate and so should be called before GenerateAllFiles.

type ImportedDescriptor

type ImportedDescriptor struct {
	// contains filtered or unexported fields
}

ImportedDescriptor describes a type that has been publicly imported from another file.

func (*ImportedDescriptor) File

func (c *ImportedDescriptor) File() *descriptor.FileDescriptorProto

func (*ImportedDescriptor) PackageName

func (c *ImportedDescriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*ImportedDescriptor) TypeName

func (id *ImportedDescriptor) TypeName() []string

type NumGen

type NumGen interface {
	Next() string
	Current() string
}

func NewNumGen

func NewNumGen() NumGen

type Object

type Object interface {
	PackageName() string // The name we use in our output (a_b_c), possibly renamed for uniqueness.
	TypeName() []string
	File() *descriptor.FileDescriptorProto
}

Object is an interface abstracting the abilities shared by enums, messages, extensions and imported objects.

type Plugin

type Plugin interface {
	// Name identifies the plugin.
	Name() string
	// Init is called once after data structures are built but before
	// code generation begins.
	Init(g *Generator)
	// Generate produces the code generated by the plugin for this file,
	// except for the imports, by calling the generator's methods P, In, and Out.
	Generate(file *FileDescriptor)
	// GenerateImports produces the import declarations for this file.
	// It is called after Generate.
	GenerateImports(file *FileDescriptor)
}

A Plugin provides functionality to add to the output during Go code generation, such as to produce RPC stubs.

type PluginImports

type PluginImports interface {
	NewImport(pkg string) Single
	GenerateImports(file *FileDescriptor)
}

type Single

type Single interface {
	Use() string
	IsUsed() bool
	Generate() string
}

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL