generator

package
v0.0.0-...-18dd212 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 1, 2014 License: BSD-3-Clause Imports: 16 Imported by: 0

Documentation

Overview 

The code generator for the plugin for the Google protocol buffer compiler.
It generates Go code from the protocol buffer description files read by the
main routine.

The marshalto generates a Marshal and MarshalTo method for each message. The `Marshal() ([]byte, error)` method results in the fact that the message implements the Marshaler interface. This allows proto.Marshal to be faster by calling the generated Marshal method rather than using reflect to Marshal the struct.

The generation of marshalling tests are enabled using one of the following extensions:

  • testgen
  • testgen_all

And benchmarks given it is enabled using one of the following extensions:

  • benchgen
  • benchgen_all

Let us look at: 

code.google.com/p/gogoprotobuf/test/example/example.proto

Btw all the output can be seen at: 

code.google.com/p/gogoprotobuf/test/example/*

The following message: 

  message B {
	option (gogoproto.description) = true;
	optional string A = 1 [(gogoproto.embed) = true];
	repeated int64 G = 2 [(gogoproto.customtype) = "github.com/dropbox/goprotoc/test.Id"];
  }

the marshalto code will generate the following code: 

func (m *B) Marshal() (data []byte, err error) {
	size := m.Size()
	data = make([]byte, size)
	n, err := m.MarshalToUsingCachedSize(data)
	if err != nil {
		return nil, err
	}
	return data[:n], nil
}

func (m *B) MarshalTo(data []byte) (n int, err error) {
	m.Size()
	return m.MarshalToUsingCachedSize(data)
}

func (m *B) MarshalToUsingCachedSize(data []byte) (n int, err error) {
	var i int
	_ = i
	var l int
	_ = l
	if m.xxx_IsASet {
		data[i] = 0xa
		i++
		i = encodeVarintCustom(data, i, uint64(len(m.a)))
		i += copy(data[i:], m.a)
	}
	if m.xxx_LenG > 0 {
		for idx := 0; idx < m.xxx_LenG; idx++ {
			num := m.g[idx]
			data[i] = 0x10
			i++
			i = encodeVarintCustom(data, i, uint64(num))
		}
	}
	if m.XXX_unrecognized != nil {
		i += copy(data[i:], m.XXX_unrecognized)
	}
	return i, nil
}

As shown above Marshal calculates the size of the not yet marshalled message and allocates the appropriate buffer. This is followed by calling the MarshalTo method which requires a preallocated buffer. The MarshalTo method allows a user to rather preallocated a reusable buffer.

The Size method is generated using the size plugin and the gogoproto.sizer, gogoproto.sizer_all extensions. The user can also using the generated Size method to check that his reusable buffer is still big enough.

The generated tests and benchmarks will keep you safe and show that this is really a significant speed improvement.

This file generates a Size method for each message. This is useful with the MarshalTo method generated by the marshalto

The size code also generates a test given it is enabled using one of the following extensions:

  • testgen
  • testgen_all

And a benchmark given it is enabled using one of the following extensions:

  • benchgen
  • benchgen_all

Let us look at: 

code.google.com/p/gogoprotobuf/test/example/example.proto

Btw all the output can be seen at: 

code.google.com/p/gogoprotobuf/test/example/*

The following message: 

  message B {
	option (gogoproto.description) = true;
	optional string A = 1 [(gogoproto.embed) = true];
	repeated int64 G = 2 [(gogoproto.customtype) = "github.com/dropbox/goprotoc/test.Id"];
  }

will generate the following code: 

  func (m *B) Size() (n int) {
	var l int
	_ = l
	if m.xxx_IsASet {
		l = len(m.a)
		n += 1 + l + sovCustom(uint64(l))
	}
	if m.xxx_LenG > 0 {
		for i := 0; i < m.xxx_LenG; i++ {
			e := m.g[i]
			n += 1 + sovCustom(uint64(e))
		}
	}
	if m.XXX_unrecognized != nil {
		n += len(m.XXX_unrecognized)
	}
	m.xxx_sizeCached = n
	return n
  }

and the following test code: 

  func TestBSize(t *testing5.T) {
	popr := math_rand5.New(math_rand5.NewSource(time5.Now().UnixNano()))
	p := NewPopulatedB(popr, true)
	data, err := dropbox_gogoprotobuf_proto2.Marshal(p)
	if err != nil {
	  panic(err)
	}
	size := g.Size()
	if len(data) != size {
	  t.Fatalf("size %v != marshalled size %v", size, len(data))
	}
  }

  func BenchmarkBSize(b *testing5.B) {
	popr := math_rand5.New(math_rand5.NewSource(616))
	total := 0
	pops := make([]*B, 1000)
	for i := 0; i < 1000; i++ {
	  pops[i] = NewPopulatedB(popr, false)
	}
	b.ResetTimer()
	for i := 0; i < b.N; i++ {
	  total += pops[i%1000].Size()
	}
	b.SetBytes(int64(total / b.N))
  }

The sovExample function is a size of varint function for the example.pb.go file.

The unmarshal code generates a Unmarshal method for each message. The `Unmarshal([]byte) error` method results in the fact that the message implements the Unmarshaler interface. The allows proto.Unmarshal to be faster by calling the generated Unmarshal method rather than using reflect.

The generation of unmarshalling tests are enabled using one of the following extensions:

  • testgen
  • testgen_all

And benchmarks given it is enabled using one of the following extensions:

  • benchgen
  • benchgen_all

Let us look at: 

code.google.com/p/gogoprotobuf/test/example/example.proto

Btw all the output can be seen at: 

code.google.com/p/gogoprotobuf/test/example/*

The following message: 

  option (gogoproto.unmarshaler_all) = true;

  message B {
	option (gogoproto.description) = true;
	optional string A = 1 [(gogoproto.embed) = true];
	repeated int64 G = 2 [(gogoproto.customtype) = "github.com/dropbox/goprotoc/test.Id"];
  }

the unmarshal will generate the following code: 

func (m *B) Unmarshal(data []byte) error {
	l := len(data)
	index := 0
	for index < l {
		var wire uint64
		for shift := uint(0); ; shift += 7 {
			if index >= l {
				return io.ErrUnexpectedEOF
			}
			b := data[index]
			index++
			wire |= (uint64(b) & 0x7F) << shift
			if b < 0x80 {
				break
			}
		}
		fieldNum := int32(wire >> 3)
		wireType := int(wire & 0x7)
		switch fieldNum {
		case 1:
			if wireType != 2 {
				return fmt.Errorf("proto: wrong wireType = %d for field a", wireType)
			}
			m.xxx_IsASet = true
			var stringLen uint64
			for shift := uint(0); ; shift += 7 {
				if index >= l {
					return io.ErrUnexpectedEOF
				}
				b := data[index]
				index++
				stringLen |= (uint64(b) & 0x7F) << shift
				if b < 0x80 {
					break
				}
			}
			postIndex := index + int(stringLen)
			if postIndex > l {
				return io.ErrUnexpectedEOF
			}
			m.a = string(data[index:postIndex])
			index = postIndex
		case 2:
			if wireType != 0 {
				return fmt.Errorf("proto: wrong wireType = %d for field g", wireType)
			}
			m.xxx_LenG += 1
			var v int64
			for shift := uint(0); ; shift += 7 {
				if index >= l {
					return io.ErrUnexpectedEOF
				}
				b := data[index]
				index++
				v |= (int64(b) & 0x7F) << shift
				if b < 0x80 {
					break
				}
			}
			m.g = append(m.g, github_com_dropbox_goprotoc_test.Id(v))
		default:
			var sizeOfWire int
			for {
				sizeOfWire++
				wire >>= 7
				if wire == 0 {
					break
				}
			}
			index -= sizeOfWire
			skippy, err := proto.Skip(data[index:])
			if err != nil {
				return err
			}
			if (index + skippy) > l {
				return io.ErrUnexpectedEOF
			}
			m.XXX_unrecognized = append(m.XXX_unrecognized, data[index:index+skippy]...)
			index += skippy
		}
	}
	return nil
}

Remember when using this code to call proto.Unmarshal. This will call m.Reset and invoke the generated Unmarshal method for you. If you call m.Unmarshal without m.Reset you could be merging protocol buffers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CamelCase 

func CamelCase(s string) string

CamelCase returns the CamelCased name. If there is an interior underscore followed by a lower case letter, drop the underscore and convert the letter to upper case. There is a remote possibility of this rewrite causing a name collision, but it's so remote we're prepared to pretend it's nonexistent - since the C++ generator lowercases names, it's extremely unlikely to have two fields with different capitalizations. In short, _my_field_name_2 becomes XMyFieldName_2.

func CamelCaseSlice 

func CamelCaseSlice(elem []string) string

CamelCaseSlice is like CamelCase, but the argument is a slice of strings to be joined with "_".

func EmbedFieldName 

func EmbedFieldName(goTyp string) string

func FileName 

func FileName(file *FileDescriptor) string

func GetCustomType 

func GetCustomType(field *descriptor.FieldDescriptorProto) (packageName string, typ string, err error)

func GetDefaultValue 

func GetDefaultValue(field *descriptor.FieldDescriptorProto) (value string)

func GoTypeToName 

func GoTypeToName(goTyp string) string

func IsMessageType 

func IsMessageType(field *descriptor.FieldDescriptorProto) bool

Is this field a message?

func IsRepeated 

func IsRepeated(field *descriptor.FieldDescriptorProto) bool

Is this field repeated?

func MakePrivate 

func MakePrivate(s string) string

Returns the private name starting with a lower case letter

func NeedsStar 

func NeedsStar(field *descriptor.FieldDescriptorProto) bool

func NewPluginImports 

func NewPluginImports(generator *Generator) *pluginImports

func RegisterPlugin 

func RegisterPlugin(p Plugin)

RegisterPlugin installs a (second-order) plugin to be run when the Go output is generated. It is typically called during initialization.

func RegisterUniquePackageName 

func RegisterUniquePackageName(pkg string, f *FileDescriptor) string

Create and remember a guaranteed unique package name for this file descriptor. Pkg is the candidate name. If f is nil, it's a builtin package like "proto" and has no file descriptor.

func SetterName 

func SetterName(fieldName string) string

func SizerName 

func SizerName(fieldName string) string

Types

type Descriptor 

type Descriptor struct {
	*descriptor.DescriptorProto
	// contains filtered or unexported fields
}

Descriptor represents a protocol buffer message.

func (*Descriptor) File 

func (c *Descriptor) File() *descriptor.FileDescriptorProto

func (*Descriptor) IsGroup 

func (d *Descriptor) IsGroup() bool

func (*Descriptor) PackageName 

func (c *Descriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*Descriptor) TypeName 

func (d *Descriptor) TypeName() []string

TypeName returns the elements of the dotted type name. The package name is not part of this name.

type EnumDescriptor 

type EnumDescriptor struct {
	*descriptor.EnumDescriptorProto
	// contains filtered or unexported fields
}

EnumDescriptor describes an enum. If it's at top level, its parent will be nil. Otherwise it will be the descriptor of the message in which it is defined.

func (*EnumDescriptor) File 

func (c *EnumDescriptor) File() *descriptor.FileDescriptorProto

func (*EnumDescriptor) PackageName 

func (c *EnumDescriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*EnumDescriptor) TypeName 

func (e *EnumDescriptor) TypeName() (s []string)

TypeName returns the elements of the dotted type name. The package name is not part of this name.

type ExtensionDescriptor 

type ExtensionDescriptor struct {
	*descriptor.FieldDescriptorProto
	// contains filtered or unexported fields
}

ExtensionDescriptor describes an extension. If it's at top level, its parent will be nil. Otherwise it will be the descriptor of the message in which it is defined.

func (*ExtensionDescriptor) DescName 

func (e *ExtensionDescriptor) DescName() string

DescName returns the variable name used for the generated descriptor.

func (*ExtensionDescriptor) File 

func (c *ExtensionDescriptor) File() *descriptor.FileDescriptorProto

func (*ExtensionDescriptor) PackageName 

func (c *ExtensionDescriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*ExtensionDescriptor) TypeName 

func (e *ExtensionDescriptor) TypeName() (s []string)

TypeName returns the elements of the dotted type name. The package name is not part of this name.

type FileDescriptor 

type FileDescriptor struct {
	*descriptor.FileDescriptorProto
	// contains filtered or unexported fields
}

FileDescriptor describes an protocol buffer descriptor file (.proto). It includes slices of all the messages and enums defined within it. Those slices are constructed by WrapTypes.

func (*FileDescriptor) Enums 

func (d *FileDescriptor) Enums() []*EnumDescriptor

func (*FileDescriptor) Messages 

func (d *FileDescriptor) Messages() []*Descriptor

func (*FileDescriptor) PackageName 

func (d *FileDescriptor) PackageName() string

PackageName is the package name we'll use in the generated code to refer to this file.

type Generator 

type Generator struct {
	*bytes.Buffer

	Request  *plugin.CodeGeneratorRequest  // The input.
	Response *plugin.CodeGeneratorResponse // The output.

	Param             map[string]string // Command-line parameters.
	PackageImportPath string            // Go import path of the package we're generating code for
	ImportPrefix      string            // String to prefix to imported package file names.
	ImportMap         map[string]string // Mapping from import name to generated name

	Pkg map[string]string // The names under which we import support packages
	// contains filtered or unexported fields
}

Generator is the type whose methods generate the output, stored in the associated response structure.

func New 

func New() *Generator

New creates a new generator and allocates the request and response protobufs.

func (*Generator) AllFiles 

func (g *Generator) AllFiles() *descriptor.FileDescriptorSet

func (*Generator) BuildTypeNameMap 

func (g *Generator) BuildTypeNameMap()

BuildTypeNameMap builds the map from fully qualified type names to objects. The key names for the map come from the input data, which puts a period at the beginning. It should be called after SetPackageNames and before GenerateAllFiles.

func (*Generator) CommandLineParameters 

func (g *Generator) CommandLineParameters(parameter string)

CommandLineParameters breaks the comma-separated list of key=value pairs in the parameter (a member of the request protobuf) into a key/value map. It then sets file name mappings defined by those entries.

func (*Generator) DefaultPackageName 

func (g *Generator) DefaultPackageName(obj Object) string

DefaultPackageName returns the package name printed for the object. If its file is in a different package, it returns the package name we're using for this file, plus ".". Otherwise it returns the empty string.

func (*Generator) Error 

func (g *Generator) Error(err error, msgs ...string)

Error reports a problem, including an error, and exits the program.

func (*Generator) Fail 

func (g *Generator) Fail(msgs ...string)

Fail reports a problem and exits the program.

func (*Generator) FileOf 

func (g *Generator) FileOf(fd *descriptor.FileDescriptorProto) *FileDescriptor

FileOf return the FileDescriptor for this FileDescriptorProto.

func (*Generator) GenerateAllFiles 

func (g *Generator) GenerateAllFiles()

GenerateAllFiles generates the output for all the files we're outputting.

func (*Generator) GeneratePlugin 

func (g *Generator) GeneratePlugin(p Plugin)

func (*Generator) GetFieldName 

func (g *Generator) GetFieldName(message *Descriptor, field *descriptor.FieldDescriptorProto) string

func (*Generator) GoBaseType 

func (g *Generator) GoBaseType(field *descriptor.FieldDescriptorProto) (typ string, wire string)

GoBaseType returns a string representing the base type name, and the wire type

func (*Generator) GoType 

func (g *Generator) GoType(message *Descriptor, field *descriptor.FieldDescriptorProto) (typ string, wire string)

GoType returns a string representing the type name, and the wire type

func (*Generator) In 

func (g *Generator) In()

In Indents the output one tab stop.

func (*Generator) IsGroup 

func (g *Generator) IsGroup(field *descriptor.FieldDescriptorProto) bool

func (*Generator) ObjectNamed 

func (g *Generator) ObjectNamed(typeName string) Object

ObjectNamed, given a fully-qualified input type name as it appears in the input data, returns the descriptor for the message or enum with that name.

func (*Generator) Out 

func (g *Generator) Out()

Out unindents the output one tab stop.

func (*Generator) P 

func (g *Generator) P(str ...interface{})

P prints the arguments to the generated output. It handles strings and int32s, plus handling indirections because they may be *string, etc.

func (*Generator) PrintComments 

func (g *Generator) PrintComments(path string)

PrintComments prints any comments from the source .proto file. The path is a comma-separated list of integers. See descriptor.proto for its format.

func (*Generator) RecordTypeUse 

func (g *Generator) RecordTypeUse(t string)

func (*Generator) SetPackageNames 

func (g *Generator) SetPackageNames()

SetPackageNames sets the package name for this run. The package name must agree across all files being generated. It also defines unique package names for all imported files.

func (*Generator) TypeName 

func (g *Generator) TypeName(obj Object) string

TypeName is the printed name appropriate for an item. If the object is in the current file, TypeName drops the package name and underscores the rest. Otherwise the object is from another package; and the result is the underscored package name followed by the item name. The result always has an initial capital.

func (*Generator) TypeNameByObject 

func (g *Generator) TypeNameByObject(typeName string) Object

func (*Generator) TypeNameWithPackage 

func (g *Generator) TypeNameWithPackage(obj Object) string

TypeNameWithPackage is like TypeName, but always includes the package name even if the object is in our own package.

func (*Generator) WrapTypes 

func (g *Generator) WrapTypes()

WrapTypes walks the incoming data, wrapping DescriptorProtos, EnumDescriptorProtos and FileDescriptorProtos into file-referenced objects within the Generator. It also creates the list of files to generate and so should be called before GenerateAllFiles.

type ImportedDescriptor 

type ImportedDescriptor struct {
	// contains filtered or unexported fields
}

ImportedDescriptor describes a type that has been publicly imported from another file.

func (*ImportedDescriptor) File 

func (c *ImportedDescriptor) File() *descriptor.FileDescriptorProto

func (*ImportedDescriptor) PackageName 

func (c *ImportedDescriptor) PackageName() string

PackageName is name in the package clause in the generated file.

func (*ImportedDescriptor) TypeName 

func (id *ImportedDescriptor) TypeName() []string

type NumGen 

type NumGen interface {
	Next() string
	Current() string
}

func NewNumGen 

func NewNumGen() NumGen

type Object 

type Object interface {
	PackageName() string // The name we use in our output (a_b_c), possibly renamed for uniqueness.
	TypeName() []string
	File() *descriptor.FileDescriptorProto
}

Object is an interface abstracting the abilities shared by enums, messages, extensions and imported objects.

type Plugin 

type Plugin interface {
	// Name identifies the plugin.
	Name() string
	// Init is called once after data structures are built but before
	// code generation begins.
	Init(g *Generator)
	// Generate produces the code generated by the plugin for this file,
	// except for the imports, by calling the generator's methods P, In, and Out.
	Generate(file *FileDescriptor)
	// GenerateImports produces the import declarations for this file.
	// It is called after Generate.
	GenerateImports(file *FileDescriptor)
}

A Plugin provides functionality to add to the output during Go code generation, such as to produce RPC stubs.

type PluginImports 

type PluginImports interface {
	NewImport(pkg string) Single
	GenerateImports(file *FileDescriptor)
}

type Single 

type Single interface {
	Use() string
	IsUsed() bool
	Generate() string
}

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.