parser

Documentation

Overview

Package parser contains the logic for parsing protobuf source code into an AST (abstract syntax tree) and also for converting an AST into a descriptor proto.

A FileDescriptorProto is very similar to an AST, but the AST this package uses is more useful because it contains more information about the source code, including details about whitespace and comments, that cannot be represented by a descriptor proto. This makes it ideal for things like code formatters, which may want to preserve things like whitespace and comment format.

Code generated by goyacc -o proto.y.go -l -p proto proto.y. DO NOT EDIT.

Index

• Constants
• Variables
• func Parse(filename string, r io.Reader, handler *reporter.Handler, version int32) (*ast.FileNode, error)
• type ExtendedSyntaxError
  • func NewExtendedSyntaxError(base error, category string) ExtendedSyntaxError
• type ParseError
  • func NewParseError(base error) ParseError
• type Result
  • func Clone(r Result) Result
  • func ResultFromAST(file *ast.FileNode, validate bool, handler *reporter.Handler) (Result, error)
  • func ResultWithoutAST(proto *descriptorpb.FileDescriptorProto) Result

Constants

const (
	CategoryEmptyDecl      = "empty_decl"
	CategoryIncompleteDecl = "incomplete_decl"
	CategoryExtraTokens    = "extra_tokens"
	CategoryIncorrectToken = "wrong_token"
	CategoryMissingToken   = "missing_token"
	CategoryDeclNotAllowed = "decl_not_allowed"
)

Variables

var ErrNoSyntax = errors.New("no syntax specified; defaulting to proto2 syntax")

ErrNoSyntax is a sentinel error that may be passed to a warning reporter. The error the reporter receives will be wrapped with source position that indicates the file that had no syntax statement.

Functions

func Parse

func Parse(filename string, r io.Reader, handler *reporter.Handler, version int32) (*ast.FileNode, error)

Parse parses the given source code info and returns an AST. The given filename is used to construct error messages and position information. The given reader supplies the source code. The given handler is used to report errors and warnings encountered while parsing. If any errors are reported, this function returns a non-nil error.

If the error returned is due to a syntax error in the source, then a non-nil AST is also returned. If the handler chooses to not abort the parse (e.g. the underlying error reporter returns nil instead of an error), the parser will attempt to recover and keep going. This allows multiple syntax errors to be reported in a single pass. And it also means that more of the AST can be populated (erroneous productions around the syntax error will of course be absent).

The degree to which the parser can recover from errors and populate the AST depends on the nature of the syntax error and if there are any tokens after the syntax error that can help the parser recover. This error recovery and partial AST production is best effort.

Types

type ExtendedSyntaxError

type ExtendedSyntaxError interface {
	error

	Category() string
	CanFormat() bool
	// contains filtered or unexported methods
}

func NewExtendedSyntaxError

func NewExtendedSyntaxError(base error, category string) ExtendedSyntaxError

type ParseError

type ParseError interface {
	error
	// contains filtered or unexported methods
}

func NewParseError

func NewParseError(base error) ParseError

type Result

type Result interface {
	// AST returns the parsed abstract syntax tree. This returns nil if the
	// Result was created without an AST.
	AST() *ast.FileNode
	// FileDescriptorProto returns the file descriptor proto.
	FileDescriptorProto() *descriptorpb.FileDescriptorProto

	// FileNode returns the root of the AST. If this result has no AST then a
	// placeholder node is returned.
	FileNode() *ast.FileNode
	// Node returns the AST node from which the given message was created. This
	// can return nil, such as if the given message is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	Node(proto.Message) ast.Node
	// OptionNode returns the AST node corresponding to the given uninterpreted
	// option. This can return nil, such as if the given option is not part of
	// the FileDescriptorProto hierarchy. If this result has no AST, this
	// returns a placeholder node.
	OptionNode(*descriptorpb.UninterpretedOption) *ast.OptionNode
	// OptionNamePartNode returns the AST node corresponding to the given name
	// part for an uninterpreted option. This can return nil, such as if the
	// given name part is not part of the FileDescriptorProto hierarchy. If this
	// result has no AST, this returns a placeholder node.
	OptionNamePartNode(*descriptorpb.UninterpretedOption_NamePart) ast.Node
	// MessageNode returns the AST node corresponding to the given message. This
	// can return nil, such as if the given message is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	MessageNode(*descriptorpb.DescriptorProto) *ast.MessageDeclNode
	// FieldNode returns the AST node corresponding to the given field. This can
	// return nil, such as if the given field is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	FieldNode(*descriptorpb.FieldDescriptorProto) *ast.FieldDeclNode
	// FieldExtendeeNode returns the containing *ExtendNode for fields that are
	// defined inside of "extend" blocks.
	FieldExtendeeNode(*descriptorpb.FieldDescriptorProto) *ast.ExtendNode
	// OneofNode returns the AST node corresponding to the given oneof. This can
	// return nil, such as if the given oneof is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	OneofNode(*descriptorpb.OneofDescriptorProto) *ast.OneofNode
	// ExtensionRangeNode returns the AST node corresponding to the given
	// extension range. This can return nil, such as if the given range is not
	// part of the FileDescriptorProto hierarchy. If this result has no AST,
	// this returns a placeholder node.
	ExtensionRangeNode(*descriptorpb.DescriptorProto_ExtensionRange) *ast.RangeNode
	// MessageReservedRangeNode returns the AST node corresponding to the given
	// reserved range. This can return nil, such as if the given range is not
	// part of the FileDescriptorProto hierarchy. If this result has no AST,
	// this returns a placeholder node.
	MessageReservedRangeNode(*descriptorpb.DescriptorProto_ReservedRange) *ast.RangeNode
	// EnumNode returns the AST node corresponding to the given enum. This can
	// return nil, such as if the given enum is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	EnumNode(*descriptorpb.EnumDescriptorProto) *ast.EnumNode
	// EnumValueNode returns the AST node corresponding to the given enum. This
	// can return nil, such as if the given enum value is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	EnumValueNode(*descriptorpb.EnumValueDescriptorProto) *ast.EnumValueNode
	// EnumReservedRangeNode returns the AST node corresponding to the given
	// reserved range. This can return nil, such as if the given range is not
	// part of the FileDescriptorProto hierarchy. If this result has no AST,
	// this returns a placeholder node.
	EnumReservedRangeNode(*descriptorpb.EnumDescriptorProto_EnumReservedRange) *ast.RangeNode
	// ServiceNode returns the AST node corresponding to the given service. This
	// can return nil, such as if the given service is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	ServiceNode(*descriptorpb.ServiceDescriptorProto) *ast.ServiceNode
	// MethodNode returns the AST node corresponding to the given method. This
	// can return nil, such as if the given method is not part of the
	// FileDescriptorProto hierarchy. If this result has no AST, this returns a
	// placeholder node.
	MethodNode(*descriptorpb.MethodDescriptorProto) *ast.RPCNode

	OptionDescriptor(*ast.OptionNode) *descriptorpb.UninterpretedOption
	OptionNamePartDescriptor(ast.Node) *descriptorpb.UninterpretedOption_NamePart
	MessageDescriptor(*ast.MessageDeclNode) *descriptorpb.DescriptorProto
	FieldDescriptor(*ast.FieldDeclNode) *descriptorpb.FieldDescriptorProto
	OneofDescriptor(ast.Node) *descriptorpb.OneofDescriptorProto
	ExtensionRangeDescriptor(*ast.RangeNode) *descriptorpb.DescriptorProto_ExtensionRange
	MessageReservedRangeDescriptor(*ast.RangeNode) *descriptorpb.DescriptorProto_ReservedRange
	EnumDescriptor(*ast.EnumNode) *descriptorpb.EnumDescriptorProto
	EnumValueDescriptor(*ast.EnumValueNode) *descriptorpb.EnumValueDescriptorProto
	EnumReservedRangeDescriptor(*ast.RangeNode) *descriptorpb.EnumDescriptorProto_EnumReservedRange
	ServiceDescriptor(*ast.ServiceNode) *descriptorpb.ServiceDescriptorProto
	MethodDescriptor(*ast.RPCNode) *descriptorpb.MethodDescriptorProto
	Descriptor(ast.Node) proto.Message

	ImportInsertionPoint() ast.SourcePos
}

Result is the result of constructing a descriptor proto from a parsed AST. From this result, the AST and the file descriptor proto can be had. This also contains numerous lookup functions, for looking up AST nodes that correspond to various elements of the descriptor hierarchy.

Results can be created without AST information, using the ResultWithoutAST() function. All functions other than AST() will still return non-nil values, allowing compile operations to work with files that have only intermediate descriptor protos and no source code. For such results, the function that return AST nodes will return placeholder nodes. The position information for placeholder nodes contains only the filename.

func Clone

func Clone(r Result) Result

Clone returns a copy of the given result. Since descriptor protos may be mutated during linking, this can return a defensive copy so that mutations don't impact concurrent operations in an unsafe way. This is called if the parse result could be re-used across concurrent operations and has unresolved references and options which will require mutation by the linker.

If the given value has a method with the following signature, it will be called to perform the operation:

Clone() Result

If the given value does not provide a Clone method and is not the implementation provided by this package, it is possible for an error to occur in creating the copy, which may result in a panic. This can happen if the AST of the given result is not actually valid and a file descriptor proto cannot be successfully derived from it.

func ResultFromAST

func ResultFromAST(file *ast.FileNode, validate bool, handler *reporter.Handler) (Result, error)

ResultFromAST constructs a descriptor proto from the given AST. The returned result includes the descriptor proto and also contains an index that can be used to lookup AST node information for elements in the descriptor proto hierarchy.

If validate is true, some basic validation is performed, to make sure the resulting descriptor proto is valid per protobuf rules and semantics. Only some language elements can be validated since some rules and semantics can only be checked after all symbols are all resolved, which happens in the linking step.

The given handler is used to report any errors or warnings encountered. If any errors are reported, this function returns a non-nil error.

func ResultWithoutAST

func ResultWithoutAST(proto *descriptorpb.FileDescriptorProto) Result

ResultWithoutAST returns a parse result that has no AST. All methods for looking up AST nodes return a placeholder node that contains only the filename in position information.