parser

package
v0.22.6 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 14, 2024 License: Apache-2.0 Imports: 20 Imported by: 6

README

Puccini TOSCA Parser

Optimized for speed via caching and concurrency. Parsing even very big and complex service templates is practically instantaneous, delayed at worst only by network and filesystem transfer.

Attempts to be as strictly compliant as possible, which is often challenging due to contradictions and unclarity in the TOSCA specification. Where ambiguous we adhere to the spirit of the spec, especially in regards to object-oriented polymorphism. Our prime directive is to ensure that an inherited node type does not break the contract of the base node type.

Coding Principles

There are over 60 different entity types in TOSCA 1.3. Writing custom code for each, even with reusable utility functions, would quickly become a maintenance nightmare, not only for fixing bugs but also for supporting future versions of TOSCA.

We have opted to combine utility functions with plenty of reflection. Reflection is used to solve generic, repeatable actions, such as reading data of various types, looking up names in the namespace, and inheriting fields from parent types. Generic code of this sort is tricky to get right, but once you do the entire domain is managed via simple annotations (field tags).

There is a cost to using such annotations, and indeed even the simple field tags are controversial within the Go commfiley. The problem is that hidden, magical things happen that are not visible in the immediate vicinity of the code in front of you. You have to look elsewhere for the systems that read these tags and do things with them. The only way to reduce this cost is good documentation: make coders aware of these systems and what they do. We will consider this documentation to be a crucial component of the codebase.

Phase 1: Read

This first phase validates syntax. Higher level grammar is validated in subsequent phases.

  1. Read textual data from files and URLs
    • Files/URLs not found
    • I/O errors
    • Textual decoding errors
  2. Parse YAML to ARD
    • YAML parsing errors
  3. Parse ARD to TOSCA data structures, normalizing all short notations to their full form
    • Required fields not set
    • Fields set to wrong YAML type
    • Unsupported fields used
  4. Handle TOSCA imports recursively and concurrently
    • Import causes an endless loop

Phase 2: Namespaces

The goal of this phase is to validate names (ensure that they are not ambiguous), and to provide a mechanism for looking up names. We take into account the import namespace_prefix and support multiple names per entity, as is necessary for the normative types.

Recursively, starting at tips of the import hierarchy:

  1. Gather all names in the file
  2. Apply the import's namespace_prefix if defined
  3. Set names in file's namespace (every entity type has its own section)
  4. Merge namespace into parent file's
    • Ambiguous names (per entity type section)

And then:

  1. Lookup fields from namespace
    • Name not found

Phase 3: Hierarchies

The entire goal of this phase is validation. The entities already have hierarchical information (their Parent field). But here we provide a hierarchy that is guaranteed to not have loops.

Recursively, starting at tips of the import hierarchy:

  1. Gather all TOSCA types in the file
  2. Place types in hierarchy (every type has its own hierarchy)
    • Type's parent causes an endless loop
    • Type's parent is incomplete
  3. Merge hierarchy into parent file's

Phase 4: Inheritance

From this phase we onward only deal with the root file, not the imported files, because it already has all names and types merged in.

This phase is complex because the order of inheritance cannot be determined generally. Not only do types inherit from each other, but also definitions within the types are themselves typed, and those types have their own hierarchy. Thus, a type cannot be resolved before its definitions are, and they cannot be resolved before their types are, and their types' parents, etc.

This includes validating TOSCA's complex inheritance contract, which extends to embedded definitions beyond simple type inheritance. For example, a capability definition within a node type must have a capability type that is compatible with that defined at the parent node type.

Our solution to this complexity is to create a task graph. We will keep resolving independent tasks (those that do not have dependencies), which should in turn make more tasks independent, continuing until all tasks are resolved.

Note that in this phase we handle not only recursive type inheritance, but non-recursive definition inheritance. For example, an interface definition inherits data from the interface type.

  1. Copy over inherited fields from parent or definition type
    • Type's parent is incomplete (due to other problems reported in this phase)
  2. If we are overriding, make sure that we are not breaking the contract
    • Between value of field in parent and child

Incredibly, the TOSCA spec does not describe inheritance. It is non-trivial and not obvious. In Puccini we had to make our own implementation decisions. Unfortunately, other TOSCA parsers may very well handle inheritance differently, leading to incompatibilities. The fault lies entirely with the TOSCA spec.

For example, consider a capability definition within a node type. We inherit first from our parent node type, and only then our capability definition will inherit from its capability type (which may be a subtype of what our parent node type has). The first entity we come across in our traversal is considered to be the final value (for subequently found entities we will treat the field as "already assigned"). The bottom line is that the parent node type takes precedence over the capability type of the capability definition.

At the end of this phase the types are considered "complete" in that we should not need to access their parents for any data. All fields have been inherited.

Phase 5: Rendering

We call the act of applying a type to a template "rendering". ("Instantiation" is what happens next, when we turn a template into an instance, which is out of the scope of Puccini and indeed out of the scope of TOSCA.)

For entities that are "assignments" ("property assignments", "capability assignments", etc.) we do more than just validate: we also change the value, for example by applying defaults.

There are furthermore various custom validations per entity. One worth mentioning here is requirement validation. Specifically, we take into account the occurrences field, which limits the number of times a requirement may be assigned. The TOSCA spec mentions that the implied default occurrences is [1,1] (the TOSCA spec oddly says that in a range the upper bound must be greater than the lower bound, so that [1,1] is impossible: one of many contradictions we must resolve). Puccini will automatically assign requirements to match the lower bound of occurrences, so if the lower bound is 2 and you have only assigned one then an additional one will be assigned for you.

At the end of this phase the templates are considered "complete" in that we should not have to access their types for any data. All fields have been rendered.

Phase 6: Normalization

Converts all the parser's results to Puccini's normalized structures.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Context 

type Context struct {
	Parser *Parser

	URL     exturl.URL
	Bases   []exturl.URL
	Quirks  parsing.Quirks
	Inputs  map[string]ard.Value
	Stylist *terminal.Stylist

	Root  *File
	Files Files
	// contains filtered or unexported fields
}

func (*Context) AddFile added in v0.22.3 

func (self *Context) AddFile(file *File)

func (*Context) AddHierarchies 

func (self *Context) AddHierarchies()

func (*Context) AddImportFile added in v0.22.3 

func (self *Context) AddImportFile(context contextpkg.Context, entityPtr parsing.EntityPtr, container *File, nameTransformer parsing.NameTransformer) *File

func (*Context) AddNamespaces 

func (self *Context) AddNamespaces()

func (*Context) Gather 

func (self *Context) Gather(pattern string) parsing.EntityPtrs

func (*Context) GetProblems 

func (self *Context) GetProblems() *problems.Problems

func (*Context) Inherit added in v0.22.3 

func (self *Context) Inherit(debug func(Tasks))

func (*Context) LookupNames 

func (self *Context) LookupNames()

func (*Context) MergeProblems added in v0.15.0 

func (self *Context) MergeProblems()

func (*Context) Normalize added in v0.22.3 

func (self *Context) Normalize() (*normal.ServiceTemplate, bool)

func (*Context) Parse added in v0.20.0 

func (self *Context) Parse(context contextpkg.Context) (*normal.ServiceTemplate, error)

func (*Context) PrintHierarchies 

func (self *Context) PrintHierarchies(indent int)

func (*Context) PrintImports 

func (self *Context) PrintImports(indent int)

func (*Context) PrintNamespaces 

func (self *Context) PrintNamespaces(indent int)

func (*Context) ReadRoot 

func (self *Context) ReadRoot(context contextpkg.Context, url exturl.URL, bases []exturl.URL, serviceTemplateName string) bool

func (*Context) Render 

func (self *Context) Render() parsing.EntityPtrs

func (*Context) SetInputs added in v0.22.3 

func (self *Context) SetInputs(inputs map[string]ard.Value)

func (*Context) TraverseEntities added in v0.22.3 

func (self *Context) TraverseEntities(log commonlog.Logger, work reflection.EntityWork, traverse reflection.EntityTraverser)

type Executor 

type Executor func(task *Task)

type File added in v0.21.0 

type File struct {
	EntityPtr       parsing.EntityPtr
	Container       *File
	Imports         Files
	NameTransformer parsing.NameTransformer
	// contains filtered or unexported fields
}

func NewEmptyFile added in v0.22.3 

func NewEmptyFile(parsingContext *parsing.Context, container *File, nameTransformer parsing.NameTransformer) *File

func NewFile added in v0.21.0 

func NewFile(entityPtr parsing.EntityPtr, container *File, nameTransformer parsing.NameTransformer) *File

func (*File) AddImport added in v0.21.0 

func (self *File) AddImport(import_ *File)

func (*File) GetContext added in v0.21.0 

func (self *File) GetContext() *parsing.Context

func (*File) PrintImports added in v0.21.0 

func (self *File) PrintImports(indent int, treePrefix terminal.TreePrefix)

func (*File) PrintNode added in v0.21.0 

func (self *File) PrintNode(indent int, treePrefix terminal.TreePrefix, last bool)

type Files added in v0.21.0 

type Files []*File

func (Files) Len added in v0.21.0 

func (self Files) Len() int

(sort.Interface)

func (Files) Less added in v0.21.0 

func (self Files) Less(i, j int) bool

(sort.Interface)

func (Files) Swap added in v0.21.0 

func (self Files) Swap(i, j int)

(sort.Interface)

type InheritContext 

type InheritContext struct {
	Tasks            Tasks
	TasksForEntities TasksForEntities
	InheritFields    InheritFields
}

func NewInheritContext 

func NewInheritContext() *InheritContext

func (*InheritContext) GetDependencies 

func (self *InheritContext) GetDependencies(entityPtr parsing.EntityPtr) parsing.EntityPtrSet

func (*InheritContext) GetInheritTask 

func (self *InheritContext) GetInheritTask(entityPtr parsing.EntityPtr) *Task

func (*InheritContext) NewExecutor 

func (self *InheritContext) NewExecutor(entityPtr parsing.EntityPtr) Executor

type InheritField 

type InheritField struct {
	Entity          reflect.Value
	Key             string
	Field           reflect.Value
	ParentEntityPtr parsing.EntityPtr
	ParentField     reflect.Value
}

func NewInheritFields 

func NewInheritFields(entityPtr parsing.EntityPtr) []*InheritField

From "inherit" tags

func (*InheritField) Inherit 

func (self *InheritField) Inherit()

func (*InheritField) InheritEntity 

func (self *InheritField) InheritEntity()

Field is compatible with *any

func (*InheritField) InheritStringsFromMap 

func (self *InheritField) InheritStringsFromMap()

Field is *map[string]string

func (*InheritField) InheritStringsFromSlice 

func (self *InheritField) InheritStringsFromSlice()

Field is *[]string

func (*InheritField) InheritStructsFromMap 

func (self *InheritField) InheritStructsFromMap()

Field is compatible with map[string]*any

func (*InheritField) InheritStructsFromSlice 

func (self *InheritField) InheritStructsFromSlice()

Field is compatible with []*any

type InheritFields 

type InheritFields map[parsing.EntityPtr][]*InheritField

func (InheritFields) Get 

func (self InheritFields) Get(entityPtr parsing.EntityPtr) []*InheritField

type LookupField 

type LookupField struct {
	Types []reflect.Type
	Names []LookupName
}

type LookupName 

type LookupName struct {
	Index int
	Name  string
	Found bool
}

type LookupProblems 

type LookupProblems map[string]*LookupField

func (LookupProblems) AddType 

func (self LookupProblems) AddType(key string, type_ reflect.Type)

func (LookupProblems) Field 

func (self LookupProblems) Field(key string) *LookupField

func (LookupProblems) Report 

func (self LookupProblems) Report(parsingContext *parsing.Context)

func (LookupProblems) SetFound 

func (self LookupProblems) SetFound(key string, index int, name string, found bool)

type Parser added in v0.22.3 

type Parser struct {
	// contains filtered or unexported fields
}

func NewParser added in v0.22.3 

func NewParser() *Parser

func (*Parser) NewContext added in v0.22.3 

func (self *Parser) NewContext() *Context

type Task 

type Task struct {
	Name         string
	Executor     Executor
	Parents      Tasks
	Dependencies Tasks
}

func NewTask 

func NewTask(name string) *Task

func (*Task) AddDependency 

func (self *Task) AddDependency(task *Task)

func (*Task) Done 

func (self *Task) Done()

func (*Task) Execute 

func (self *Task) Execute()

func (*Task) IsIndependent 

func (self *Task) IsIndependent() bool

func (*Task) Print 

func (self *Task) Print(indent int)

func (*Task) PrintDependencies 

func (self *Task) PrintDependencies(indent int, treePrefix terminal.TreePrefix)

func (*Task) PrintDependency 

func (self *Task) PrintDependency(indent int, treePrefix terminal.TreePrefix, last bool)

type TaskList 

type TaskList []*Task

func (TaskList) Len 

func (self TaskList) Len() int

(sort.Interface)

func (TaskList) Less 

func (self TaskList) Less(i, j int) bool

(sort.Interface)

func (TaskList) Swap 

func (self TaskList) Swap(i, j int)

(sort.Interface)

type Tasks 

type Tasks map[*Task]struct{}

func (Tasks) Add 

func (self Tasks) Add(task *Task)

func (Tasks) Drain 

func (self Tasks) Drain()

func (Tasks) FindIndependent 

func (self Tasks) FindIndependent() (*Task, bool)

func (Tasks) Print 

func (self Tasks) Print(indent int)

func (Tasks) Remove 

func (self Tasks) Remove(task *Task)

func (Tasks) Validate 

func (self Tasks) Validate() bool

type TasksForEntities 

type TasksForEntities map[parsing.EntityPtr]*Task

type YAMLDecodeError added in v0.12.0 

type YAMLDecodeError struct {
	DecodeError *yamlkeys.DecodeError
}

func NewYAMLDecodeError added in v0.12.0 

func NewYAMLDecodeError(decodeError *yamlkeys.DecodeError) *YAMLDecodeError

func (*YAMLDecodeError) Error added in v0.12.0 

func (self *YAMLDecodeError) Error() string

(error interface)

func (*YAMLDecodeError) Problem added in v0.12.0 

func (self *YAMLDecodeError) Problem(stylist terminal.Stylist) (string, string, string, int, int)

problems.Problematic interface

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.