docs

package
v3.50.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 19, 2021 License: MIT Imports: 18 Imported by: 0

Documentation

Overview

Package docs provides useful functions for creating documentation from Benthos components

Index

Constants

This section is empty.

Variables

View Source 
var (

	// ErrBadLabel is returned when creating a component with a bad label.
	ErrBadLabel = fmt.Errorf("should match the regular expression /%v/ and must not start with an underscore", labelExpression)
)

Functions

func BloblangFunctionsMarkdown added in v3.26.0 

func BloblangFunctionsMarkdown() ([]byte, error)

BloblangFunctionsMarkdown returns a markdown document for all Bloblang functions.

func BloblangMethodsMarkdown added in v3.26.0 

func BloblangMethodsMarkdown() ([]byte, error)

BloblangMethodsMarkdown returns a markdown document for all Bloblang methods.

func ComponentFieldsFromConf added in v3.48.0 

func ComponentFieldsFromConf(conf interface{}) (inferred map[string]FieldSpecs)

ComponentFieldsFromConf walks the children of a YAML node and returns a list of fields extracted from it. This can be used in order to infer a field spec for a parsed component.

TODO: V5 Remove this eventually.

func FieldsTemplate added in v3.48.0 

func FieldsTemplate(lintableExamples bool) string

FieldsTemplate returns a Go template for rendering markdown field documentation. The context should have a field `.Fields` of the type `[]FieldSpecCtx`.

func GetPluginConfigYAML added in v3.48.0 

func GetPluginConfigYAML(name string, node *yaml.Node) (yaml.Node, error)

GetPluginConfigYAML extracts a plugin configuration node from a component config. This exists because there are two styles of plugin config, the old style (within `plugin`):

type: foo plugin: 

bar: baz

And the new style:

foo: 

bar: baz

func GetYAMLPath added in v3.50.0 

func GetYAMLPath(root *yaml.Node, path ...string) (*yaml.Node, error)

GetYAMLPath attempts to obtain a specific value within a YAML tree by following a sequence of path identifiers.

func RegisterDocs added in v3.43.0 

func RegisterDocs(spec ComponentSpec)

RegisterDocs stores the documentation spec for a component.

func SanitiseComponentConfig added in v3.43.0 

func SanitiseComponentConfig(componentType Type, raw interface{}, filter FieldFilter) error

SanitiseComponentConfig reduces a raw component configuration into only the fields for the component name configured.

TODO: V4 Remove this

func SanitiseYAML added in v3.48.0 

func SanitiseYAML(cType Type, node *yaml.Node, conf SanitiseConfig) error

SanitiseYAML takes a yaml.Node and a config spec and sorts the fields of the node according to the spec. Also optionally removes the `type` field from this and all nested components.

func ValidateLabel added in v3.43.0 

func ValidateLabel(label string) error

ValidateLabel attempts to validate the contents of a component label.

Types

type AnnotatedExample added in v3.25.0 

type AnnotatedExample struct {
	// A title for the example.
	Title string `json:"title"`

	// Summary of the example.
	Summary string `json:"summary"`

	// A config snippet to show.
	Config string `json:"config"`
}

AnnotatedExample is an isolated example for a component.

type ComponentSpec 

type ComponentSpec struct {
	// Name of the component
	Name string `json:"name"`

	// Type of the component (input, output, etc)
	Type Type `json:"type"`

	// The status of the component.
	Status Status `json:"status"`

	// Plugin is true for all plugin components.
	Plugin bool `json:"plugin"`

	// Summary of the component (in markdown, must be short).
	Summary string `json:"summary"`

	// Description of the component (in markdown).
	Description string `json:"description"`

	// Categories that describe the purpose of the component.
	Categories []string `json:"categories"`

	// Footnotes of the component (in markdown).
	Footnotes string `json:"footnotes"`

	// Examples demonstrating use cases for the component.
	Examples []AnnotatedExample `json:"examples,omitempty"`

	// A summary of each field in the component configuration.
	Config FieldSpec `json:"config"`

	// Version is the Benthos version this component was introduced.
	Version string `json:"version,omitempty"`
}

ComponentSpec describes a Benthos component.

func GetDocs added in v3.43.0 

func GetDocs(name string, ctype Type) (ComponentSpec, bool)

GetDocs attempts to locate a documentation spec for a component identified by a unique name and type combination.

func GetInferenceCandidate added in v3.43.0 

func GetInferenceCandidate(docProvider Provider, t Type, defaultType string, raw interface{}) (string, ComponentSpec, error)

GetInferenceCandidate checks a generic config structure for a component and returns either the inferred type name or an error if one cannot be inferred.

func GetInferenceCandidateFromYAML added in v3.48.0 

func GetInferenceCandidateFromYAML(docProv Provider, t Type, defaultType string, node *yaml.Node) (string, ComponentSpec, error)

GetInferenceCandidateFromYAML checks a yaml node config structure for a component and returns either the inferred type name or an error if one cannot be inferred.

func (*ComponentSpec) AsMarkdown 

func (c *ComponentSpec) AsMarkdown(nest bool, fullConfigExample interface{}) ([]byte, error)

AsMarkdown renders the spec of a component, along with a full configuration example, into a markdown document.

type FieldFilter added in v3.43.0 

type FieldFilter func(spec FieldSpec) bool

FieldFilter defines a filter closure that returns a boolean for a component field indicating whether the field should be kept within a generated config.

func ShouldDropDeprecated added in v3.43.0 

func ShouldDropDeprecated(b bool) FieldFilter

ShouldDropDeprecated returns a field filter that removes all deprecated fields when the boolean argument is true.

type FieldKind added in v3.48.0 

type FieldKind string

FieldKind represents a field kind. 

var (
	KindScalar  FieldKind = "scalar"
	KindArray   FieldKind = "array"
	Kind2DArray FieldKind = "2darray"
	KindMap     FieldKind = "map"
)

ValueType variants.

type FieldSpec 

type FieldSpec struct {
	// Name of the field (as it appears in config).
	Name string `json:"name"`

	// Type of the field.
	//
	// TODO: Make this mandatory
	Type FieldType `json:"type"`

	// Kind of the field.
	Kind FieldKind `json:"kind"`

	// Description of the field purpose (in markdown).
	Description string `json:"description"`

	// IsAdvanced is true for optional fields that will not be present in most
	// configs.
	IsAdvanced bool `json:"is_advanced"`

	// IsDeprecated is true for fields that are deprecated and only exist
	// for backwards compatibility reasons.
	IsDeprecated bool `json:"is_deprecated"`

	// IsOptional is a boolean flag indicating that a field is optional, even
	// if there is no default. This prevents linting errors when the field
	// is missing.
	IsOptional bool `json:"is_optional"`

	// Default value of the field.
	Default *interface{} `json:"default,omitempty"`

	// Interpolation indicates that the field supports interpolation
	// functions.
	Interpolated bool `json:"interpolated"`

	// Examples is a slice of optional example values for a field.
	Examples []interface{} `json:"examples,omitempty"`

	// AnnotatedOptions for this field. Each option should have a summary.
	AnnotatedOptions [][2]string `json:"annotated_options,omitempty"`

	// Options for this field.
	Options []string `json:"options,omitempty"`

	// Children fields of this field (it must be an object).
	Children FieldSpecs `json:"children,omitempty"`

	// Version is an explicit version when this field was introduced.
	Version string `json:"version,omitempty"`
	// contains filtered or unexported fields
}

FieldSpec describes a component config field.

func FieldAdvanced 

func FieldAdvanced(name, description string, examples ...interface{}) FieldSpec

FieldAdvanced returns a field spec for an advanced field.

func FieldBool added in v3.28.0 

func FieldBool(name, description string, examples ...interface{}) FieldSpec

FieldBool returns a field spec for a common bool typed field.

func FieldCommon 

func FieldCommon(name, description string, examples ...interface{}) FieldSpec

FieldCommon returns a field spec for a common field.

func FieldComponent added in v3.43.0 

func FieldComponent() FieldSpec

FieldComponent returns a field spec for a component.

func FieldDeprecated 

func FieldDeprecated(name string, description ...string) FieldSpec

FieldDeprecated returns a field spec for a deprecated field.

func FieldFloat added in v3.47.0 

func FieldFloat(name, description string, examples ...interface{}) FieldSpec

FieldFloat returns a field spec for a common float typed field.

func FieldFromYAML added in v3.48.0 

func FieldFromYAML(name string, node *yaml.Node) FieldSpec

FieldFromYAML infers a field spec from a YAML node. This mechanism has many limitations and should only be used for pre-hydrating field specs for old components with struct based config.

func FieldInt added in v3.47.0 

func FieldInt(name, description string, examples ...interface{}) FieldSpec

FieldInt returns a field spec for a common int typed field.

func FieldString added in v3.28.0 

func FieldString(name, description string, examples ...interface{}) FieldSpec

FieldString returns a field spec for a common string typed field.

func (FieldSpec) Advanced 

func (f FieldSpec) Advanced() FieldSpec

Advanced marks this field as being advanced, and therefore not commonly used.

func (FieldSpec) Array added in v3.43.0 

func (f FieldSpec) Array() FieldSpec

Array determines that this field is an array of the field type.

func (FieldSpec) ArrayOfArrays added in v3.48.0 

func (f FieldSpec) ArrayOfArrays() FieldSpec

ArrayOfArrays determines that this is an array of arrays of the field type.

func (FieldSpec) AtVersion added in v3.34.0 

func (f FieldSpec) AtVersion(v string) FieldSpec

AtVersion specifies the version at which this fields behaviour was last modified.

func (FieldSpec) ChildDefaultAndTypesFromStruct added in v3.48.0 

func (f FieldSpec) ChildDefaultAndTypesFromStruct(conf interface{}) FieldSpec

ChildDefaultAndTypesFromStruct enriches a field specs children with a type string and default value from another field spec inferred from a config struct.

TODO: V5 Remove this eventually.

func (FieldSpec) DefaultAndTypeFrom added in v3.48.0 

func (f FieldSpec) DefaultAndTypeFrom(from FieldSpec) FieldSpec

DefaultAndTypeFrom enriches a field spec with a type string and default value from another field spec.

func (FieldSpec) FlattenChildrenForDocs added in v3.47.0 

func (f FieldSpec) FlattenChildrenForDocs() []FieldSpecCtx

FlattenChildrenForDocs converts the children of a field into a flat list, where the names contain hints as to their position in a structured hierarchy. This makes it easier to list the fields in documentation.

func (FieldSpec) HasAnnotatedOptions added in v3.33.0 

func (f FieldSpec) HasAnnotatedOptions(options ...string) FieldSpec

HasAnnotatedOptions returns a new FieldSpec that specifies a specific list of annotated options. Either

func (FieldSpec) HasDefault added in v3.28.0 

func (f FieldSpec) HasDefault(v interface{}) FieldSpec

HasDefault returns a new FieldSpec that specifies a default value.

func (FieldSpec) HasOptions 

func (f FieldSpec) HasOptions(options ...string) FieldSpec

HasOptions returns a new FieldSpec that specifies a specific list of options.

func (FieldSpec) HasType 

func (f FieldSpec) HasType(t FieldType) FieldSpec

HasType returns a new FieldSpec that specifies a specific type.

func (FieldSpec) IsInterpolated added in v3.43.0 

func (f FieldSpec) IsInterpolated() FieldSpec

IsInterpolated indicates that the field supports interpolation functions.

func (FieldSpec) JSONSchema added in v3.49.0 

func (f FieldSpec) JSONSchema() interface{}

JSONSchema serializes a field spec into a JSON schema structure.

func (FieldSpec) LintOptions added in v3.47.0 

func (f FieldSpec) LintOptions() FieldSpec

LintOptions enforces that a field value matches one of the provided options and returns a linting error if that is not the case. This is currently opt-in because some fields express options that are only a subset due to deprecated functionality.

TODO: V4 Switch this to opt-out.

func (FieldSpec) LintYAML added in v3.48.0 

func (f FieldSpec) LintYAML(ctx LintContext, node *yaml.Node) []Lint

LintYAML returns a list of linting errors found by checking a field definition against a yaml node.

func (FieldSpec) Linter added in v3.43.0 

func (f FieldSpec) Linter(fn LintFunc) FieldSpec

Linter adds a linting function to a field. When linting is performed on a config the provided function will be called with a boxed variant of the field value, allowing it to perform linting on that value.

func (FieldSpec) Map added in v3.43.0 

func (f FieldSpec) Map() FieldSpec

Map determines that this field is a map of arbitrary keys to a field type.

func (FieldSpec) OmitWhen added in v3.43.0 

func (f FieldSpec) OmitWhen(fn func(field, parent interface{}) (why string, shouldOmit bool)) FieldSpec

OmitWhen specifies a custom func that, when provided a generic config struct, returns a boolean indicating when the field can be safely omitted from a config.

func (FieldSpec) Optional added in v3.48.0 

func (f FieldSpec) Optional() FieldSpec

Optional marks this field as being optional, and therefore its absence in a config is not considered an error if when a default is not provided.

func (FieldSpec) SanitiseYAML added in v3.48.0 

func (f FieldSpec) SanitiseYAML(node *yaml.Node, conf SanitiseConfig) error

SanitiseYAML attempts to reduce a parsed config (as a *yaml.Node) down into a minimal representation without changing the behaviour of the config. The fields of the result will also be sorted according to the field spec.

func (FieldSpec) Scalar added in v3.48.0 

func (f FieldSpec) Scalar() FieldSpec

Scalar determines that this field is a scalar type (the default).

func (FieldSpec) SetYAMLPath added in v3.50.0 

func (f FieldSpec) SetYAMLPath(docsProvider Provider, root, value *yaml.Node, path ...string) error

SetYAMLPath sets the value of a node within a YAML document identified by a path to a value.

func (FieldSpec) ToYAML added in v3.48.0 

func (f FieldSpec) ToYAML(recurse bool) (*yaml.Node, error)

ToYAML creates a YAML node from a field spec. If a default value has been specified then it is used. Otherwise, a zero value is generated. If recurse is enabled and the field has children then all children will also have values generated.

func (FieldSpec) Unlinted added in v3.43.0 

func (f FieldSpec) Unlinted() FieldSpec

Unlinted returns a field spec that will not be lint checked during a config parse.

func (FieldSpec) WithChildren 

func (f FieldSpec) WithChildren(children ...FieldSpec) FieldSpec

WithChildren returns a new FieldSpec that has child fields.

func (FieldSpec) YAMLLabelsToPaths added in v3.50.0 

func (f FieldSpec) YAMLLabelsToPaths(docsProvider Provider, node *yaml.Node, labelsToPaths map[string][]string, path []string)

YAMLLabelsToPaths walks a YAML tree using a field spec as a reference point. When a component of the YAML tree has a label field it is added to the provided labelsToPaths map with the path to the component.

func (FieldSpec) YAMLToValue added in v3.48.0 

func (f FieldSpec) YAMLToValue(node *yaml.Node, conf ToValueConfig) (interface{}, error)

YAMLToValue converts a yaml node into a generic value by referencing the expected type.

type FieldSpecCtx added in v3.48.0 

type FieldSpecCtx struct {
	Spec FieldSpec

	// FullName describes the full dot path name of the field relative to
	// the root of the documented component.
	FullName string

	// ExamplesMarshalled is a list of examples marshalled into YAML format.
	ExamplesMarshalled []string

	// DefaultMarshalled is a marshalled string of the default value, if there is one.
	DefaultMarshalled string
}

FieldSpecCtx provides a field spec and rendered extras for documentation templates to use.

type FieldSpecs 

type FieldSpecs []FieldSpec

FieldSpecs is a slice of field specs for a component.

func FieldsFromConf added in v3.48.0 

func FieldsFromConf(conf interface{}) FieldSpecs

FieldsFromConf attempts to infer field documents from a config struct.

func FieldsFromYAML added in v3.48.0 

func FieldsFromYAML(node *yaml.Node) FieldSpecs

FieldsFromYAML walks the children of a YAML node and returns a list of fields extracted from it. This can be used in order to infer a field spec for a parsed component.

func (FieldSpecs) Add 

func (f FieldSpecs) Add(specs ...FieldSpec) FieldSpecs

Add more field specs.

func (FieldSpecs) DefaultAndTypeFrom added in v3.48.0 

func (f FieldSpecs) DefaultAndTypeFrom(from FieldSpecs) FieldSpecs

DefaultAndTypeFrom enriches a field spec with a type string and default value from another field spec.

func (FieldSpecs) JSONSchema added in v3.49.0 

func (f FieldSpecs) JSONSchema() map[string]interface{}

JSONSchema serializes a field spec into a JSON schema structure.

func (FieldSpecs) LintYAML added in v3.48.0 

func (f FieldSpecs) LintYAML(ctx LintContext, node *yaml.Node) []Lint

LintYAML walks a yaml node and returns a list of linting errors found.

func (FieldSpecs) Merge 

func (f FieldSpecs) Merge(specs FieldSpecs) FieldSpecs

Merge with another set of FieldSpecs.

func (FieldSpecs) SanitiseYAML added in v3.48.0 

func (f FieldSpecs) SanitiseYAML(node *yaml.Node, conf SanitiseConfig) error

SanitiseYAML attempts to reduce a parsed config (as a *yaml.Node) down into a minimal representation without changing the behaviour of the config. The fields of the result will also be sorted according to the field spec.

func (FieldSpecs) SetYAMLPath added in v3.50.0 

func (f FieldSpecs) SetYAMLPath(docsProvider Provider, root, value *yaml.Node, path ...string) error

SetYAMLPath sets the value of a node within a YAML document identified by a path to a value.

func (FieldSpecs) ToYAML added in v3.48.0 

func (f FieldSpecs) ToYAML() (*yaml.Node, error)

ToYAML creates a YAML node from a list of field specs. If a default value has been specified for a given field then it is used. Otherwise, a zero value is generated.

func (FieldSpecs) YAMLLabelsToPaths added in v3.50.0 

func (f FieldSpecs) YAMLLabelsToPaths(docsProvider Provider, node *yaml.Node, labelsToPaths map[string][]string, path []string)

YAMLLabelsToPaths walks a YAML tree using a field spec as a reference point. When a component of the YAML tree has a label field it is added to the provided labelsToPaths map with the path to the component.

func (FieldSpecs) YAMLToMap added in v3.48.0 

func (f FieldSpecs) YAMLToMap(node *yaml.Node, conf ToValueConfig) (map[string]interface{}, error)

YAMLToMap converts a yaml node into a generic map structure by referencing expected fields, adding default values to the map when the node does not contain them.

type FieldType added in v3.28.0 

type FieldType string

FieldType represents a field type. 

var (
	FieldTypeString  FieldType = "string"
	FieldTypeInt     FieldType = "int"
	FieldTypeFloat   FieldType = "float"
	FieldTypeBool    FieldType = "bool"
	FieldTypeObject  FieldType = "object"
	FieldTypeUnknown FieldType = "unknown"

	// Core component types, only components that can be a child of another
	// component config are listed here.
	FieldTypeInput     FieldType = "input"
	FieldTypeBuffer    FieldType = "buffer"
	FieldTypeCache     FieldType = "cache"
	FieldTypeCondition FieldType = "condition"
	FieldTypeProcessor FieldType = "processor"
	FieldTypeRateLimit FieldType = "rate_limit"
	FieldTypeOutput    FieldType = "output"
	FieldTypeMetrics   FieldType = "metrics"
	FieldTypeTracer    FieldType = "tracer"
)

ValueType variants.

func (FieldType) IsCoreComponent added in v3.43.0 

func (t FieldType) IsCoreComponent() (Type, bool)

IsCoreComponent returns the core component type of a field if applicable.

type Lint added in v3.43.0 

type Lint struct {
	Line   int
	Column int // Optional, omitted from lint report unless >= 1
	Level  LintLevel
	What   string
}

Lint describes a single linting issue found with a Benthos config.

func LintBloblangField added in v3.43.0 

func LintBloblangField(ctx LintContext, line, col int, v interface{}) []Lint

LintBloblangField is function for linting a config field expected to be an interpolation string.

func LintBloblangMapping added in v3.43.0 

func LintBloblangMapping(ctx LintContext, line, col int, v interface{}) []Lint

LintBloblangMapping is function for linting a config field expected to be a bloblang mapping.

func LintYAML added in v3.48.0 

func LintYAML(ctx LintContext, cType Type, node *yaml.Node) []Lint

LintYAML takes a yaml.Node and a config spec and returns a list of linting errors found in the config.

func NewLintError added in v3.43.0 

func NewLintError(line int, msg string) Lint

NewLintError returns an error lint.

func NewLintWarning added in v3.43.0 

func NewLintWarning(line int, msg string) Lint

NewLintWarning returns a warning lint.

type LintContext added in v3.43.0 

type LintContext struct {
	// A map of label names to the line they were defined at.
	LabelsToLine map[string]int

	// DocsProvider provides documentation for component implementations.
	DocsProvider Provider
}

LintContext is provided to linting functions, and provides context about the wider configuration.

func NewLintContext added in v3.43.0 

func NewLintContext() LintContext

NewLintContext creates a new linting context.

type LintFunc added in v3.43.0 

type LintFunc func(ctx LintContext, line, col int, value interface{}) []Lint

LintFunc is a common linting function for field values.

type LintLevel added in v3.43.0 

type LintLevel int

LintLevel describes the severity level of a linting error. 

const (
	LintError   LintLevel = iota
	LintWarning LintLevel = iota
)

Lint levels

type MappedDocsProvider added in v3.49.0 

type MappedDocsProvider struct {
	// contains filtered or unexported fields
}

MappedDocsProvider stores component documentation in maps, protected by a mutex, allowing safe concurrent use.

func NewMappedDocsProvider added in v3.49.0 

func NewMappedDocsProvider() *MappedDocsProvider

NewMappedDocsProvider creates a new (empty) provider of component docs.

func (*MappedDocsProvider) GetDocs added in v3.49.0 

func (m *MappedDocsProvider) GetDocs(name string, ctype Type) (ComponentSpec, bool)

GetDocs attempts to obtain component implementation docs.

func (*MappedDocsProvider) RegisterDocs added in v3.49.0 

func (m *MappedDocsProvider) RegisterDocs(spec ComponentSpec)

RegisterDocs adds the documentation of a component implementation.

type Provider added in v3.49.0 

type Provider interface {
	GetDocs(name string, ctype Type) (ComponentSpec, bool)
}

Provider stores the component spec definitions of various component implementations.

type SanitiseConfig added in v3.43.0 

type SanitiseConfig struct {
	RemoveTypeField  bool
	RemoveDeprecated bool
	ForExample       bool
	Filter           FieldFilter
	DocsProvider     Provider
}

SanitiseConfig contains fields describing the desired behaviour of the config sanitiser such as removing certain fields.

func (SanitiseConfig) GetDocs added in v3.49.0 

func (c SanitiseConfig) GetDocs(name string, ctype Type) (ComponentSpec, bool)

GetDocs attempts to obtain documentation for a component implementation from a docs provider in the config, or if omitted uses the global provider.

type Status added in v3.32.0 

type Status string

Status of a component. 

var (
	StatusStable       Status = "stable"
	StatusBeta         Status = "beta"
	StatusExperimental Status = "experimental"
	StatusDeprecated   Status = "deprecated"
)

Component statuses.

type ToValueConfig added in v3.48.0 

type ToValueConfig struct {
	// Whether an problem in the config node detected during conversion
	// should return a "best attempt" structure rather than an error.
	Passive bool

	// When a field spec is for a non-scalar type (a component) fall back to
	// decoding it into an interface, otherwise the raw yaml.Node is
	// returned in its place.
	FallbackToInterface bool
}

ToValueConfig describes custom options for how documentation fields should be used to convert a parsed node to a value type.

type Type added in v3.43.0 

type Type string

Type of a component. 

var (
	TypeBuffer    Type = "buffer"
	TypeCache     Type = "cache"
	TypeInput     Type = "input"
	TypeMetrics   Type = "metrics"
	TypeOutput    Type = "output"
	TypeProcessor Type = "processor"
	TypeRateLimit Type = "rate_limit"
	TypeTracer    Type = "tracer"
)

Component types.

func Types added in v3.43.0 

func Types() []Type

Types returns a slice containing all component types.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.