docs

package
v4.28.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: May 30, 2024 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package docs provides useful functions for creating documentation from Benthos components

Index

Constants

This section is empty.

Variables

View Source 
var (

	// ErrBadLabel is returned when creating a component with a bad label.
	ErrBadLabel = fmt.Errorf("should match the regular expression /%v/ and must not start with an underscore", labelExpression)
)

Functions

func BloblangFunctionsMarkdown 

func BloblangFunctionsMarkdown() ([]byte, error)

BloblangFunctionsMarkdown returns a markdown document for all Bloblang functions.

func BloblangMethodsMarkdown 

func BloblangMethodsMarkdown() ([]byte, error)

BloblangMethodsMarkdown returns a markdown document for all Bloblang methods.

func ComponentFieldsFromConf 

func ComponentFieldsFromConf(conf any) (inferred map[string]FieldSpecs)

ComponentFieldsFromConf walks the children of a YAML node and returns a list of fields extracted from it. This can be used in order to infer a field spec for a parsed component.

TODO: V5 Remove this eventually.

func FieldsTemplate 

func FieldsTemplate(lintableExamples bool) string

FieldsTemplate returns a Go template for rendering markdown field documentation. The context should have a field `.Fields` of the type `[]FieldSpecCtx`.

func GetPluginConfigYAML 

func GetPluginConfigYAML(name string, node *yaml.Node) (yaml.Node, error)

GetPluginConfigYAML extracts a plugin configuration node from a component config. This exists because there are two styles of plugin config.

func GetYAMLPath 

func GetYAMLPath(root *yaml.Node, path ...string) (*yaml.Node, error)

GetYAMLPath attempts to obtain a specific value within a YAML tree by following a sequence of path identifiers.

func MarshalYAML 

func MarshalYAML(v yaml.Node) ([]byte, error)

MarshalYAML marshals a structure into YAML with consistent formatting across all Benthos components.

func ReservedFieldsByType 

func ReservedFieldsByType(t Type) map[string]FieldSpec

ReservedFieldsByType returns a map of fields for a specific type.

func SanitiseYAML 

func SanitiseYAML(cType Type, node *yaml.Node, conf SanitiseConfig) error

SanitiseYAML takes a yaml.Node and a config spec and sorts the fields of the node according to the spec. Also optionally removes the `type` field from this and all nested components.

func UnmarshalYAML 

func UnmarshalYAML(rawBytes []byte) (*yaml.Node, error)

UnmarshalYAML attempts to parse a byte slice as a YAML document and returns the root of the underlying document contents.

func ValidateLabel 

func ValidateLabel(label string) error

ValidateLabel attempts to validate the contents of a component label.

Types

type AnnotatedExample 

type AnnotatedExample struct {
	// A title for the example.
	Title string `json:"title"`

	// Summary of the example.
	Summary string `json:"summary"`

	// A config snippet to show.
	Config string `json:"config"`
}

AnnotatedExample is an isolated example for a component.

type ComponentSpec 

type ComponentSpec struct {
	// Name of the component
	Name string `json:"name"`

	// Type of the component (input, output, etc)
	Type Type `json:"type"`

	// The status of the component.
	Status Status `json:"status"`

	// Plugin is true for all plugin components.
	Plugin bool `json:"plugin"`

	// Summary of the component (in Asciidoc, must be short).
	Summary string `json:"summary,omitempty"`

	// Description of the component (in Asciidoc).
	Description string `json:"description,omitempty"`

	// Categories that describe the purpose of the component.
	Categories []string `json:"categories"`

	// Footnotes of the component (in Asciidoc).
	Footnotes string `json:"footnotes,omitempty"`

	// Examples demonstrating use cases for the component.
	Examples []AnnotatedExample `json:"examples,omitempty"`

	// A summary of each field in the component configuration.
	Config FieldSpec `json:"config"`

	// Version is the Benthos version this component was introduced.
	Version string `json:"version,omitempty"`
}

ComponentSpec describes a Benthos component.

func GetInferenceCandidateFromMap 

func GetInferenceCandidateFromMap(docProv Provider, t Type, m map[string]any) (string, ComponentSpec, error)

GetInferenceCandidateFromMap checks a map config structure for a component and returns either the inferred type name or an error if one cannot be inferred.

func GetInferenceCandidateFromYAML 

func GetInferenceCandidateFromYAML(docProv Provider, t Type, node *yaml.Node) (string, ComponentSpec, error)

GetInferenceCandidateFromYAML checks a yaml node config structure for a component and returns either the inferred type name or an error if one cannot be inferred.

func (*ComponentSpec) AsMarkdown 

func (c *ComponentSpec) AsMarkdown(prov Provider, nest bool, fullConfigExample any) ([]byte, error)

AsMarkdown renders the spec of a component, along with a full configuration example, into a markdown document.

type ComponentWalkYAMLFunc 

type ComponentWalkYAMLFunc func(c WalkedYAMLComponent) error

ComponentWalkYAMLFunc is called for each component type within a YAML config, where the node representing that component is provided along with the type and implementation name.

type FieldFilter 

type FieldFilter func(spec FieldSpec, v any) bool

FieldFilter defines a filter closure that returns a boolean for a component field indicating whether the field should be kept within a generated config.

func ShouldDropDeprecated 

func ShouldDropDeprecated(b bool) FieldFilter

ShouldDropDeprecated returns a field filter that removes all deprecated fields when the boolean argument is true.

type FieldKind 

type FieldKind string

FieldKind represents a field kind. 

var (
	KindScalar  FieldKind = "scalar"
	KindArray   FieldKind = "array"
	Kind2DArray FieldKind = "2darray"
	KindMap     FieldKind = "map"
)

ValueType variants.

type FieldSpec 

type FieldSpec struct {
	// Name of the field (as it appears in config).
	Name string `json:"name"`

	// Type of the field.
	Type FieldType `json:"type"`

	// Kind of the field.
	Kind FieldKind `json:"kind"`

	// Description of the field purpose (in Asciidoc).
	Description string `json:"description,omitempty"`

	// IsAdvanced is true for optional fields that will not be present in most
	// configs.
	IsAdvanced bool `json:"is_advanced,omitempty"`

	// IsDeprecated is true for fields that are deprecated and only exist
	// for backwards compatibility reasons.
	IsDeprecated bool `json:"is_deprecated,omitempty"`

	// IsOptional is a boolean flag indicating that a field is optional, even
	// if there is no default. This prevents linting errors when the field
	// is missing.
	IsOptional bool `json:"is_optional,omitempty"`

	// IsSecret indicates whether the field represents information that is
	// generally considered sensitive such as passwords or access tokens.
	IsSecret bool `json:"is_secret,omitempty"`

	// Default value of the field.
	Default *any `json:"default,omitempty"`

	// Interpolation indicates that the field supports interpolation
	// functions.
	Interpolated bool `json:"interpolated,omitempty"`

	// Bloblang indicates that a string field is a Bloblang mapping.
	Bloblang bool `json:"bloblang,omitempty"`

	// Examples is a slice of optional example values for a field.
	Examples []any `json:"examples,omitempty"`

	// AnnotatedOptions for this field. Each option should have a summary.
	AnnotatedOptions [][2]string `json:"annotated_options,omitempty"`

	// Options for this field.
	Options []string `json:"options,omitempty"`

	// Children fields of this field (it must be an object).
	Children FieldSpecs `json:"children,omitempty"`

	// Version is an explicit version when this field was introduced.
	Version string `json:"version,omitempty"`

	// Linter is an optional bloblang mapping that should be used in order to
	// lint a field.
	Linter string `json:"linter,omitempty"`

	// Scrubber is an optional bloblang mapping that should be used in order to
	// scrub sensitive information from field values when echoed.
	Scrubber string `json:"scrubber,omitempty"`
	// contains filtered or unexported fields
}

FieldSpec describes a component config field.

func FieldAnything 

func FieldAnything(name, description string, examples ...any) FieldSpec

FieldAnything returns a field spec for any typed field.

func FieldBloblang 

func FieldBloblang(name, description string, examples ...any) FieldSpec

FieldBloblang returns a field spec for a string typed field containing a Bloblang mapping.

func FieldBool 

func FieldBool(name, description string, examples ...any) FieldSpec

FieldBool returns a field spec for a common bool typed field.

func FieldBuffer 

func FieldBuffer(name, description string, examples ...any) FieldSpec

FieldBuffer returns a field spec for a buffer typed field.

func FieldCache 

func FieldCache(name, description string, examples ...any) FieldSpec

FieldCache returns a field spec for a cache typed field.

func FieldComponent 

func FieldComponent() FieldSpec

FieldComponent returns a field spec for a component.

func FieldFloat 

func FieldFloat(name, description string, examples ...any) FieldSpec

FieldFloat returns a field spec for a common float typed field.

func FieldFromYAML 

func FieldFromYAML(name string, node *yaml.Node) FieldSpec

FieldFromYAML infers a field spec from a YAML node. This mechanism has many limitations and should only be used for pre-hydrating field specs for old components with struct based config.

func FieldInput 

func FieldInput(name, description string, examples ...any) FieldSpec

FieldInput returns a field spec for an input typed field.

func FieldInt 

func FieldInt(name, description string, examples ...any) FieldSpec

FieldInt returns a field spec for a common int typed field.

func FieldInterpolatedString 

func FieldInterpolatedString(name, description string, examples ...any) FieldSpec

FieldInterpolatedString returns a field spec for a string typed field supporting dynamic interpolated functions.

func FieldMetrics 

func FieldMetrics(name, description string, examples ...any) FieldSpec

FieldMetrics returns a field spec for a metrics typed field.

func FieldObject 

func FieldObject(name, description string, examples ...any) FieldSpec

FieldObject returns a field spec for an object typed field.

func FieldOutput 

func FieldOutput(name, description string, examples ...any) FieldSpec

FieldOutput returns a field spec for an output typed field.

func FieldProcessor 

func FieldProcessor(name, description string, examples ...any) FieldSpec

FieldProcessor returns a field spec for a processor typed field.

func FieldRateLimit 

func FieldRateLimit(name, description string, examples ...any) FieldSpec

FieldRateLimit returns a field spec for a rate limit typed field.

func FieldScanner 

func FieldScanner(name, description string, examples ...any) FieldSpec

FieldScanner returns a field spec for a scanner typed field.

func FieldString 

func FieldString(name, description string, examples ...any) FieldSpec

FieldString returns a field spec for a common string typed field.

func FieldTracer 

func FieldTracer(name, description string, examples ...any) FieldSpec

FieldTracer returns a field spec for a tracer typed field.

func FieldURL 

func FieldURL(name, description string, examples ...any) FieldSpec

FieldURL returns a field spec for a string typed field containing a URL, both linting rules and scrubbers are added.

func MetricsMappingFieldSpec 

func MetricsMappingFieldSpec(name string) FieldSpec

MetricsMappingFieldSpec is a field spec that describes a Bloblang mapping for renaming metrics.

func (FieldSpec) Advanced 

func (f FieldSpec) Advanced() FieldSpec

Advanced marks this field as being advanced, and therefore not commonly used.

func (FieldSpec) AnyToValue 

func (f FieldSpec) AnyToValue(v any, conf ToValueConfig) (any, error)

AnyToValue converts any value into a generic value containing fleshed out defaults by referencing the expected type.

func (FieldSpec) Array 

func (f FieldSpec) Array() FieldSpec

Array determines that this field is an array of the field type.

func (FieldSpec) ArrayOfArrays 

func (f FieldSpec) ArrayOfArrays() FieldSpec

ArrayOfArrays determines that this is an array of arrays of the field type.

func (FieldSpec) AtVersion 

func (f FieldSpec) AtVersion(v string) FieldSpec

AtVersion specifies the version at which this fields behaviour was last modified.

func (FieldSpec) CheckRequired 

func (f FieldSpec) CheckRequired() bool

CheckRequired returns true if this field, due to various factors, is a field that must be specified within a config. The factors at play are:

- Whether the field has a default value - Whether the field was explicitly marked as optional - Whether the field is an object with children, none of which are required.

func (FieldSpec) ChildDefaultAndTypesFromStruct 

func (f FieldSpec) ChildDefaultAndTypesFromStruct(conf any) FieldSpec

ChildDefaultAndTypesFromStruct enriches a field specs children with a type string and default value from another field spec inferred from a config struct.

TODO: V5 Remove this eventually.

func (FieldSpec) DefaultAndTypeFrom 

func (f FieldSpec) DefaultAndTypeFrom(from FieldSpec) FieldSpec

DefaultAndTypeFrom enriches a field spec with a type string and default value from another field spec.

func (FieldSpec) Deprecated 

func (f FieldSpec) Deprecated() FieldSpec

Deprecated marks this field as being deprecated.

func (FieldSpec) FlattenChildrenForDocs 

func (f FieldSpec) FlattenChildrenForDocs() []FieldSpecCtx

FlattenChildrenForDocs converts the children of a field into a flat list, where the names contain hints as to their position in a structured hierarchy. This makes it easier to list the fields in documentation.

func (FieldSpec) GetDocsForPath 

func (f FieldSpec) GetDocsForPath(docsProvider Provider, path ...string) (FieldSpec, error)

GetDocsForPath attempts to find the documentation for a given node of a config identified by a path.

func (FieldSpec) GetLintFunc 

func (f FieldSpec) GetLintFunc() LintFunc

func (FieldSpec) HasAnnotatedOptions 

func (f FieldSpec) HasAnnotatedOptions(options ...string) FieldSpec

HasAnnotatedOptions returns a new FieldSpec that specifies a specific list of annotated options. Field values are linted to ensure they match one of the given options by a case insensitive match, use a custom lint function in order to change this default behaviour.

func (FieldSpec) HasDefault 

func (f FieldSpec) HasDefault(v any) FieldSpec

HasDefault returns a new FieldSpec that specifies a default value.

func (FieldSpec) HasOptions 

func (f FieldSpec) HasOptions(options ...string) FieldSpec

HasOptions returns a new FieldSpec that specifies a specific list of options. Field values are linted to ensure they match one of the given options by a case insensitive match, use a custom lint function in order to change this default behaviour.

func (FieldSpec) HasType 

func (f FieldSpec) HasType(t FieldType) FieldSpec

HasType returns a new FieldSpec that specifies a specific type.

func (FieldSpec) IsBloblang 

func (f FieldSpec) IsBloblang() FieldSpec

IsBloblang indicates that the field is a Bloblang mapping.

func (FieldSpec) IsInterpolated 

func (f FieldSpec) IsInterpolated() FieldSpec

IsInterpolated indicates that the field supports interpolation functions.

func (FieldSpec) JSONSchema 

func (f FieldSpec) JSONSchema() any

JSONSchema serializes a field spec into a JSON schema structure.

func (FieldSpec) LintYAML 

func (f FieldSpec) LintYAML(ctx LintContext, node *yaml.Node) []Lint

LintYAML returns a list of linting errors found by checking a field definition against a yaml node.

func (FieldSpec) LinterBlobl 

func (f FieldSpec) LinterBlobl(blobl string) FieldSpec

LinterBlobl adds a linting function to a field. When linting is performed on a config the provided bloblang mapping will be called with a boxed variant of the field value, allowing it to perform linting on that value, where an array of lints (strings) should be returned.

It is important to note that for fields defined as a non-scalar (array, array of arrays, map, etc) the linting rule will be executed on the highest level (array) and also the individual scalar values. If your field is a high level type then make sure your linting rule checks the type of the value provided in order to limit when the linting is performed.

Note that a linting rule defined this way will only be effective in the binary that defines it as the function cannot be serialized into a portable schema.

func (FieldSpec) LinterFunc 

func (f FieldSpec) LinterFunc(fn LintFunc) FieldSpec

LinterFunc adds a linting function to a field. When linting is performed on a config the provided function will be called with a boxed variant of the field value, allowing it to perform linting on that value.

It is important to note that for fields defined as a non-scalar (array, array of arrays, map, etc) the linting rule will be executed on the highest level (array) and also the individual scalar values. If your field is a high level type then make sure your linting rule checks the type of the value provided in order to limit when the linting is performed.

Note that a linting rule defined this way will only be effective in the binary that defines it as the function cannot be serialized into a portable schema.

func (FieldSpec) Map 

func (f FieldSpec) Map() FieldSpec

Map determines that this field is a map of arbitrary keys to a field type.

func (FieldSpec) OmitWhen 

func (f FieldSpec) OmitWhen(fn func(field, parent any) (why string, shouldOmit bool)) FieldSpec

OmitWhen specifies a custom func that, when provided a generic config struct, returns a boolean indicating when the field can be safely omitted from a config.

func (FieldSpec) Optional 

func (f FieldSpec) Optional() FieldSpec

Optional marks this field as being optional, and therefore its absence in a config is not considered an error even when a default value is not provided.

func (FieldSpec) ParsedConfigFromAny 

func (f FieldSpec) ParsedConfigFromAny(v any) (pConf *ParsedConfig, err error)

func (FieldSpec) SanitiseYAML 

func (f FieldSpec) SanitiseYAML(node *yaml.Node, conf SanitiseConfig) error

SanitiseYAML attempts to reduce a parsed config (as a *yaml.Node) down into a minimal representation without changing the behaviour of the config. The fields of the result will also be sorted according to the field spec.

func (FieldSpec) Scalar 

func (f FieldSpec) Scalar() FieldSpec

Scalar determines that this field is a scalar type (the default).

func (FieldSpec) ScrubValue 

func (f FieldSpec) ScrubValue(v any) (any, error)

func (FieldSpec) Secret 

func (f FieldSpec) Secret() FieldSpec

Secret marks this field as being a secret, which means it represents information that is generally considered sensitive such as passwords or access tokens.

func (FieldSpec) SetYAMLPath 

func (f FieldSpec) SetYAMLPath(docsProvider Provider, root, value *yaml.Node, path ...string) error

SetYAMLPath sets the value of a node within a YAML document identified by a path to a value.

func (FieldSpec) ToYAML 

func (f FieldSpec) ToYAML(recurse bool) (*yaml.Node, error)

ToYAML creates a YAML node from a field spec. If a default value has been specified then it is used. Otherwise, a zero value is generated. If recurse is enabled and the field has children then all children will also have values generated.

func (FieldSpec) WalkYAML 

func (f FieldSpec) WalkYAML(node *yaml.Node, prov Provider, fn ComponentWalkYAMLFunc) error

WalkYAML walks each node of a YAML tree and for any component types within the config a provided func is called.

func (FieldSpec) WithChildren 

func (f FieldSpec) WithChildren(children ...FieldSpec) FieldSpec

WithChildren returns a new FieldSpec that has child fields.

func (FieldSpec) YAMLLabelsToPaths 

func (f FieldSpec) YAMLLabelsToPaths(docsProvider Provider, node *yaml.Node, labelsToPaths map[string][]string, path []string)

YAMLLabelsToPaths walks a YAML tree using a field spec as a reference point. When a component of the YAML tree has a label field it is added to the provided labelsToPaths map with the path to the component.

func (FieldSpec) YAMLToValue 

func (f FieldSpec) YAMLToValue(node *yaml.Node, conf ToValueConfig) (any, error)

YAMLToValue converts a yaml node into a generic value by referencing the expected type.

type FieldSpecCtx 

type FieldSpecCtx struct {
	Spec FieldSpec

	// FullName describes the full dot path name of the field relative to
	// the root of the documented component.
	FullName string

	// ExamplesMarshalled is a list of examples marshalled into YAML format.
	ExamplesMarshalled []string

	// DefaultMarshalled is a marshalled string of the default value, if there is one.
	DefaultMarshalled string
}

FieldSpecCtx provides a field spec and rendered extras for documentation templates to use.

type FieldSpecs 

type FieldSpecs []FieldSpec

FieldSpecs is a slice of field specs for a component.

func FieldsFromConf 

func FieldsFromConf(conf any) FieldSpecs

FieldsFromConf attempts to infer field documents from a config struct.

func FieldsFromYAML 

func FieldsFromYAML(node *yaml.Node) FieldSpecs

FieldsFromYAML walks the children of a YAML node and returns a list of fields extracted from it. This can be used in order to infer a field spec for a parsed component.

func (FieldSpecs) Add 

func (f FieldSpecs) Add(specs ...FieldSpec) FieldSpecs

Add more field specs.

func (FieldSpecs) AnyToMap 

func (f FieldSpecs) AnyToMap(v any, conf ToValueConfig) (map[string]any, error)

AnyToMap converts a raw map value node into a generic map structure referencing expected fields, adding default values to the map when the node does not contain them.

func (FieldSpecs) DefaultAndTypeFrom 

func (f FieldSpecs) DefaultAndTypeFrom(from FieldSpecs) FieldSpecs

DefaultAndTypeFrom enriches a field spec with a type string and default value from another field spec.

func (FieldSpecs) GetDocsForPath 

func (f FieldSpecs) GetDocsForPath(docsProvider Provider, path ...string) (FieldSpec, error)

GetDocsForPath attempts to find the documentation for a given node of a config identified by a path.

func (FieldSpecs) JSONSchema 

func (f FieldSpecs) JSONSchema() map[string]any

JSONSchema serializes a field spec into a JSON schema structure.

func (FieldSpecs) LintYAML 

func (f FieldSpecs) LintYAML(ctx LintContext, node *yaml.Node) []Lint

LintYAML walks a yaml node and returns a list of linting errors found.

func (FieldSpecs) Merge 

func (f FieldSpecs) Merge(specs FieldSpecs) FieldSpecs

Merge with another set of FieldSpecs.

func (FieldSpecs) ParsedConfigFromAny 

func (f FieldSpecs) ParsedConfigFromAny(v any) (pConf *ParsedConfig, err error)

func (FieldSpecs) SanitiseYAML 

func (f FieldSpecs) SanitiseYAML(node *yaml.Node, conf SanitiseConfig) error

SanitiseYAML attempts to reduce a parsed config (as a *yaml.Node) down into a minimal representation without changing the behaviour of the config. The fields of the result will also be sorted according to the field spec.

func (FieldSpecs) SetDefault 

func (f FieldSpecs) SetDefault(v any, path ...string)

SetDefault attempts to override the current default value of a field, identified by a series of names used to walk the config spec in order to reach the intended field. This function currently does NOT support walking through arrays.

func (FieldSpecs) SetYAMLPath 

func (f FieldSpecs) SetYAMLPath(docsProvider Provider, root, value *yaml.Node, path ...string) error

SetYAMLPath sets the value of a node within a YAML document identified by a path to a value.

func (FieldSpecs) ToYAML 

func (f FieldSpecs) ToYAML() (*yaml.Node, error)

ToYAML creates a YAML node from a list of field specs. If a default value has been specified for a given field then it is used. Otherwise, a zero value is generated.

func (FieldSpecs) WalkYAML 

func (f FieldSpecs) WalkYAML(node *yaml.Node, prov Provider, fn ComponentWalkYAMLFunc) error

WalkYAML walks each node of a YAML tree and for any component types within the config a provided func is called.

func (FieldSpecs) YAMLLabelsToPaths 

func (f FieldSpecs) YAMLLabelsToPaths(docsProvider Provider, node *yaml.Node, labelsToPaths map[string][]string, path []string)

YAMLLabelsToPaths walks a YAML tree using a field spec as a reference point. When a component of the YAML tree has a label field it is added to the provided labelsToPaths map with the path to the component.

func (FieldSpecs) YAMLToMap 

func (f FieldSpecs) YAMLToMap(node *yaml.Node, conf ToValueConfig) (map[string]any, error)

YAMLToMap converts a yaml node into a generic map structure by referencing expected fields, adding default values to the map when the node does not contain them.

type FieldType 

type FieldType string

FieldType represents a field type. 

var (
	FieldTypeString  FieldType = "string"
	FieldTypeInt     FieldType = "int"
	FieldTypeFloat   FieldType = "float"
	FieldTypeBool    FieldType = "bool"
	FieldTypeObject  FieldType = "object"
	FieldTypeUnknown FieldType = "unknown"

	// Core component types, only components that can be a child of another
	// component config are listed here.
	FieldTypeInput     FieldType = "input"
	FieldTypeBuffer    FieldType = "buffer"
	FieldTypeCache     FieldType = "cache"
	FieldTypeProcessor FieldType = "processor"
	FieldTypeRateLimit FieldType = "rate_limit"
	FieldTypeOutput    FieldType = "output"
	FieldTypeMetrics   FieldType = "metrics"
	FieldTypeTracer    FieldType = "tracer"
	FieldTypeScanner   FieldType = "scanner"
)

ValueType variants.

func (FieldType) IsCoreComponent 

func (t FieldType) IsCoreComponent() (Type, bool)

IsCoreComponent returns the core component type of a field if applicable.

type Lint 

type Lint struct {
	Line   int
	Column int // Optional, set to 1 by default
	Level  LintLevel
	Type   LintType
	What   string
}

Lint describes a single linting issue found with a Benthos config.

func LintBloblangField 

func LintBloblangField(ctx LintContext, line, col int, v any) []Lint

LintBloblangField is function for linting a config field expected to be an interpolation string.

func LintBloblangMapping 

func LintBloblangMapping(ctx LintContext, line, col int, v any) []Lint

LintBloblangMapping is function for linting a config field expected to be a bloblang mapping.

func LintYAML 

func LintYAML(ctx LintContext, cType Type, node *yaml.Node) []Lint

LintYAML takes a yaml.Node and a config spec and returns a list of linting errors found in the config.

func NewLintError 

func NewLintError(line int, t LintType, err error) Lint

NewLintError returns an error lint.

func NewLintWarning 

func NewLintWarning(line int, t LintType, msg string) Lint

NewLintWarning returns a warning lint.

func (Lint) Error 

func (l Lint) Error() string

Error returns a formatted string explaining the lint error prefixed with its location within the file.

type LintConfig 

type LintConfig struct {
	// Provides documentation for component implementations.
	DocsProvider Provider

	// Provides an isolated context for Bloblang parsing.
	BloblangEnv *bloblang.Environment

	// Reject any deprecated components or fields as linting errors.
	RejectDeprecated bool

	// Require labels for components.
	RequireLabels bool
}

LintConfig describes which rules apply when linting benthos configs, and also determines which component and bloblang environments are used.

func NewLintConfig 

func NewLintConfig(prov Provider) LintConfig

NewLintConfig creates a default linting config.

type LintContext 

type LintContext struct {
	// contains filtered or unexported fields
}

LintContext is provided to linting functions, and provides context about the wider configuration.

func NewLintContext 

func NewLintContext(conf LintConfig) LintContext

NewLintContext creates a new linting context.

type LintFunc 

type LintFunc func(ctx LintContext, line, col int, value any) []Lint

LintFunc is a common linting function for field values.

type LintLevel 

type LintLevel int

LintLevel describes the severity level of a linting error. 

const (
	LintError   LintLevel = iota
	LintWarning LintLevel = iota
)

Lint levels.

type LintType 

type LintType int

LintType is a discrete linting type. 

const (
	// LintCustom means a custom linting rule failed.
	LintCustom LintType = iota

	// LintFailedRead means a configuration could not be read.
	LintFailedRead LintType = iota

	// LintMissingEnvVar means a configuration contained an environment variable
	// interpolation without a default and the variable was undefined.
	LintMissingEnvVar LintType = iota

	// LintInvalidOption means the field value was not one of the explicit list
	// of options.
	LintInvalidOption LintType = iota

	// LintBadLabel means the label contains invalid characters.
	LintBadLabel LintType = iota

	// LintMissingLabel means the label is missing when required.
	LintMissingLabel LintType = iota

	// LintDuplicateLabel means the label collides with another label.
	LintDuplicateLabel LintType = iota

	// LintBadBloblang means the field contains invalid Bloblang.
	LintBadBloblang LintType = iota

	// LintShouldOmit means the field should be omitted.
	LintShouldOmit LintType = iota

	// LintComponentMissing means a component value was expected but the type is
	// missing.
	LintComponentMissing LintType = iota

	// LintComponentNotFound means the specified component value is not
	// recognised.
	LintComponentNotFound LintType = iota

	// LintUnknown means the field is unknown.
	LintUnknown LintType = iota

	// LintMissing means a field was required but missing.
	LintMissing LintType = iota

	// LintExpectedArray means an array value was expected but something else
	// was provided.
	LintExpectedArray LintType = iota

	// LintExpectedObject means an object value was expected but something else
	// was provided.
	LintExpectedObject LintType = iota

	// LintExpectedScalar means a scalar value was expected but something else
	// was provided.
	LintExpectedScalar LintType = iota

	// LintDeprecated means a field is deprecated and should not be used.
	LintDeprecated LintType = iota
)

type MappedDocsProvider 

type MappedDocsProvider struct {
	// contains filtered or unexported fields
}

MappedDocsProvider stores component documentation in maps, protected by a mutex, allowing safe concurrent use.

func NewMappedDocsProvider 

func NewMappedDocsProvider() *MappedDocsProvider

NewMappedDocsProvider creates a new (empty) provider of component docs.

func (*MappedDocsProvider) Clone 

func (m *MappedDocsProvider) Clone() *MappedDocsProvider

Clone returns a copied version of the provider that can be modified independently.

func (*MappedDocsProvider) GetDocs 

func (m *MappedDocsProvider) GetDocs(name string, ctype Type) (ComponentSpec, bool)

GetDocs attempts to obtain component implementation docs.

func (*MappedDocsProvider) RegisterDocs 

func (m *MappedDocsProvider) RegisterDocs(spec ComponentSpec)

RegisterDocs adds the documentation of a component implementation.

type ParsedConfig 

type ParsedConfig struct {
	// contains filtered or unexported fields
}

ParsedConfig represents a plugin configuration that has been validated and parsed from a ConfigSpec, and allows plugin constructors to access configuration fields.

func (*ParsedConfig) Contains 

func (p *ParsedConfig) Contains(path ...string) bool

Contains checks whether the parsed config contains a given field identified by its name.

func (*ParsedConfig) Field 

func (p *ParsedConfig) Field(path ...string) (any, bool)

Field accesses a Field from the parsed config by its name and returns the value if the Field is found and a boolean indicating whether it was found. Nested fields can be accessed by specifying the series of Field names.

func (*ParsedConfig) FieldAny 

func (p *ParsedConfig) FieldAny(path ...string) (any, error)

FieldAny accesses a field from the parsed config by its name that can assume any value type. If the field is not found an error is returned.

func (*ParsedConfig) FieldAnyList 

func (p *ParsedConfig) FieldAnyList(path ...string) ([]*ParsedConfig, error)

FieldAnyList accesses a field that is a list of any value types from the parsed config by its name and returns the value as an array of *ParsedConfig types, where each one represents an object or value in the list. Returns an error if the field is not found, or is not a list of values.

func (*ParsedConfig) FieldAnyMap 

func (p *ParsedConfig) FieldAnyMap(path ...string) (map[string]*ParsedConfig, error)

FieldAnyMap accesses a field that is an object of arbitrary keys and any values from the parsed config by its name and returns a map of *ParsedConfig types, where each one represents an object or value in the map. Returns an error if the field is not found, or is not an object.

func (*ParsedConfig) FieldBool 

func (p *ParsedConfig) FieldBool(path ...string) (bool, error)

FieldBool accesses a bool field from the parsed config by its name and returns the value. Returns an error if the field is not found or is not a bool.

func (*ParsedConfig) FieldDuration 

func (p *ParsedConfig) FieldDuration(path ...string) (time.Duration, error)

FieldDuration accesses a duration string field from the parsed config by its name. If the field is not found or is not a valid duration string an error is returned.

func (*ParsedConfig) FieldFloat 

func (p *ParsedConfig) FieldFloat(path ...string) (float64, error)

FieldFloat accesses a float field from the parsed config by its name and returns the value. Returns an error if the field is not found or is not a float.

func (*ParsedConfig) FieldFloatList 

func (p *ParsedConfig) FieldFloatList(path ...string) ([]float64, error)

FieldFloatList accesses a field that is a list of floats from the parsed config by its name and returns the value. Returns an error if the field is not found, or is not a list of floats.

func (*ParsedConfig) FieldFloatMap 

func (p *ParsedConfig) FieldFloatMap(path ...string) (map[string]float64, error)

FieldFloatMap accesses a field that is an object of arbitrary keys and float values from the parsed config by its name and returns the value. Returns an error if the field is not found, or is not an object of floats.

func (*ParsedConfig) FieldInt 

func (p *ParsedConfig) FieldInt(path ...string) (int, error)

FieldInt accesses an int field from the parsed config by its name and returns the value. Returns an error if the field is not found or is not an int.

func (*ParsedConfig) FieldIntList 

func (p *ParsedConfig) FieldIntList(path ...string) ([]int, error)

FieldIntList accesses a field that is a list of integers from the parsed config by its name and returns the value. Returns an error if the field is not found, or is not a list of integers.

func (*ParsedConfig) FieldIntMap 

func (p *ParsedConfig) FieldIntMap(path ...string) (map[string]int, error)

FieldIntMap accesses a field that is an object of arbitrary keys and integer values from the parsed config by its name and returns the value. Returns an error if the field is not found, or is not an object of integers.

func (*ParsedConfig) FieldObjectList 

func (p *ParsedConfig) FieldObjectList(path ...string) ([]*ParsedConfig, error)

FieldObjectList accesses a field that is a list of objects from the parsed config by its name and returns the value as an array of *ParsedConfig types, where each one represents an object in the list. Returns an error if the field is not found, or is not a list of objects.

func (*ParsedConfig) FieldObjectListOfLists 

func (p *ParsedConfig) FieldObjectListOfLists(path ...string) ([][]*ParsedConfig, error)

FieldObjectListOfLists accesses a field that is a list of lists of objects from the parsed config by its name and returns the value as an array of arrays of *ParsedConfig types, where each one represents an object in the list. Returns an error if the field is not found, or is not a list of lists of objects.

func (*ParsedConfig) FieldObjectMap 

func (p *ParsedConfig) FieldObjectMap(path ...string) (map[string]*ParsedConfig, error)

FieldObjectMap accesses a field that is a map of objects from the parsed config by its name and returns the value as a map of *ParsedConfig types, where each one represents an object in the map. Returns an error if the field is not found, or is not a map of objects.

func (*ParsedConfig) FieldString 

func (p *ParsedConfig) FieldString(path ...string) (string, error)

FieldString accesses a string field from the parsed config by its name. If the field is not found or is not a string an error is returned.

func (*ParsedConfig) FieldStringList 

func (p *ParsedConfig) FieldStringList(path ...string) ([]string, error)

FieldStringList accesses a field that is a list of strings from the parsed config by its name and returns the value. Returns an error if the field is not found, or is not a list of strings.

func (*ParsedConfig) FieldStringListOfLists 

func (p *ParsedConfig) FieldStringListOfLists(path ...string) ([][]string, error)

FieldStringListOfLists accesses a field that is a list of lists of strings from the parsed config by its name and returns the value. Returns an error if the field is not found, or is not a list of lists of strings.

func (*ParsedConfig) FieldStringMap 

func (p *ParsedConfig) FieldStringMap(path ...string) (map[string]string, error)

FieldStringMap accesses a field that is an object of arbitrary keys and string values from the parsed config by its name and returns the value. Returns an error if the field is not found, or is not an object of strings.

func (*ParsedConfig) FullDotPath 

func (p *ParsedConfig) FullDotPath(path ...string) string

func (*ParsedConfig) Line 

func (p *ParsedConfig) Line() (int, bool)

func (*ParsedConfig) Namespace 

func (p *ParsedConfig) Namespace(path ...string) *ParsedConfig

Namespace returns a version of the parsed config at a given field namespace. This is useful for extracting multiple fields under the same grouping.

func (*ParsedConfig) Raw 

func (p *ParsedConfig) Raw() any

type Provider 

type Provider interface {
	GetDocs(name string, ctype Type) (ComponentSpec, bool)
}

Provider stores the component spec definitions of various component implementations.

type SanitiseConfig 

type SanitiseConfig struct {
	RemoveTypeField  bool
	RemoveDeprecated bool
	ScrubSecrets     bool
	ForExample       bool
	Filter           FieldFilter
	DocsProvider     Provider
}

SanitiseConfig contains fields describing the desired behaviour of the config sanitiser such as removing certain fields.

func NewSanitiseConfig 

func NewSanitiseConfig(prov Provider) SanitiseConfig

NewSanitiseConfig creates a new sanitise config.

type Status 

type Status string

Status of a component. 

var (
	StatusStable       Status = "stable"
	StatusBeta         Status = "beta"
	StatusExperimental Status = "experimental"
	StatusDeprecated   Status = "deprecated"
)

Component statuses.

type ToValueConfig 

type ToValueConfig struct {
	// Whether an problem in the config node detected during conversion
	// should return a "best attempt" structure rather than an error.
	Passive bool

	// When a field spec is for a non-scalar type (a component) fall back to
	// decoding it into an interface, otherwise the raw yaml.Node is
	// returned in its place.
	FallbackToAny bool
}

ToValueConfig describes custom options for how documentation fields should be used to convert a parsed node to a value type.

type Type 

type Type string

Type of a component. 

var (
	TypeBuffer    Type = "buffer"
	TypeCache     Type = "cache"
	TypeInput     Type = "input"
	TypeMetrics   Type = "metrics"
	TypeOutput    Type = "output"
	TypeProcessor Type = "processor"
	TypeRateLimit Type = "rate_limit"
	TypeTracer    Type = "tracer"
	TypeScanner   Type = "scanner"
)

Component types.

func Types 

func Types() []Type

Types returns a slice containing all component types.

type WalkedYAMLComponent 

type WalkedYAMLComponent struct {
	ComponentType Type
	Name          string
	Label         string
	Conf          *yaml.Node
}

WalkedYAMLComponent is a struct containing information about a component yielded via the WalkYAML method.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.