Documentation ¶
Overview ¶
Package processor contains logical message processors that can be pipelined within benthos using the pipeline.Processor type.
Index ¶
- func Descriptions() string
- type BlobToMulti
- type BoundsCheck
- type BoundsCheckConfig
- type Combine
- type CombineConfig
- type Config
- type MultiToBlob
- type Sample
- type SampleConfig
- type SelectParts
- type SelectPartsConfig
- type Type
- func New(conf Config, log log.Modular, stats metrics.Type) (Type, error)
- func NewBlobToMulti(conf Config, log log.Modular, stats metrics.Type) (Type, error)
- func NewBoundsCheck(conf Config, log log.Modular, stats metrics.Type) (Type, error)
- func NewCombine(conf Config, log log.Modular, stats metrics.Type) (Type, error)
- func NewMultiToBlob(conf Config, log log.Modular, stats metrics.Type) (Type, error)
- func NewSample(conf Config, log log.Modular, stats metrics.Type) (Type, error)
- func NewSelectParts(conf Config, log log.Modular, stats metrics.Type) (Type, error)
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Descriptions ¶
func Descriptions() string
Descriptions returns a formatted string of collated descriptions of each type.
Types ¶
type BlobToMulti ¶
type BlobToMulti struct {
// contains filtered or unexported fields
}
BlobToMulti is a processor that takes messages with a single part in a benthos multiple part blob format and decodes them into multiple part messages.
func (*BlobToMulti) ProcessMessage ¶
ProcessMessage takes a message with 1 part in multiple part blob format and returns a multiple part message by decoding it.
type BoundsCheck ¶
type BoundsCheck struct {
// contains filtered or unexported fields
}
BoundsCheck is a processor that checks each message against a set of bounds and rejects messages if they aren't within them.
func (*BoundsCheck) ProcessMessage ¶
ProcessMessage checks each message against a set of bounds.
type BoundsCheckConfig ¶
type BoundsCheckConfig struct { MaxParts int `json:"max_parts" yaml:"max_parts"` MinParts int `json:"min_parts" yaml:"min_parts"` MaxPartSize int `json:"max_part_size" yaml:"max_part_size"` }
BoundsCheckConfig contains any bounds configuration for the BoundsCheck processor.
func NewBoundsCheckConfig ¶
func NewBoundsCheckConfig() BoundsCheckConfig
NewBoundsCheckConfig returns a BoundsCheckConfig with default values.
type Combine ¶ added in v0.3.1
type Combine struct {
// contains filtered or unexported fields
}
Combine is a processor that takes messages with a single part in a benthos multiple part blob format and decodes them into multiple part messages.
func (*Combine) ProcessMessage ¶ added in v0.3.1
ProcessMessage takes a single message and buffers it, drops it, returning a NoAck response, until eventually it has N buffered messages, at which point it combines those messages into one multiple part message which is sent on.
type CombineConfig ¶ added in v0.3.1
type CombineConfig struct {
Parts int `json:"parts" yaml:"parts"`
}
CombineConfig contains configuration for the Combine processor.
func NewCombineConfig ¶ added in v0.3.1
func NewCombineConfig() CombineConfig
NewCombineConfig returns a CombineConfig with default values.
type Config ¶
type Config struct { Type string `json:"type" yaml:"type"` BoundsCheck BoundsCheckConfig `json:"bounds_check" yaml:"bounds_check"` SelectParts SelectPartsConfig `json:"select_parts" yaml:"select_parts"` BlobToMulti struct{} `json:"blob_to_multi" yaml:"blob_to_multi"` MultiToBlob struct{} `json:"multi_to_blob" yaml:"multi_to_blob"` Sample SampleConfig `json:"sample" yaml:"sample"` Combine CombineConfig `json:"combine" yaml:"combine"` }
Config is the all encompassing configuration struct for all processor types.
func NewConfig ¶
func NewConfig() Config
NewConfig returns a configuration struct fully populated with default values.
func (*Config) UnmarshalJSON ¶
UnmarshalJSON ensures that when parsing configs that are in a slice the default values are still applied.
func (*Config) UnmarshalYAML ¶
UnmarshalYAML ensures that when parsing configs that are in a slice the default values are still applied.
type MultiToBlob ¶
type MultiToBlob struct {
// contains filtered or unexported fields
}
MultiToBlob is a processor that takes messages with potentially multiple parts and converts them into a single part message using the benthos binary format. This message can be converted back to multiple parts using the BlobToMulti processor.
func (MultiToBlob) ProcessMessage ¶
ProcessMessage takes a message of > 0 parts and returns a single part message that can be later converted back to the original parts.
type Sample ¶ added in v0.3.2
type Sample struct {
// contains filtered or unexported fields
}
Sample is a processor that checks each message against a set of bounds and rejects messages if they aren't within them.
type SampleConfig ¶ added in v0.3.2
SampleConfig contains any configuration for the Sample processor.
func NewSampleConfig ¶ added in v0.3.2
func NewSampleConfig() SampleConfig
NewSampleConfig returns a SampleConfig with default values.
type SelectParts ¶ added in v0.4.1
type SelectParts struct {
// contains filtered or unexported fields
}
SelectParts is a processor that checks each message against a set of bounds and rejects messages if they aren't within them.
func (*SelectParts) ProcessMessage ¶ added in v0.4.1
ProcessMessage checks each message against a set of bounds.
type SelectPartsConfig ¶ added in v0.4.1
type SelectPartsConfig struct {
Parts []int `json:"parts" yaml:"parts"`
}
SelectPartsConfig contains any configuration for the SelectParts processor.
func NewSelectPartsConfig ¶ added in v0.4.1
func NewSelectPartsConfig() SelectPartsConfig
NewSelectPartsConfig returns a SelectPartsConfig with default values.
type Type ¶
type Type interface { // ProcessMessage attempts to process a message. Since processing can fail // this call returns both a message in case of success, a response in case // of failure, and a bool flag indicating (true == success) which of the two // should be used. ProcessMessage(msg *types.Message) (*types.Message, types.Response, bool) }
Type reads a message, performs a processing operation, and returns a message and a flag indicating whether that message should be propagated or not.
func NewBlobToMulti ¶
NewBlobToMulti returns a BlobToMulti processor.
func NewBoundsCheck ¶
NewBoundsCheck returns a BoundsCheck processor.
func NewCombine ¶ added in v0.3.1
NewCombine returns a Combine processor.
func NewMultiToBlob ¶
NewMultiToBlob returns a MultiToBlob processor.