Documentation
¶
Overview ¶
Package lib contains a collection of packages used to create a Benthos stream pipeline. These packages can be used by third parties in order to easily orchestrate stream processing applications using Benthos as a foundation.
Please refer to the stream package for examples of how to create Benthos streams with your own custom processor implementations.
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
Package api implements a type used for creating the Benthos HTTP API.
|
Package api implements a type used for creating the Benthos HTTP API. |
|
Package broker implements types used for routing inputs to outputs in non-trivial arrangements, such as fan-out or fan-in models.
|
Package broker implements types used for routing inputs to outputs in non-trivial arrangements, such as fan-out or fan-in models. |
|
Package buffer is both a types.Consumer and types.Producer implementation that is able to sit between other stream components, effectively decoupling their transaction channels by storing messages in a buffer implementation.
|
Package buffer is both a types.Consumer and types.Producer implementation that is able to sit between other stream components, effectively decoupling their transaction channels by storing messages in a buffer implementation. |
|
parallel
Package parallel contains implementations of various buffer types where the buffer can be consumed by any number of parallel consumer threads.
|
Package parallel contains implementations of various buffer types where the buffer can be consumed by any number of parallel consumer threads. |
|
single
Package single contains implementations of various buffer types where the buffer can only be consumed by a single thread (but any number of writers).
|
Package single contains implementations of various buffer types where the buffer can only be consumed by a single thread (but any number of writers). |
|
Package cache implements the types.Cache interface for storing and retrieving key/value pairs from a range of storage strategies.
|
Package cache implements the types.Cache interface for storing and retrieving key/value pairs from a range of storage strategies. |
|
Package input defines consumers for aggregating data from a variety of sources.
|
Package input defines consumers for aggregating data from a variety of sources. |
|
reader
Package reader defines implementations of an interface for generic message reading from various third party sources.
|
Package reader defines implementations of an interface for generic message reading from various third party sources. |
|
Package log contains utilities for logging in a modular interface.
|
Package log contains utilities for logging in a modular interface. |
|
Package manager implements the types.Manager interface used for creating and sharing resources across a Benthos service.
|
Package manager implements the types.Manager interface used for creating and sharing resources across a Benthos service. |
|
Package message contains implementations of types.Message.
|
Package message contains implementations of types.Message. |
|
mapper
Package mapper implements ways of splicing and mapping batches of messages so that their derivative parts can be processed individually and merged back into the original batch.
|
Package mapper implements ways of splicing and mapping batches of messages so that their derivative parts can be processed individually and merged back into the original batch. |
|
metadata
Package metadata contains implementations of types.Metadata.
|
Package metadata contains implementations of types.Metadata. |
|
Package metrics contains a type for aggregating and propagating metrics to various services based on configuration.
|
Package metrics contains a type for aggregating and propagating metrics to various services based on configuration. |
|
Package output defines all sinks for sending Benthos messages to a variety of third party destinations.
|
Package output defines all sinks for sending Benthos messages to a variety of third party destinations. |
|
writer
Package writer defines implementations of an interface for generic message writing that outputs to various third party sinks.
|
Package writer defines implementations of an interface for generic message writing that outputs to various third party sinks. |
|
Package pipeline contains structures that implement both the Producer and Consumer interfaces.
|
Package pipeline contains structures that implement both the Producer and Consumer interfaces. |
|
Package processor contains implementations of types.Processor, which perform an arbitrary operation on a message and either returns >0 messages to be propagated towards a sink, or a response to be sent back to the message source.
|
Package processor contains implementations of types.Processor, which perform an arbitrary operation on a message and either returns >0 messages to be propagated towards a sink, or a response to be sent back to the message source. |
|
condition
Package condition contains logical operators that, based on their configuration, return boolean values from messages under certain circumstances.
|
Package condition contains logical operators that, based on their configuration, return boolean values from messages under certain circumstances. |
|
Package response contains implementations of types.Response.
|
Package response contains implementations of types.Response. |
|
Package stream creates and manages a full Benthos stream pipeline, consisting of an input layer of consumers, an optional buffer layer, a processing pipelines layer, and an output layer of producers: Inputs -> Buffer -> Processing Pipelines -> Outputs The number of parallel input consumers, processing pipelines, and output producers depends on the configuration of the stream.
|
Package stream creates and manages a full Benthos stream pipeline, consisting of an input layer of consumers, an optional buffer layer, a processing pipelines layer, and an output layer of producers: Inputs -> Buffer -> Processing Pipelines -> Outputs The number of parallel input consumers, processing pipelines, and output producers depends on the configuration of the stream. |
|
manager
Package manager creates and manages multiple streams, providing an API for performing CRUD operations.
|
Package manager creates and manages multiple streams, providing an API for performing CRUD operations. |
|
test
|
|
|
integration
Package integration implements integration tests using docker.
|
Package integration implements integration tests using docker. |
|
Package types defines any general structs and interfaces used throughout the benthos code base.
|
Package types defines any general structs and interfaces used throughout the benthos code base. |
|
config
Package config contains utilities for reading and parsing service configuration files.
|
Package config contains utilities for reading and parsing service configuration files. |
|
disk
Package disk contains cross platform disk statistics utilities.
|
Package disk contains cross platform disk statistics utilities. |
|
http/auth
Package auth provides configuration fields and implementations of HTTP request authentication strategies.
|
Package auth provides configuration fields and implementations of HTTP request authentication strategies. |
|
service
Package service contains utilities for bootstrapping a server such as config, logging and metrics.
|
Package service contains utilities for bootstrapping a server such as config, logging and metrics. |
|
text
Package text includes utilities for working with text that might contain variables.
|
Package text includes utilities for working with text that might contain variables. |
|
throttle
Package throttle implements throttle strategies.
|
Package throttle implements throttle strategies. |
|
tls
Package tls provides Benthos configuration fields and wrappers for a crypto/tls config.
|
Package tls provides Benthos configuration fields and wrappers for a crypto/tls config. |
Click to show internal directories.
Click to hide internal directories.