Documentation ¶
Overview ¶
Package writer defines implementations of an interface for generic message writing that outputs to various third party sinks.
Index ¶
- type AMQP
- type AMQPConfig
- type AMQPExchangeDeclareConfig
- type AmazonAWSCredentialsConfig
- type AmazonS3
- type AmazonS3Config
- type AmazonSQS
- type AmazonSQSConfig
- type Elasticsearch
- type ElasticsearchConfig
- type Files
- type FilesConfig
- type HDFS
- type HDFSConfig
- type HTTPClient
- type HTTPClientConfig
- type Kafka
- type KafkaConfig
- type Kinesis
- type KinesisConfig
- type MQTT
- type MQTTConfig
- type RedisList
- type RedisListConfig
- type RedisPubSub
- type RedisPubSubConfig
- type RedisStreams
- type RedisStreamsConfig
- type Type
- type Websocket
- type WebsocketConfig
- type ZMQ4Config
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type AMQP ¶ added in v0.15.1
type AMQP struct {
// contains filtered or unexported fields
}
AMQP is an output type that serves AMQP messages.
func (*AMQP) CloseAsync ¶ added in v0.15.1
func (a *AMQP) CloseAsync()
CloseAsync shuts down the AMQP output and stops processing messages.
func (*AMQP) WaitForClose ¶ added in v0.15.1
WaitForClose blocks until the AMQP output has closed down.
type AMQPConfig ¶ added in v0.15.1
type AMQPConfig struct { URL string `json:"url" yaml:"url"` Exchange string `json:"exchange" yaml:"exchange"` ExchangeDeclare AMQPExchangeDeclareConfig `json:"exchange_declare" yaml:"exchange_declare"` BindingKey string `json:"key" yaml:"key"` Persistent bool `json:"persistent" yaml:"persistent"` Mandatory bool `json:"mandatory" yaml:"mandatory"` Immediate bool `json:"immediate" yaml:"immediate"` TLS btls.Config `json:"tls" yaml:"tls"` }
AMQPConfig contains configuration fields for the AMQP output type.
func NewAMQPConfig ¶ added in v0.15.1
func NewAMQPConfig() AMQPConfig
NewAMQPConfig creates a new AMQPConfig with default values.
type AMQPExchangeDeclareConfig ¶ added in v0.24.0
type AMQPExchangeDeclareConfig struct { Enabled bool `json:"enabled" yaml:"enabled"` Type string `json:"type" yaml:"type"` Durable bool `json:"durable" yaml:"durable"` }
AMQPExchangeDeclareConfig contains fields indicating whether the target AMQP exchange needs to be declared, as well as any fields specifying how to accomplish that.
type AmazonAWSCredentialsConfig ¶ added in v0.7.5
type AmazonAWSCredentialsConfig struct { ID string `json:"id" yaml:"id"` Secret string `json:"secret" yaml:"secret"` Token string `json:"token" yaml:"token"` Role string `json:"role" yaml:"role"` }
AmazonAWSCredentialsConfig contains configuration params for AWS credentials.
type AmazonS3 ¶ added in v0.7.5
type AmazonS3 struct {
// contains filtered or unexported fields
}
AmazonS3 is a benthos writer.Type implementation that writes messages to an Amazon S3 bucket.
func NewAmazonS3 ¶ added in v0.7.5
NewAmazonS3 creates a new Amazon S3 bucket writer.Type.
func (*AmazonS3) CloseAsync ¶ added in v0.7.5
func (a *AmazonS3) CloseAsync()
CloseAsync begins cleaning up resources used by this reader asynchronously.
func (*AmazonS3) Connect ¶ added in v0.7.5
Connect attempts to establish a connection to the target S3 bucket.
func (*AmazonS3) WaitForClose ¶ added in v0.7.5
WaitForClose will block until either the reader is closed or a specified timeout occurs.
type AmazonS3Config ¶ added in v0.7.5
type AmazonS3Config struct { Region string `json:"region" yaml:"region"` Bucket string `json:"bucket" yaml:"bucket"` Path string `json:"path" yaml:"path"` Credentials AmazonAWSCredentialsConfig `json:"credentials" yaml:"credentials"` TimeoutS int64 `json:"timeout_s" yaml:"timeout_s"` }
AmazonS3Config contains configuration fields for the AmazonS3 output type.
func NewAmazonS3Config ¶ added in v0.7.5
func NewAmazonS3Config() AmazonS3Config
NewAmazonS3Config creates a new Config with default values.
type AmazonSQS ¶ added in v0.8.1
type AmazonSQS struct {
// contains filtered or unexported fields
}
AmazonSQS is a benthos writer.Type implementation that writes messages to an Amazon SQS queue.
func NewAmazonSQS ¶ added in v0.8.1
NewAmazonSQS creates a new Amazon SQS writer.Type.
func (*AmazonSQS) CloseAsync ¶ added in v0.8.1
func (a *AmazonSQS) CloseAsync()
CloseAsync begins cleaning up resources used by this reader asynchronously.
func (*AmazonSQS) Connect ¶ added in v0.8.1
Connect attempts to establish a connection to the target SQS queue.
func (*AmazonSQS) WaitForClose ¶ added in v0.8.1
WaitForClose will block until either the reader is closed or a specified timeout occurs.
type AmazonSQSConfig ¶ added in v0.8.1
type AmazonSQSConfig struct { Region string `json:"region" yaml:"region"` URL string `json:"url" yaml:"url"` Credentials AmazonAWSCredentialsConfig `json:"credentials" yaml:"credentials"` }
AmazonSQSConfig contains configuration fields for the output AmazonSQS type.
func NewAmazonSQSConfig ¶ added in v0.8.1
func NewAmazonSQSConfig() AmazonSQSConfig
NewAmazonSQSConfig creates a new Config with default values.
type Elasticsearch ¶ added in v0.10.7
type Elasticsearch struct {
// contains filtered or unexported fields
}
Elasticsearch is a writer type that writes messages into elasticsearch.
func NewElasticsearch ¶ added in v0.10.7
func NewElasticsearch(conf ElasticsearchConfig, log log.Modular, stats metrics.Type) (*Elasticsearch, error)
NewElasticsearch creates a new Elasticsearch writer type.
func (*Elasticsearch) CloseAsync ¶ added in v0.10.7
func (e *Elasticsearch) CloseAsync()
CloseAsync shuts down the Elasticsearch writer and stops processing messages.
func (*Elasticsearch) Connect ¶ added in v0.10.7
func (e *Elasticsearch) Connect() error
Connect attempts to establish a connection to a Elasticsearch broker.
func (*Elasticsearch) WaitForClose ¶ added in v0.10.7
func (e *Elasticsearch) WaitForClose(timeout time.Duration) error
WaitForClose blocks until the Elasticsearch writer has closed down.
type ElasticsearchConfig ¶ added in v0.10.7
type ElasticsearchConfig struct { URLs []string `json:"urls" yaml:"urls"` ID string `json:"id" yaml:"id"` Index string `json:"index" yaml:"index"` Type string `json:"type" yaml:"type"` TimeoutMS int `json:"timeout_ms" yaml:"timeout_ms"` Auth auth.BasicAuthConfig `json:"basic_auth" yaml:"basic_auth"` }
ElasticsearchConfig contains configuration fields for the Elasticsearch output type.
func NewElasticsearchConfig ¶ added in v0.10.7
func NewElasticsearchConfig() ElasticsearchConfig
NewElasticsearchConfig creates a new ElasticsearchConfig with default values.
type Files ¶ added in v0.7.7
type Files struct {
// contains filtered or unexported fields
}
Files is a benthos writer.Type implementation that writes message parts each to their own file.
func (*Files) CloseAsync ¶ added in v0.7.7
func (f *Files) CloseAsync()
CloseAsync begins cleaning up resources used by this reader asynchronously.
func (*Files) WaitForClose ¶ added in v0.7.7
WaitForClose will block until either the reader is closed or a specified timeout occurs.
type FilesConfig ¶ added in v0.7.7
type FilesConfig struct {
Path string `json:"path" yaml:"path"`
}
FilesConfig contains configuration fields for the files output type.
func NewFilesConfig ¶ added in v0.7.7
func NewFilesConfig() FilesConfig
NewFilesConfig creates a new Config with default values.
type HDFS ¶ added in v0.30.0
type HDFS struct {
// contains filtered or unexported fields
}
HDFS is a benthos writer.Type implementation that writes messages to a HDFS directory.
func (*HDFS) CloseAsync ¶ added in v0.30.0
func (h *HDFS) CloseAsync()
CloseAsync begins cleaning up resources used by this reader asynchronously.
func (*HDFS) Connect ¶ added in v0.30.0
Connect attempts to establish a connection to the target HDFS host.
func (*HDFS) WaitForClose ¶ added in v0.30.0
WaitForClose will block until either the reader is closed or a specified timeout occurs.
type HDFSConfig ¶ added in v0.30.0
type HDFSConfig struct { Hosts []string `json:"hosts" yaml:"hosts"` User string `json:"user" yaml:"user"` Directory string `json:"directory" yaml:"directory"` Path string `json:"path" yaml:"path"` }
HDFSConfig contains configuration fields for the HDFS output type.
func NewHDFSConfig ¶ added in v0.30.0
func NewHDFSConfig() HDFSConfig
NewHDFSConfig creates a new Config with default values.
type HTTPClient ¶ added in v0.14.0
type HTTPClient struct {
// contains filtered or unexported fields
}
HTTPClient is an output type that sends messages as HTTP requests to a target server endpoint.
func NewHTTPClient ¶ added in v0.14.0
func NewHTTPClient(conf HTTPClientConfig, log log.Modular, stats metrics.Type) (*HTTPClient, error)
NewHTTPClient creates a new HTTPClient writer type.
func (*HTTPClient) CloseAsync ¶ added in v0.14.0
func (h *HTTPClient) CloseAsync()
CloseAsync shuts down the HTTPClient output and stops processing messages.
func (*HTTPClient) Connect ¶ added in v0.14.0
func (h *HTTPClient) Connect() error
Connect does nothing.
func (*HTTPClient) WaitForClose ¶ added in v0.14.0
func (h *HTTPClient) WaitForClose(timeout time.Duration) error
WaitForClose blocks until the HTTPClient output has closed down.
type HTTPClientConfig ¶ added in v0.14.0
HTTPClientConfig contains configuration fields for the HTTPClient output type.
func NewHTTPClientConfig ¶ added in v0.14.0
func NewHTTPClientConfig() HTTPClientConfig
NewHTTPClientConfig creates a new HTTPClientConfig with default values.
type Kafka ¶
type Kafka struct {
// contains filtered or unexported fields
}
Kafka is a writer type that writes messages into kafka.
func (*Kafka) CloseAsync ¶
func (k *Kafka) CloseAsync()
CloseAsync shuts down the Kafka writer and stops processing messages.
func (*Kafka) WaitForClose ¶
WaitForClose blocks until the Kafka writer has closed down.
type KafkaConfig ¶
type KafkaConfig struct { Addresses []string `json:"addresses" yaml:"addresses"` ClientID string `json:"client_id" yaml:"client_id"` Key string `json:"key" yaml:"key"` RoundRobinPartitions bool `json:"round_robin_partitions" yaml:"round_robin_partitions"` Topic string `json:"topic" yaml:"topic"` Compression string `json:"compression" yaml:"compression"` MaxMsgBytes int `json:"max_msg_bytes" yaml:"max_msg_bytes"` TimeoutMS int `json:"timeout_ms" yaml:"timeout_ms"` AckReplicas bool `json:"ack_replicas" yaml:"ack_replicas"` TargetVersion string `json:"target_version" yaml:"target_version"` TLS btls.Config `json:"tls" yaml:"tls"` }
KafkaConfig contains configuration fields for the Kafka output type.
func NewKafkaConfig ¶
func NewKafkaConfig() KafkaConfig
NewKafkaConfig creates a new KafkaConfig with default values.
type Kinesis ¶ added in v0.26.0
type Kinesis struct {
// contains filtered or unexported fields
}
Kinesis is a benthos writer.Type implementation that writes messages to an Amazon Kinesis stream.
func NewKinesis ¶ added in v0.26.0
NewKinesis creates a new Amazon Kinesis writer.Type.
func (*Kinesis) CloseAsync ¶ added in v0.26.0
func (a *Kinesis) CloseAsync()
CloseAsync begins cleaning up resources used by this reader asynchronously.
func (*Kinesis) Connect ¶ added in v0.26.0
Connect creates a new Kinesis client and ensures that the target Kinesis stream exists.
func (*Kinesis) WaitForClose ¶ added in v0.26.0
WaitForClose will block until either the reader is closed or a specified timeout occurs.
type KinesisConfig ¶ added in v0.26.0
type KinesisConfig struct { Endpoint string `json:"endpoint" yaml:"endpoint"` Region string `json:"region" yaml:"region"` Stream string `json:"stream" yaml:"stream"` HashKey string `json:"hash_key" yaml:"hash_key"` PartitionKey string `json:"partition_key" yaml:"partition_key"` Credentials AmazonAWSCredentialsConfig `json:"credentials" yaml:"credentials"` retries.Config `json:",inline" yaml:",inline"` }
KinesisConfig contains configuration fields for the Kinesis output type.
func NewKinesisConfig ¶ added in v0.26.0
func NewKinesisConfig() KinesisConfig
NewKinesisConfig creates a new Config with default values.
type MQTT ¶ added in v0.8.7
type MQTT struct {
// contains filtered or unexported fields
}
MQTT is an output type that serves MQTT messages.
func (*MQTT) CloseAsync ¶ added in v0.8.7
func (m *MQTT) CloseAsync()
CloseAsync shuts down the MQTT output and stops processing messages.
func (*MQTT) WaitForClose ¶ added in v0.8.7
WaitForClose blocks until the MQTT output has closed down.
type MQTTConfig ¶ added in v0.8.7
type MQTTConfig struct { URLs []string `json:"urls" yaml:"urls"` QoS uint8 `json:"qos" yaml:"qos"` Topic string `json:"topic" yaml:"topic"` ClientID string `json:"client_id" yaml:"client_id"` }
MQTTConfig contains configuration fields for the MQTT output type.
func NewMQTTConfig ¶ added in v0.8.7
func NewMQTTConfig() MQTTConfig
NewMQTTConfig creates a new MQTTConfig with default values.
type RedisList ¶
type RedisList struct {
// contains filtered or unexported fields
}
RedisList is an output type that serves RedisList messages.
func NewRedisList ¶
NewRedisList creates a new RedisList output type.
func (*RedisList) CloseAsync ¶
func (r *RedisList) CloseAsync()
CloseAsync shuts down the RedisList output and stops processing messages.
func (*RedisList) WaitForClose ¶
WaitForClose blocks until the RedisList output has closed down.
type RedisListConfig ¶
type RedisListConfig struct { URL string `json:"url" yaml:"url"` Key string `json:"key" yaml:"key"` }
RedisListConfig contains configuration fields for the RedisList output type.
func NewRedisListConfig ¶
func NewRedisListConfig() RedisListConfig
NewRedisListConfig creates a new RedisListConfig with default values.
type RedisPubSub ¶ added in v0.26.1
type RedisPubSub struct {
// contains filtered or unexported fields
}
RedisPubSub is an output type that serves RedisPubSub messages.
func NewRedisPubSub ¶ added in v0.26.1
func NewRedisPubSub( conf RedisPubSubConfig, log log.Modular, stats metrics.Type, ) (*RedisPubSub, error)
NewRedisPubSub creates a new RedisPubSub output type.
func (*RedisPubSub) CloseAsync ¶ added in v0.26.1
func (r *RedisPubSub) CloseAsync()
CloseAsync shuts down the RedisPubSub output and stops processing messages.
func (*RedisPubSub) Connect ¶ added in v0.26.1
func (r *RedisPubSub) Connect() error
Connect establishes a connection to an RedisPubSub server.
func (*RedisPubSub) WaitForClose ¶ added in v0.26.1
func (r *RedisPubSub) WaitForClose(timeout time.Duration) error
WaitForClose blocks until the RedisPubSub output has closed down.
type RedisPubSubConfig ¶ added in v0.26.1
type RedisPubSubConfig struct { URL string `json:"url" yaml:"url"` Channel string `json:"channel" yaml:"channel"` }
RedisPubSubConfig contains configuration fields for the RedisPubSub output type.
func NewRedisPubSubConfig ¶ added in v0.26.1
func NewRedisPubSubConfig() RedisPubSubConfig
NewRedisPubSubConfig creates a new RedisPubSubConfig with default values.
type RedisStreams ¶ added in v0.26.1
type RedisStreams struct {
// contains filtered or unexported fields
}
RedisStreams is an output type that serves RedisStreams messages.
func NewRedisStreams ¶ added in v0.26.1
func NewRedisStreams( conf RedisStreamsConfig, log log.Modular, stats metrics.Type, ) (*RedisStreams, error)
NewRedisStreams creates a new RedisStreams output type.
func (*RedisStreams) CloseAsync ¶ added in v0.26.1
func (r *RedisStreams) CloseAsync()
CloseAsync shuts down the RedisStreams output and stops processing messages.
func (*RedisStreams) Connect ¶ added in v0.26.1
func (r *RedisStreams) Connect() error
Connect establishes a connection to an RedisStreams server.
func (*RedisStreams) WaitForClose ¶ added in v0.26.1
func (r *RedisStreams) WaitForClose(timeout time.Duration) error
WaitForClose blocks until the RedisStreams output has closed down.
type RedisStreamsConfig ¶ added in v0.26.1
type RedisStreamsConfig struct { URL string `json:"url" yaml:"url"` Stream string `json:"stream" yaml:"stream"` BodyKey string `json:"body_key" yaml:"body_key"` MaxLenApprox int64 `json:"max_length" yaml:"max_length"` }
RedisStreamsConfig contains configuration fields for the RedisStreams output type.
func NewRedisStreamsConfig ¶ added in v0.26.1
func NewRedisStreamsConfig() RedisStreamsConfig
NewRedisStreamsConfig creates a new RedisStreamsConfig with default values.
type Type ¶
type Type interface { // Connect attempts to establish a connection to the sink, if unsuccessful // returns an error. If the attempt is successful (or not necessary) returns // nil. Connect() error // Write should block until either the message is sent (and acknowledged) to // a sink, or a transport specific error has occurred, or the Type is // closed. Write(msg types.Message) error types.Closable }
Type is a type that writes Benthos messages to a third party sink. If the protocol supports a form of acknowledgement then it will be returned by the call to Write.
type Websocket ¶ added in v0.13.2
type Websocket struct {
// contains filtered or unexported fields
}
Websocket is an output type that serves Websocket messages.
func NewWebsocket ¶ added in v0.13.2
NewWebsocket creates a new Websocket output type.
func (*Websocket) CloseAsync ¶ added in v0.13.2
func (w *Websocket) CloseAsync()
CloseAsync shuts down the Websocket output and stops processing messages.
func (*Websocket) Connect ¶ added in v0.13.2
Connect establishes a connection to an Websocket server.
func (*Websocket) WaitForClose ¶ added in v0.13.2
WaitForClose blocks until the Websocket output has closed down.
type WebsocketConfig ¶ added in v0.13.2
type WebsocketConfig struct { URL string `json:"url" yaml:"url"` auth.Config `json:",inline" yaml:",inline"` }
WebsocketConfig contains configuration fields for the Websocket output type.
func NewWebsocketConfig ¶ added in v0.13.2
func NewWebsocketConfig() WebsocketConfig
NewWebsocketConfig creates a new WebsocketConfig with default values.
type ZMQ4Config ¶ added in v0.9.0
type ZMQ4Config struct{}
ZMQ4Config empty stub for when ZMQ4 is not compiled.