Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func New ¶
func New(env *Options) destination.Destination
New returns a valid Blacksmith destination.Destination for a message broker.
Types ¶
type Driver ¶
type Driver string
Driver is a custom type allowing the user to only pass supported drivers when creating the destination.
var DriverAWSSNS Driver = "aws/sns"
DriverAWSSNS is used to leverage AWS SNS as the destination's driver.
Environment variables:
- AWS_ACCESS_KEY_ID (required)
- AWS_SECRET_ACCESS_KEY (required)
- AWS_REGION
var DriverAWSSQS Driver = "aws/sqs"
DriverAWSSQS is used to leverage AWS SQS as the destination's driver.
Environment variables:
- AWS_ACCESS_KEY_ID (required)
- AWS_SECRET_ACCESS_KEY (required)
- AWS_REGION
var DriverAzureServiceBus Driver = "azure/servicebus"
DriverAzureServiceBus is used to leverage Azure Service Bus as the destination's driver.
Environment variables:
- SERVICEBUS_CONNECTION_STRING (required)
var DriverGooglePubSub Driver = "google/pubsub"
DriverGooglePubSub is used to leverage Googe Pub / Sub as the destination's driver.
Environment variables:
- GOOGLE_APPLICATION_CREDENTIALS (required)
var DriverKafka Driver = "kafka"
DriverKafka is used to leverage Apache Kafka as the destination's driver.
Environment variables:
- KAFKA_BROKERS (required) Example: "127.0.0.1:9092,127.0.0.1:9093,127.0.0.1:9094"
var DriverNATS Driver = "nats"
DriverNATS is used to leverage NATS as the destination's driver.
Environment variables:
- NATS_SERVER_URL (required) Example: "nats://127.0.0.1:4222"
var DriverRabbitMQ Driver = "rabbitmq"
DriverRabbitMQ is used to leverage RabbitMQ as the destination's driver.
Environment variables:
- RABBIT_SERVER_URL (required) Example: "amqp://guest:guest@127.0.0.1:5672/"
type Message ¶
Message represents a message published by the source and received by the message broker.
type Options ¶
type Options struct {
// Realtime indicates if the pubsub adapter of the Blacksmith application shall
// be used to load events to the destination in realtime or not. When false, the
// Interval will be used.
Realtime bool
// Interval represents an interval or a CRON string at which a job shall be
// loaded to the destination. It is used as the time-lapse between retries in
// case of a job failure.
//
// Defaults to "@every 1h".
Interval string
// MaxRetries indicates the maximum number of retries per job the scheduler will
// attempt to execute before it succeed. When the limit is reached, the job is
// marked as "discarded".
//
// Defaults to 72.
MaxRetries uint16
// Name indicates the identifier of the topics to use in Blacksmith. The computed
// name is "topic(<name>)". This does not have any consequences on the store's
// name used in the cloud provider.
//
// Examples: "mytopic"
// Required.
Name string
// Driver is the driver to leverage for using this destination.
//
// Required.
Driver Driver
// Connection is the driver's specific connection string to use for publishing
// messages.
//
// Format for AWS SNS: "arn:aws:sns:<region>:<id>:<topic>"
// Format for AWS SQS: "arn:aws:sqs:<region>:<id>:<queue>"
// Format for Azure Service Bus: "<topic>"
// Format for Google Pub / Sub: "<project>/<topic>"
// Format for Apache Kafka: "<topic>"
// Format for NATS: "<subject>"
// Format for RabbitMQ: "<exchange>"
Connection string
// Params can be used to add specific configuration per driver.
//
// Supported fields for AWS SNS / SQS:
// url.Values{
// "region": {"<region>"}, // Required if environment variable 'AWS_REGION' is not set.
// }
Params url.Values
}
Options is the options the destination can take as an input to be configured.
type Publish ¶
type Publish struct {
Message Message `json:"message"`
// contains filtered or unexported fields
}
Publish implements the Blacksmith destination.Action interface for the action "publish". It holds the complete job's structure to load into the destination.
func (Publish) Load ¶
func (a Publish) Load(tk *destination.Toolkit, queue *store.Queue, then chan<- destination.Then)
Load is the function being run by the scheduler to load the data into the destination. It is in charge of the "L" in the ETL process.
func (Publish) Marshal ¶
func (a Publish) Marshal(tk *destination.Toolkit) (*destination.Job, error)
Marshal is the function being run when the action receives data into the Publish receiver. It allows to transform and enrich the data before saving it in the store adapter.
func (Publish) Schedule ¶
func (a Publish) Schedule() *destination.Schedule
Schedule allows the action to override the schedule options of its destination. Do not override.
type Topic ¶
type Topic struct {
// contains filtered or unexported fields
}
Topic implements the Blacksmith destination.Destination interface for working with message brokers.
func (*Topic) Actions ¶
func (d *Topic) Actions() map[string]destination.Action
Actions return a list of actions the destination Blob is able to handle.
func (*Topic) Init ¶
func (d *Topic) Init(tk *destination.Toolkit) error
Init is part of the destination.WithHooks interface. It allows to properly open the connection with the message broker. It is called when starting the scheduler service.
func (*Topic) Options ¶
func (d *Topic) Options() *destination.Options
Options returns common destination options for a blob storage. They will be shared across every actions of this destination, except when overridden.
Source Files