pubsub

Documentation

Overview

Package pubsub contains two generic interfaces for publishing data to queues and subscribing and consuming data from those queues.

// Publisher is a generic interface to encapsulate how we want our publishers
// to behave. Until we find reason to change, we're forcing all publishers
// to emit protobufs.
type Publisher interface {
    // Publish will publish a message.
    Publish(ctx context.Context, key string, msg proto.Message) error
    // Publish will publish a []byte message.
    PublishRaw(ctx context.Context, key string, msg []byte) error
}

// Subscriber is a generic interface to encapsulate how we want our subscribers
// to behave. For now the system will auto stop if it encounters any errors. If
// a user encounters a closed channel, they should check the Err() method to see
// what happened.
type Subscriber interface {
    // Start will return a channel of raw messages
    Start() <-chan SubscriberMessage
    // Err will contain any errors returned from the consumer connection.
    Err() error
    // Stop will initiate a graceful shutdown of the subscriber connection
    Stop() error
}

Where a `SubscriberMessage` is an interface that gives implementations a hook for acknowledging/delete messages. Take a look at the docs for each implementation in `pubsub` to see how they behave.

There are currently 3 implementations of each type of `pubsub` interfaces:

For pubsub via Amazon's SNS/SQS, you can use the `pubsub/aws` package.

For pubsub via Google's Pubsub, you can use the `pubsub/gcp` package.

For pubsub via Kafka topics, you can use the `pubsub/kafka` package.

For publishing via HTTP, you can use the `pubsub/http` package.

Index

• Variables
• type MultiPublisher
• type Publisher
• type Subscriber
• type SubscriberMessage

Variables

var Log = logrus.New()

Log is the structured logger used throughout the package.

Types

type MultiPublisher

type MultiPublisher interface {
	Publisher

	// PublishMulti will publish multiple messages with a context.
	PublishMulti(context.Context, []string, []proto.Message) error
	// PublishMultiRaw will publish multiple raw byte array messages with a context.
	PublishMultiRaw(context.Context, []string, [][]byte) error
}

MultiPublisher is an interface for publishers who support sending multiple messages in a single request, in addition to individual messages.

type Publisher

type Publisher interface {
	// Publish will publish a message with context.
	Publish(context.Context, string, proto.Message) error
	// Publish will publish a raw byte array as a message with context.
	PublishRaw(context.Context, string, []byte) error
}

Publisher is a generic interface to encapsulate how we want our publishers to behave. Until we find reason to change, we're forcing all pubslishers to emit protobufs.

type Subscriber

type Subscriber interface {
	// Start will return a channel of raw messages.
	Start() <-chan SubscriberMessage
	// Err will contain any errors returned from the consumer connection.
	Err() error
	// Stop will initiate a graceful shutdown of the subscriber connection.
	Stop() error
}

Subscriber is a generic interface to encapsulate how we want our subscribers to behave. For now the system will auto stop if it encounters any errors. If a user encounters a closed channel, they should check the Err() method to see what happened.

type SubscriberMessage

type SubscriberMessage interface {
	Message() []byte
	ExtendDoneDeadline(time.Duration) error
	Done() error
}

SubscriberMessage is a struct to encapsulate subscriber messages and provide a mechanism for acknowledging messages _after_ they've been processed.