pulsarutils

package
v0.4.54 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 21, 2024 License: Apache-2.0 Imports: 13 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CompactAndPublishSequences

func CompactAndPublishSequences(ctx *armadacontext.Context, sequences []*armadaevents.EventSequence, producer pulsar.Producer, maxMessageSizeInBytes uint) error

CompactAndPublishSequences reduces the number of sequences to the smallest possible, while respecting per-job set ordering and max Pulsar message size, and then publishes to Pulsar.

func NewMessageId

func NewMessageId(id int) pulsar.MessageID

func NewPulsarClient

func NewPulsarClient(config *configuration.PulsarConfig) (pulsar.Client, error)

func PublishSequences

func PublishSequences(ctx *armadacontext.Context, producer pulsar.Producer, sequences []*armadaevents.EventSequence) error

PublishSequences publishes several event sequences to Pulsar. For efficiency, all sequences are queued for publishing and then flushed. Returns once all sequences have been received by Pulsar.

To reduce the number of separate sequences sent and ensure limit message size, call eventutil.CompactEventSequences(sequences) and eventutil.LimitSequencesByteSize(sequences, int(srv.MaxAllowedMessageSize)) before passing to this function.

Types

type MockMessageId

type MockMessageId struct {
	pulsar.MessageID
	// contains filtered or unexported fields
}

type MockPulsarMessage

type MockPulsarMessage struct {
	pulsar.Message
	// contains filtered or unexported fields
}

func NewPulsarMessage

func NewPulsarMessage(id int, publishTime time.Time, payload []byte) MockPulsarMessage

func (MockPulsarMessage) ID

func (MockPulsarMessage) Payload

func (m MockPulsarMessage) Payload() []byte

func (MockPulsarMessage) Properties

func (m MockPulsarMessage) Properties() map[string]string

func (MockPulsarMessage) PublishTime

func (m MockPulsarMessage) PublishTime() time.Time

type Publisher added in v0.4.44

type Publisher interface {
	PublishMessages(ctx *armadacontext.Context, es *armadaevents.EventSequence) error
	Close()
}

Publisher is an interface to be implemented by structs that handle publishing messages to pulsar

type PulsarPublisher added in v0.4.44

type PulsarPublisher struct {
	// contains filtered or unexported fields
}

PulsarPublisher is the default implementation of Publisher

func NewPulsarPublisher added in v0.4.44

func NewPulsarPublisher(
	pulsarClient pulsar.Client,
	producerOptions pulsar.ProducerOptions,
	maxAllowedMessageSize uint,
) (*PulsarPublisher, error)

func (*PulsarPublisher) Close added in v0.4.44

func (p *PulsarPublisher) Close()

func (*PulsarPublisher) PublishMessages added in v0.4.44

func (p *PulsarPublisher) PublishMessages(ctx *armadacontext.Context, es *armadaevents.EventSequence) error

PublishMessages publishes all event sequences to pulsar. Event sequences for a given jobset will be combined into single event sequences up to maxMessageBatchSize.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL