Documentation
¶
Overview ¶
Package parallel contains implementations of various buffer types where the buffer can be consumed by any number of parallel consumer threads. Therefore, since it is possible for consumers to requeue a message if the propagation failed, it is possible for messages to be consumed out of sequence.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var (
BoltDBMessagesBucket = []byte("benthos_messages")
)
Static keys holding BoltDB bucket names for the Benthos buffer.
Functions ¶
This section is empty.
Types ¶
type AckFunc ¶
AckFunc is a func returned when a message is read from a parallel buffer. The func should be called when the message is finished with, and a flag indicates whether the message was successfully propagated and can be removed from the buffer. Returns the current backlog of the buffer in bytes, or an error if the message was not successfully removed.
If an error is returned it is safe to call the function again. Otherwise, it is not.
It is safe to call this func even if the buffer has closed.
type BoltDB ¶
type BoltDB struct {
// contains filtered or unexported fields
}
BoltDB is a parallel buffer implementation that allows multiple parallel consumers to read and purge messages from a BoltDB database asynchronously.
func NewBoltDB ¶
func NewBoltDB(conf BoltDBConfig) (*BoltDB, error)
NewBoltDB creates a memory based parallel buffer.
func (*BoltDB) Close ¶
func (m *BoltDB) Close()
Close closes the Buffer so that blocked readers or writers become unblocked.
func (*BoltDB) CloseOnceEmpty ¶
func (m *BoltDB) CloseOnceEmpty()
CloseOnceEmpty closes the Buffer once the buffer has been emptied, this is a way for a writer to signal to a reader that it is finished writing messages, and therefore the reader can close once it is caught up. This call blocks until the close is completed.
func (*BoltDB) NextMessage ¶
NextMessage reads the next oldest message, the message is preserved until the returned AckFunc is called.
func (*BoltDB) PushMessage ¶
PushMessage adds a message to the stack. Returns the backlog in bytes.
type BoltDBConfig ¶
type BoltDBConfig struct {
File string `json:"file" yaml:"file"`
PrefetchCount int `json:"prefetch_count" yaml:"prefetch_count"`
}
BoltDBConfig contains configuration params for the BoltDB buffer type.
func NewBoltDBConfig ¶
func NewBoltDBConfig() BoltDBConfig
NewBoltDBConfig returns a BoltDBConfig with default parameters.
type Memory ¶
type Memory struct {
// contains filtered or unexported fields
}
Memory is a parallel buffer implementation that allows multiple parallel consumers to read and purge messages from the buffer asynchronously.
func (*Memory) Close ¶
func (m *Memory) Close()
Close closes the Buffer so that blocked readers or writers become unblocked.
func (*Memory) CloseOnceEmpty ¶
func (m *Memory) CloseOnceEmpty()
CloseOnceEmpty closes the Buffer once the buffer has been emptied, this is a way for a writer to signal to a reader that it is finished writing messages, and therefore the reader can close once it is caught up. This call blocks until the close is completed.
func (*Memory) NextMessage ¶
NextMessage reads the next oldest message, the message is preserved until the returned AckFunc is called.
Source Files