async

README

middlewares/async

The middlewares/async package contains middlewares to be used by source to provide asynchronous writes into the sink, enabling the sources to read concurrently.

Sinks (and their interface abstract.Sink) must stay as simple as possible. When write concurrency (parallel writes) is desired, one should create multiple sinks (with complete pipelines) instead of using the same sink for concurrent writes, as the latter enormously and unnecessarily complicates the implementation of a sink.

To provide concurrency on the source side and support both replication and snapshot sources, asynchonous middlewares in this package are provided.

Documentation

Index

• func Measurer(logger log.Logger) func(abstract.AsyncSink) abstract.AsyncSink
• func Synchronizer(logger log.Logger) func(abstract.Sinker) abstract.AsyncSink

Functions

func Measurer

func Measurer(logger log.Logger) func(abstract.AsyncSink) abstract.AsyncSink

Measurer calculate the size of items passing through it

func Synchronizer

func Synchronizer(logger log.Logger) func(abstract.Sinker) abstract.AsyncSink

Synchronizer provides AsyncPush which is executed synchronously with the underlying synchronous Push. In other words, it provides synchronous replication pipeline.

It has the required built-in mutexes, so that it does not call underlying sink concurrently.