Documentation
¶
Overview ¶
Package reduce reads messages from isb attaches watermark to read messages invoke the read-loop with the read messages
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type DataForward ¶
type DataForward struct {
// contains filtered or unexported fields
}
DataForward reads data from isb and forwards them to readloop
func NewDataForward ¶
func NewDataForward(ctx context.Context, udf applier.ReduceApplier, vertexInstance *dfv1.VertexInstance, fromBuffer isb.BufferReader, toBuffers map[string]isb.BufferWriter, pbqManager *pbq.Manager, whereToDecider forward.ToWhichStepDecider, fw fetch.Fetcher, watermarkPublishers map[string]publish.Publisher, windowingStrategy window.Windower, opts ...Option) (*DataForward, error)
func (*DataForward) Start ¶
func (d *DataForward) Start()
Start starts forwarding messages to readloop
type Option ¶
func WithReadBatchSize ¶
WithReadBatchSize sets the read batch size
type Options ¶
type Options struct {
// contains filtered or unexported fields
}
Options for forwarding the message
func DefaultOptions ¶
func DefaultOptions() *Options
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
store/wal
Package wal implements write-ahead-log.
|
Package wal implements write-ahead-log. |
|
Package pnf processes and then forwards messages belonging to a window.
|
Package pnf processes and then forwards messages belonging to a window. |
|
Package readloop is responsible for the first part of reduce subsystem.
|
Package readloop is responsible for the first part of reduce subsystem. |
Click to show internal directories.
Click to hide internal directories.