idlehandler

Documentation

Index

• func PublishIdleWatermark(ctx context.Context, fromBufferPartitionIndex int32, ...)
• type SourceIdleHandler
  • func NewSourceIdleHandler(config *dfv1.Watermark, fetcher fetch.SourceFetcher, ...) *SourceIdleHandler
  • func (iw *SourceIdleHandler) IsSourceIdling() bool
  • func (iw *SourceIdleHandler) PublishSourceIdleWatermark(partitions []int32)
  • func (iw *SourceIdleHandler) Reset()

Functions

func PublishIdleWatermark

func PublishIdleWatermark(ctx context.Context, fromBufferPartitionIndex int32, toBufferPartition isb.BufferWriter, wmPublisher publish.Publisher, idleManager wmb.IdleManager, logger *zap.SugaredLogger, vertexName string, pipelineName string, vertexType dfv1.VertexType, vertexReplica int32, wm wmb.Watermark)

PublishIdleWatermark publishes a ctrl message with isb.Kind set to WMB. We only send one ctrl message when

Types

type SourceIdleHandler

type SourceIdleHandler struct {
	// contains filtered or unexported fields
}

SourceIdleHandler handles operations related to idle watermarks for source.

func NewSourceIdleHandler

func NewSourceIdleHandler(config *dfv1.Watermark, fetcher fetch.SourceFetcher, publisher publish.SourcePublisher) *SourceIdleHandler

NewSourceIdleHandler creates a new instance of SrcIdleHandler.

func (*SourceIdleHandler) IsSourceIdling

func (iw *SourceIdleHandler) IsSourceIdling() bool

IsSourceIdling will return true if source has been idling and the step interval has passed.

func (*SourceIdleHandler) PublishSourceIdleWatermark

func (iw *SourceIdleHandler) PublishSourceIdleWatermark(partitions []int32)

PublishSourceIdleWatermark publishes an idle watermark.

func (*SourceIdleHandler) Reset

func (iw *SourceIdleHandler) Reset()

Reset resets the updatedTS to the current time.