Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type PublishOption ¶
type PublishOption func(*publishOptions)
func IsSource ¶ added in v0.5.6
func IsSource() PublishOption
IsSource indicates it's a source publisher
func WithAutoRefreshHeartbeatDisabled ¶ added in v0.5.2
func WithAutoRefreshHeartbeatDisabled() PublishOption
func WithDefaultPartitionIdx ¶ added in v1.1.0
func WithDefaultPartitionIdx(partitionIdx int32) PublishOption
WithDefaultPartitionIdx sets the default partition index
func WithDelay ¶ added in v0.5.6
func WithDelay(t time.Duration) PublishOption
WithDelay sets the watermark delay
func WithPodHeartbeatRate ¶
func WithPodHeartbeatRate(rate int64) PublishOption
type Publisher ¶
type Publisher interface {
io.Closer
// PublishWatermark publishes the watermark.
PublishWatermark(w wmb.Watermark, o isb.Offset, toVertexPartitionIdx int32)
// PublishIdleWatermark publishes the idle watermark.
PublishIdleWatermark(wm wmb.Watermark, o isb.Offset, toVertexPartitionIdx int32)
// GetLatestWatermark returns the latest published watermark.
GetLatestWatermark() wmb.Watermark
}
Publisher interface defines how to publish Watermark for a ProcessorEntitier.
func NewPublish ¶
func NewPublish(ctx context.Context, processorEntity entity.ProcessorEntitier, watermarkStores store.WatermarkStore, toVertexPartitionCount int32, inputOpts ...PublishOption) Publisher
NewPublish returns `Publish`.
type SourcePublisher ¶ added in v1.1.0
type SourcePublisher interface {
// PublishSourceWatermarks publishes source watermarks.
PublishSourceWatermarks([]*isb.ReadMessage)
// PublishIdleWatermarks publishes idle watermarks for the given partitions.
PublishIdleWatermarks(wm time.Time, partitions []int32)
}
SourcePublisher publishes source watermarks based on a list of isb.ReadMessage and also publishes idle watermarks. It internally calls the main WM Publisher to publish the watermark. The only difference is that in source we do not care about offsets in the first Publish within the source itself (huh? :-D). Also, when the source boots up, it has to load the watermark information so it can know about the global source WM state.
func NewSourcePublish ¶ added in v1.1.0
func NewSourcePublish(ctx context.Context, pipelineName, vertexName string, srcPublishWMStores store.WatermarkStore, opts ...PublishOption) SourcePublisher
NewSourcePublish returns a new source publisher.
Source Files