jetstream

Documentation

Index

• func BuildSourcePublisherStores(ctx context.Context, vertexInstance *v1alpha1.VertexInstance) (store.WatermarkStorer, error)
• func BuildWatermarkProgressors(ctx context.Context, vertexInstance *v1alpha1.VertexInstance, ...) (fetch.Fetcher, map[string]publish.Publisher, error)

Functions

func BuildSourcePublisherStores

func BuildSourcePublisherStores(ctx context.Context, vertexInstance *v1alpha1.VertexInstance) (store.WatermarkStorer, error)

BuildSourcePublisherStores builds the watermark stores for source publisher.

func BuildWatermarkProgressors

func BuildWatermarkProgressors(ctx context.Context, vertexInstance *v1alpha1.VertexInstance, fromBuffer v1alpha1.Buffer) (fetch.Fetcher, map[string]publish.Publisher, error)

BuildWatermarkProgressors is used to populate fetchWatermark, and a map of publishWatermark with edge name as the key. These are used as watermark progressors in the pipeline, and is attached to each edge of the vertex. The function is used only when watermarking is enabled on the pipeline.