sequtil

Documentation

Overview

Package sequtil contains sequencer utility methods.

Index

• func LeaseGroup(outer *stopper.Context, leases types.Leases, group *types.TableGroup, ...)
• type Copier
  • func (c *Copier) Run(ctx *stopper.Context) error
• type EachFn
• type FlushFn
• type ProgressFn

Functions

func LeaseGroup

func LeaseGroup(
	outer *stopper.Context,
	leases types.Leases,
	group *types.TableGroup,
	fn func(*stopper.Context, *types.TableGroup),
)

LeaseGroup ensures that multiple sequencers do not operate on the same tables. This function will create a goroutine within the context that acquires a lease based on the group name. The callback will be executed as a goroutine within a suitably nested stopper.

Types

type Copier

type Copier struct {
	Config   *sequencer.Config           // Controls for flush behavior.
	Each     EachFn                      // Optional callback to receive each batch.
	Flush    FlushFn                     // Optional callback to receive aggregated data.
	Progress ProgressFn                  // Optional callback when the source has become idle.
	Source   <-chan *types.StagingCursor // Input data.
}

A Copier consumes a channel of types.StagingCursor, assembles the individual temporal batches into large batches, and invokes event callbacks to process the larger batches.

func (*Copier) Run

func (c *Copier) Run(ctx *stopper.Context) error

Run copies data from the source to the target. It is a blocking call that will return if the context is stopped, the channel is closed, or if the target returns an error.

type EachFn

type EachFn func(ctx *stopper.Context, batch *types.TemporalBatch, fragment bool) error

EachFn is a callback from a Copier.

type FlushFn

type FlushFn func(ctx *stopper.Context, batch *types.MultiBatch, fragment bool) error

FlushFn is a callback from a Copier.

type ProgressFn

type ProgressFn func(ctx *stopper.Context, progress hlc.Range) error

ProgressFn is a callback from a Copier.