Documentation ¶
Overview ¶
Package logstream provides beam utilities to interact with LogStream and LogStreamState datastore entities.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BackfillExpireAtFromCreated ¶
func BackfillExpireAtFromCreated(s beam.Scope, cloudProject string, logStreamKeys beam.PCollection, opts BackfillOptions) beam.PCollection
BackfillExpireAtFromCreated takes a PCollection<KeyBatch> and updates all the entities' ExpireAt timestamp from its Created timestamp according to expiry.
Types ¶
type BackfillOptions ¶
type BackfillOptions struct { // DryRun controls whether the datastore updates should be applied. DryRun bool // Workers controls the number of go worker in each beam worker. Workers int // BatchSize controls the number of elements each go worker attempt to process // at a time. BatchSize int // SkipCreatedAfter entities created after this timestamp will be skipped. // Older entities are always finalized. This allow us to do updates without // using a transaction since this is the only process that will update those // entities. SkipCreatedAfter time.Time // Expiry is the duration added to entities creation time to obtain the expiry // date. Expiry time.Duration // RetryCount is the number of times a failed batch will be retried. RetryCount int }
type RandomizedExponentialBackoff ¶
type RandomizedExponentialBackoff struct { retry.ExponentialBackoff MaxIncreaseRatio float64 }
RandomizedExponentialBackoff is similar to ExponentialBackoff but the actual delay is a random duration between [thisDelay, thisDelay * (1+MaxIncreaseRatio)).
Click to show internal directories.
Click to hide internal directories.