passert

Documentation

Overview

Package passert contains verification transformations for testing pipelines. The transformations are not tied to any particular runner, i.e., they can notably be used for remote execution runners, such as Dataflow.

Index

• func Diff(s beam.Scope, a, b beam.PCollection) (left, both, right beam.PCollection)
• func Empty(s beam.Scope, col beam.PCollection) beam.PCollection
• func Equals(s beam.Scope, col beam.PCollection, values ...interface{}) beam.PCollection
• func False(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection
• func Hash(s beam.Scope, col beam.PCollection, name, hash string, size int)
• func True(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection

Functions

func Diff

func Diff(s beam.Scope, a, b beam.PCollection) (left, both, right beam.PCollection)

Diff splits 2 incoming PCollections into 3: left only, both, right only. Duplicates are preserved, so a value may appear multiple times and in multiple collections. Coder equality is used to determine equality. Should only be used for small collections, because all values are held in memory at the same time.

func Empty

func Empty(s beam.Scope, col beam.PCollection) beam.PCollection

Empty asserts that col is empty.

func Equals

func Equals(s beam.Scope, col beam.PCollection, values ...interface{}) beam.PCollection

Equals verifies the given collection has the same values as the given values, under coder equality. The values can be provided as single PCollection.

func False

func False(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection

False asserts that the given predicate does not satisfy any element in the condition.

func Hash

func Hash(s beam.Scope, col beam.PCollection, name, hash string, size int)

Hash validates that the incoming PCollection<string> has the given size and base64-encoded MD5 hash code. It buffers the entire PCollection in memory and sorts it for determinism.

func True

func True(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection

True asserts that all elements satisfy the given predicate.