Documentation
¶
Overview ¶
Package filter contains transformations for removing pipeline elements based on various conditions.
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func Distinct ¶
func Distinct(s beam.Scope, col beam.PCollection) beam.PCollection
Distinct removes all duplicates from a collection, under coder equality. It expects a PCollection<T> as input and returns a PCollection<T> with duplicates removed.
func Exclude ¶
func Exclude(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection
Exclude filters the elements of a PCollection<A> based on the given function, which must be of the form: A -> bool. Exclude removes all element for which the filter function returns true. It returns a PCollection of the same type as the input. For example:
words := beam.Create(s, "a", "b", "long", "alsolong")
long := filter.Exclude(s, words, func(s string) bool {
return len(s) < 3
})
Here, "long" will contain "long" and "alsolong" at runtime.
func Include ¶
func Include(s beam.Scope, col beam.PCollection, fn interface{}) beam.PCollection
Include filters the elements of a PCollection<A> based on the given function, which must be of the form: A -> bool. Include removes all element for which the filter function returns false. It returns a PCollection of the same type as the input. For example:
words := beam.Create(s, "a", "b", "long", "alsolong")
short := filter.Include(s, words, func(s string) bool {
return len(s) < 3
})
Here, "short" will contain "a" and "b" at runtime.
Types ¶
This section is empty.
Source Files