pseq

Documentation

Overview

Package pseq contains utilities for parallelisation

Index

• func Each[A any](seq iter.Seq[A], f func(A))
• func Each2[A, B any](seq iter.Seq2[A, B], f func(A, B))
• func MapMerge[A, B any](seq iter.Seq[A], f func(A) B, m func(B, B) B) B
• func MapReduce[A, B, C any, K comparable](seq iter.Seq[A], m func(A) iter.Seq2[K, B], r func(K, iter.Seq[B]) C) iter.Seq2[K, C]

Functions

func Each

func Each[A any](seq iter.Seq[A], f func(A))

Each runs f in parallel, passing in all elements from seq.

func Each2

func Each2[A, B any](seq iter.Seq2[A, B], f func(A, B))

Each2 runs f in parallel, passing in all pairs from seq.

func MapMerge

func MapMerge[A, B any](seq iter.Seq[A], f func(A) B, m func(B, B) B) B

MapMerge runs f in parallel, merging the results using m. m must be associative and commutative.

func MapReduce

func MapReduce[A, B, C any, K comparable](seq iter.Seq[A], m func(A) iter.Seq2[K, B], r func(K, iter.Seq[B]) C) iter.Seq2[K, C]

MapReduce runs a MapReduce pipeline. It runs m (the map phase) in parallel to split the sequence of inputs into key/value pairs. It then spawns one goroutine running r per key (the reduce phase). Lastly, the outputs of all reducers are output as a sequence.