mr

package
v1.0.22-0...-5969f35 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Apr 5, 2021 License: MIT Imports: 7 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var (
	// ErrCancelWithNil is an error that mapreduce was cancelled with nil.
	ErrCancelWithNil = errors.New("mapreduce cancelled with nil")
	// ErrReduceNoOutput is an error that reduce did not output a value.
	ErrReduceNoOutput = errors.New("reduce not writing value")
)

Functions

func Finish

func Finish(fns ...func() error) error

Finish runs fns parallelly, cancelled on any error.

func FinishVoid

func FinishVoid(fns ...func())

FinishVoid runs fns parallelly.

func Map

func Map(generate GenerateFunc, mapper MapFunc, opts ...Option) chan interface{}

Map maps all elements generated from given generate func, and returns an output channel.

func MapReduce

func MapReduce(generate GenerateFunc, mapper MapperFunc, reducer ReducerFunc, opts ...Option) (interface{}, error)

MapReduce maps all elements generated from given generate func, and reduces the output elemenets with given reducer.

func MapReduceVoid

func MapReduceVoid(generate GenerateFunc, mapper MapperFunc, reducer VoidReducerFunc, opts ...Option) error

MapReduceVoid maps all elements generated from given generate, and reduce the output elements with given reducer.

func MapReduceWithSource

func MapReduceWithSource(source <-chan interface{}, mapper MapperFunc, reducer ReducerFunc,
	opts ...Option) (interface{}, error)

MapReduceWithSource maps all elements from source, and reduce the output elements with given reducer.

func MapVoid

func MapVoid(generate GenerateFunc, mapper VoidMapFunc, opts ...Option)

MapVoid maps all elements from given generate but no output.

Types

type GenerateFunc

type GenerateFunc func(source chan<- interface{})

GenerateFunc is used to let callers send elements into source.

type MapFunc

type MapFunc func(item interface{}, writer Writer)

MapFunc is used to do element processing and write the output to writer.

type MapperFunc

type MapperFunc func(item interface{}, writer Writer, cancel func(error))

MapperFunc is used to do element processing and write the output to writer, use cancel func to cancel the processing.

type Option

type Option func(opts *mapReduceOptions)

Option defines the method to customize the mapreduce.

func WithWorkers

func WithWorkers(workers int) Option

WithWorkers customizes a mapreduce processing with given workers.

type ReducerFunc

type ReducerFunc func(pipe <-chan interface{}, writer Writer, cancel func(error))

ReducerFunc is used to reduce all the mapping output and write to writer, use cancel func to cancel the processing.

type VoidMapFunc

type VoidMapFunc func(item interface{})

VoidMapFunc is used to do element processing, but no output.

type VoidReducerFunc

type VoidReducerFunc func(pipe <-chan interface{}, cancel func(error))

VoidReducerFunc is used to reduce all the mapping output, but no output. Use cancel func to cancel the processing.

type Writer

type Writer interface {
	Write(v interface{})
}

Writer interface wraps Write method.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL