Documentation ¶
Index ¶
- Variables
- func Finish(fns ...func() error) error
- func FinishVoid(fns ...func())
- func Map(generate GenerateFunc, mapper MapFunc, opts ...Option) chan interface{}
- func MapReduce(generate GenerateFunc, mapper MapperFunc, reducer ReducerFunc, opts ...Option) (interface{}, error)
- func MapReduceVoid(generate GenerateFunc, mapper MapperFunc, reducer VoidReducerFunc, ...) error
- func MapReduceWithSource(source <-chan interface{}, mapper MapperFunc, reducer ReducerFunc, ...) (interface{}, error)
- func MapVoid(generate GenerateFunc, mapper VoidMapFunc, opts ...Option)
- type GenerateFunc
- type MapFunc
- type MapperFunc
- type Option
- type ReducerFunc
- type VoidMapFunc
- type VoidReducerFunc
- type Writer
Constants ¶
This section is empty.
Variables ¶
var ( // ErrCancelWithNil is an error that mapreduce was cancelled with nil. ErrCancelWithNil = errors.New("mapreduce cancelled with nil") // ErrReduceNoOutput is an error that reduce did not output a value. ErrReduceNoOutput = errors.New("reduce not writing value") )
Functions ¶
func Map ¶
func Map(generate GenerateFunc, mapper MapFunc, opts ...Option) chan interface{}
Map maps all elements generated from given generate func, and returns an output channel.
func MapReduce ¶
func MapReduce(generate GenerateFunc, mapper MapperFunc, reducer ReducerFunc, opts ...Option) (interface{}, error)
MapReduce maps all elements generated from given generate func, and reduces the output elemenets with given reducer.
func MapReduceVoid ¶
func MapReduceVoid(generate GenerateFunc, mapper MapperFunc, reducer VoidReducerFunc, opts ...Option) error
MapReduceVoid maps all elements generated from given generate, and reduce the output elements with given reducer.
func MapReduceWithSource ¶
func MapReduceWithSource(source <-chan interface{}, mapper MapperFunc, reducer ReducerFunc, opts ...Option) (interface{}, error)
MapReduceWithSource maps all elements from source, and reduce the output elements with given reducer.
func MapVoid ¶
func MapVoid(generate GenerateFunc, mapper VoidMapFunc, opts ...Option)
MapVoid maps all elements from given generate but no output.
Types ¶
type GenerateFunc ¶
type GenerateFunc func(source chan<- interface{})
GenerateFunc is used to let callers send elements into source.
type MapFunc ¶
type MapFunc func(item interface{}, writer Writer)
MapFunc is used to do element processing and write the output to writer.
type MapperFunc ¶
MapperFunc is used to do element processing and write the output to writer, use cancel func to cancel the processing.
type Option ¶
type Option func(opts *mapReduceOptions)
Option defines the method to customize the mapreduce.
func WithWorkers ¶
WithWorkers customizes a mapreduce processing with given workers.
type ReducerFunc ¶
ReducerFunc is used to reduce all the mapping output and write to writer, use cancel func to cancel the processing.
type VoidMapFunc ¶
type VoidMapFunc func(item interface{})
VoidMapFunc is used to do element processing, but no output.
type VoidReducerFunc ¶
type VoidReducerFunc func(pipe <-chan interface{}, cancel func(error))
VoidReducerFunc is used to reduce all the mapping output, but no output. Use cancel func to cancel the processing.