mapreduce

package
v0.0.0-...-a2846b3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 16, 2016 License: BSD-3-Clause Imports: 2 Imported by: 0

Documentation

Overview

The Tideland Go Library mapreduce package implements the Map/Reduce algorithm for the processing and aggregation mass data.

A type implementing the MapReducer interface has to be implemented and passed to the MapReduce() function. The type is responsible for the input, the mapping, the reducing and the consuming while the package provides the runtime environment for it.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func MapReduce

func MapReduce(mr MapReducer) error

MapReduce applies a map and a reduce function to keys and values in parallel.

Types

type KeyValue

type KeyValue interface {
	// Key returns the key for the mapping.
	Key() string

	// Value returns the payload value for processing.
	Value() interface{}
}

KeyValue is a pair of a string key and any data as value.

type KeyValueChan

type KeyValueChan chan KeyValue

KeyValueChan is a channel for the transfer of key/value pairs.

func (KeyValueChan) Close

func (c KeyValueChan) Close()

Close the channel for key/value pairs.

type MapReducer

type MapReducer interface {
	// Input has to return the input channel for the
	// date to process.
	Input() KeyValueChan

	// Map maps a key/value pair to another one and emits it.
	Map(in KeyValue, emit KeyValueChan)

	// Reduce reduces the values delivered via the input
	// channel to the emit channel.
	Reduce(in, emit KeyValueChan)

	// Consume allows the MapReducer to consume the
	// processed data.
	Consume(in KeyValueChan) error
}

MapReducer has to be implemented to control the map/reducing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL