forwarder

Documentation

Overview

Package forwarder defines the interfaces for data forwarders in different type of vertices.

Index

• type GoWhere
  • func (gw GoWhere) WhereTo(ks []string, ts []string, id string) ([]VertexBuffer, error)
• type StarterStopper
• type ToWhichStepDecider
• type VertexBuffer

Types

type GoWhere

type GoWhere func([]string, []string, string) ([]VertexBuffer, error)

GoWhere is the step decider on where it needs to go

func (GoWhere) WhereTo

func (gw GoWhere) WhereTo(ks []string, ts []string, id string) ([]VertexBuffer, error)

WhereTo decides where the data goes to.

type StarterStopper

type StarterStopper interface {
	Start() <-chan struct{}
	Stop()
	ForceStop()
}

StarterStopper starts/stops the forwarding.

type ToWhichStepDecider

type ToWhichStepDecider interface {
	// WhereTo decides where to forward the result based on the name of the step it returns.
	// It supports 2 additional keywords which need not be a step name. They are "ALL" and "DROP"
	// where former means, forward to all the neighbouring steps and latter means do not forward anywhere.
	//
	// Parameters:
	// - keys: Used by shuffle to decide which partition to write, if the toVertex is 'reduce' and has
	// multiple partitions. It is deterministic messages with same set of keys will always go to the same partition.
	//
	// - tags: Used for conditional forwarding.
	//
	// - id: Used by shuffle to decide which partition to write, if the toVertex is a 'map' and has
	// multiple partitions. It is deterministic messages with same id will always go to the same partition.
	WhereTo([]string, []string, string) ([]VertexBuffer, error)
}

ToWhichStepDecider decides which step to forward after applying the WhereTo function.

type VertexBuffer

type VertexBuffer struct {
	ToVertexName         string
	ToVertexPartitionIdx int32
}

VertexBuffer points to the partition of a buffer owned by the vertex.