simpleprocessor

Documentation

Overview

This package provides the Processor interface which is implemented by all the other packages in the parent package

Index

• type ChunkProcessor
• type Processor

Types

type ChunkProcessor

type ChunkProcessor interface {
	Processor

	// New returns a new copy of the chunkProcessor based on its current configuration
	New() ChunkProcessor

	//It will be in Syncing by default, and then for our simple implementation case, go into validating, running, and then success or failure
	// It returns a receive only channel, as we can't send on to the state
	// In the future, DataChunkState may be a struct that contains metadata like progress (e.g. download progress). This is a #1 reason to use channels, for allowing rich state updates
	// If the state is only a sequence from Waiting to Validating to Determining to Running to a terminated state, then we can simply have functions which represent these things
	// and run them synchronously, having a manager update the higher up state
	State() <-chan processors.DataChunkState

	// TODO: figure out if the data/merging with config should be abstracted out like we did with default configs and GetConfig above
	Run(data interface{})

	// Output fetches the data output on successful completion
	Output() interface{}

	//GetError returns the error which occurred if it goes from running to failed
	GetError() error
}

For now, most builtin processors will only emit these states: WAITING, RUNNING, and succeeded/failed

type Processor

type Processor interface {
	// Configure configures the processor with the given configuration or an error if its invalid
	// This should not depend on any previously set configuration or data
	Configure(config interface{}) error

	// GetConfig returns the current configuration
	// It must return a pointer
	GetConfig() interface{}

	Metadata() processors.Processor
}

Processor is a simple system for Go builtin processors. It should abstract away Runs, etc and only need configuration and data Each chunk should be able to spawn a new processor. In the future, this may change Also, each processor should only be configured once, for that chunk, thus any reconfiguration logic is not implemented yet