processor

Documentation

Overview

Package processor provides a way to run a function against a collection of various input sources.

Index

• type ProcessFunc
• type Processor
  • func NewProcessor() *Processor
  • func (p *Processor) ProcessFiles(ctx context.Context, paths []string, fn ProcessFunc) error
  • func (p *Processor) SetProgressReporter(reporter ProgressReporter)
• type ProgressReporter

Types

type ProcessFunc

type ProcessFunc func(ctx context.Context, r io.Reader) error

ProcessFunc is provided to the processor and will be called on each input source that needs processing.

type Processor

type Processor struct {
	// contains filtered or unexported fields
}

Processor is used to process multiple input sources.

func NewProcessor

func NewProcessor() *Processor

NewProcessor creates a new processor.

func (*Processor) ProcessFiles

func (p *Processor) ProcessFiles(ctx context.Context, paths []string, fn ProcessFunc) error

ProcessFiles will run the given ProcessFunc on the set of input file paths. Zip files are also supported and each individual file in the zip will be processed.

func (*Processor) SetProgressReporter

func (p *Processor) SetProgressReporter(reporter ProgressReporter)

SetProgressReporter sets the progress reporter to use.

type ProgressReporter

type ProgressReporter interface {
	// Started will be called when a new path is being processed.
	// index is the 0th based index of the path in the total number of paths.
	// path uniquely identifies a resource and does not have to be a file path (e.g. a URI)
	Started(path string, index int, total int)

	// Reader returns a new wrapped reader that will update and report progress as data
	// is being read from it.
	Reader(r io.Reader) io.Reader

	// AddToTotalSize is called when the total number of bytes to be processed has changed.
	// For example like reading from a zip file.
	AddToTotalSize(add int64)
}

ProgressReporter is used to report progress updates.