Documentation
¶
Overview ¶
Package parallel implements helpers for fast processing of line oriented inputs.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var New = NewProcessor
New is a preferred way to create a new parallel processor.
Functions ¶
This section is empty.
Types ¶
type BytesBatch ¶
type BytesBatch struct {
// contains filtered or unexported fields
}
BytesBatch is a slice of byte slices.
func NewBytesBatch ¶
func NewBytesBatch() *BytesBatch
NewBytesBatch creates a new BytesBatch with a given capacity.
func NewBytesBatchCapacity ¶
func NewBytesBatchCapacity(cap int) *BytesBatch
NewBytesBatchCapacity creates a new BytesBatch with a given capacity.
func (*BytesBatch) Size ¶
func (bb *BytesBatch) Size() int
Size returns the number of elements in the batch.
func (*BytesBatch) Slice ¶
func (bb *BytesBatch) Slice() [][]byte
Slice returns a slice of byte slices.
type Processor ¶
type Processor struct {
BatchSize int
RecordSeparator byte
NumWorkers int
SkipEmptyLines bool
Verbose bool
R io.Reader
W io.Writer
F TransformerFunc
}
Processor can process lines in parallel.
func NewProcessor ¶
NewProcessor creates a new line processor.
func (*Processor) RunWorkers ¶
RunWorkers allows to quickly set the number of workers.
type SimpleTransformerFunc ¶
SimpleTransformerFunc converts bytes to bytes.
type TransformerFunc ¶
TransformerFunc takes a slice of bytes and returns a slice of bytes and a an error. A common denominator of functions that transform data.
func ToTransformerFunc ¶
func ToTransformerFunc(f SimpleTransformerFunc) TransformerFunc
ToTransformerFunc takes a simple transformer and wraps it so it can be used in places where a TransformerFunc is expected.
Source Files
Directories