proc

Documentation

Index

• Constants
• func EOS(batch zbuf.Batch, err error) bool
• func FromFunction(pctx *Context, parent Interface, f Function) *applier
• func NopDone(puller zbuf.Puller) *done
• type Context
  • func DefaultContext() *Context
  • func NewContext(ctx context.Context, zctx *zed.Context, logger *zap.Logger) *Context
  • func (c *Context) Cancel()
• type DataAdaptor
• type Function
• type Interface
• type Result
• type Scheduler

Constants

const BatchLen = 100

Functions

func EOS

func EOS(batch zbuf.Batch, err error) bool

func FromFunction

func FromFunction(pctx *Context, parent Interface, f Function) *applier

func NopDone

func NopDone(puller zbuf.Puller) *done

Types

type Context

type Context struct {
	context.Context
	Logger   *zap.Logger
	Warnings chan string
	Zctx     *zed.Context
	// contains filtered or unexported fields
}

Context provides states used by all procs to provide the outside context in which they are running.

func DefaultContext

func DefaultContext() *Context

func NewContext

func NewContext(ctx context.Context, zctx *zed.Context, logger *zap.Logger) *Context

func (*Context) Cancel

func (c *Context) Cancel()

type DataAdaptor

type DataAdaptor interface {
	PoolID(context.Context, string) (ksuid.KSUID, error)
	CommitObject(context.Context, ksuid.KSUID, string) (ksuid.KSUID, error)
	Layout(context.Context, dag.Source) order.Layout
	NewScheduler(context.Context, *zed.Context, dag.Source, extent.Span, zbuf.Filter) (Scheduler, error)
	Open(context.Context, *zed.Context, string, zbuf.Filter) (zbuf.PullerCloser, error)
}

type Function

type Function interface {
	fmt.Stringer
	Apply(*zed.Record) (*zed.Record, error)
	Warning() string
}

type Interface

type Interface interface {
	zbuf.Puller
	Done()
}

proc.Interface is the interface to objects that operate on Batches of zed.Records and are arranged into a flowgraph to perform pattern matching and analytics. A proc is generally single-threaded unless lengths are taken to implement concurrency within a Proc. The model is receiver-driven, stream-oriented data processing. Downstream procs Pull() batches of data from upstream procs. Normally, a proc pulls data until end of stream (nil batch and nil error) or error (nil batch and non-nil error). If a proc wants to end before end of stream, it calls the Done() method on its parent. A proc implementation may assume calls to Pull() and Done() are single threaded so any arrangement of calls to Pull() and Done() cannot be done concurrently. In short, never call Done() concurrently to another goroutine calling Pull().

type Result

type Result struct {
	Batch zbuf.Batch
	Err   error
}

Result is a convenient way to bundle the result of Proc.Pull() to send over channels.

type Scheduler

type Scheduler interface {
	PullScanTask() (zbuf.PullerCloser, error)
	Stats() zbuf.ScannerStats
}