table

Documentation

Index

• func FromBuffer(cr flux.ColReader) flux.Table
• func NewDataset(id execute.DatasetID, cache *BuilderCache) execute.Dataset
• func Stream(key flux.GroupKey, cols []flux.ColMeta, ...) (flux.Table, error)
• func StreamWithContext(ctx context.Context, key flux.GroupKey, cols []flux.ColMeta, ...) (flux.Table, error)
• func Values(cr flux.ColReader, j int) array.Interface
• type ArrowBuilder
  • func GetArrowBuilder(key flux.GroupKey, cache *BuilderCache) (builder *ArrowBuilder, created bool)
  • func NewArrowBuilder(key flux.GroupKey, mem memory.Allocator) *ArrowBuilder
  • func (a *ArrowBuilder) AddCol(c flux.ColMeta) (int, error)
  • func (a *ArrowBuilder) CheckCol(c flux.ColMeta) (int, error)
  • func (a *ArrowBuilder) Cols() []flux.ColMeta
  • func (a *ArrowBuilder) Key() flux.GroupKey
  • func (a *ArrowBuilder) Release()
  • func (a *ArrowBuilder) Table() (flux.Table, error)
• type BufferedBuilder
  • func GetBufferedBuilder(key flux.GroupKey, cache *BuilderCache) (builder *BufferedBuilder, created bool)
  • func NewBufferedBuilder(key flux.GroupKey, mem memory.Allocator) *BufferedBuilder
  • func (b *BufferedBuilder) AppendBuffer(cr flux.ColReader) error
  • func (b *BufferedBuilder) Release()
  • func (b *BufferedBuilder) Table() (flux.Table, error)
• type BufferedTable
  • func (b *BufferedTable) Cols() []flux.ColMeta
  • func (b *BufferedTable) Do(f func(flux.ColReader) error) error
  • func (b *BufferedTable) Done()
  • func (b *BufferedTable) Empty() bool
  • func (b *BufferedTable) Key() flux.GroupKey
• type Builder
• type BuilderCache
  • func (d *BuilderCache) DiscardTable(key flux.GroupKey)
  • func (d *BuilderCache) ExpireTable(key flux.GroupKey)
  • func (d *BuilderCache) ForEach(f func(key flux.GroupKey, builder Builder) error) error
  • func (d *BuilderCache) Get(key flux.GroupKey, b interface{}) bool
  • func (d *BuilderCache) Table(key flux.GroupKey) (flux.Table, error)
• type SendFunc
• type StreamWriter
  • func (s *StreamWriter) Cols() []flux.ColMeta
  • func (s *StreamWriter) Key() flux.GroupKey
  • func (s *StreamWriter) UnsafeWrite(vs []array.Interface) error
  • func (s *StreamWriter) UnsafeWriteBuffer(cr flux.ColReader) error
  • func (s *StreamWriter) Write(vs []array.Interface) error

Functions

func FromBuffer

func FromBuffer(cr flux.ColReader) flux.Table

FromBuffer constructs a flux.Table from a single flux.ColReader.

func NewDataset

func NewDataset(id execute.DatasetID, cache *BuilderCache) execute.Dataset

NewDataset constructs an execute.Dataset that is compatible with the BuilderCache.

This dataset does not support triggers and will only flush tables when the dataset is finished.

func Stream

func Stream(key flux.GroupKey, cols []flux.ColMeta, f func(ctx context.Context, w *StreamWriter) error) (flux.Table, error)

Stream will call StreamWithContext with a background context.

func StreamWithContext

func StreamWithContext(ctx context.Context, key flux.GroupKey, cols []flux.ColMeta, f func(ctx context.Context, w *StreamWriter) error) (flux.Table, error)

StreamWithContext will create a table that streams column readers through the flux.Table. This method will return only after the function buffers the first column reader. This first column reader is used to identify the group key and columns for the entire table stream.

Implementors using this *must* return at least one table. If the function returns without returning at least one table, then an error will be returned. If the first table that is returned is empty, then this will return an empty table and further buffers will not be used.

func Values

func Values(cr flux.ColReader, j int) array.Interface

Values returns the array from the column reader as an array.Interface.

Types

type ArrowBuilder

type ArrowBuilder struct {
	GroupKey  flux.GroupKey
	Columns   []flux.ColMeta
	Builders  []array.Builder
	Allocator memory.Allocator
}

ArrowBuilder is a Builder that uses arrow array builders as the underlying builder mechanism.

func GetArrowBuilder

func GetArrowBuilder(key flux.GroupKey, cache *BuilderCache) (builder *ArrowBuilder, created bool)

GetArrowBuilder is a convenience method for retrieving an ArrowBuilder from the BuilderCache.

func NewArrowBuilder

func NewArrowBuilder(key flux.GroupKey, mem memory.Allocator) *ArrowBuilder

NewArrowBuilder constructs a new ArrowBuilder.

func (*ArrowBuilder) AddCol

func (a *ArrowBuilder) AddCol(c flux.ColMeta) (int, error)

AddCol will add a column with the given metadata. If the column exists, an error is returned.

func (*ArrowBuilder) CheckCol

func (a *ArrowBuilder) CheckCol(c flux.ColMeta) (int, error)

CheckCol will check if a column exists with the label and the same type. This will return an error if the column does not exist or has an incompatible type.

func (*ArrowBuilder) Cols

func (a *ArrowBuilder) Cols() []flux.ColMeta

func (*ArrowBuilder) Key

func (a *ArrowBuilder) Key() flux.GroupKey

func (*ArrowBuilder) Release

func (a *ArrowBuilder) Release()

func (*ArrowBuilder) Table

func (a *ArrowBuilder) Table() (flux.Table, error)

Table constructs a flux.Table from the current builders.

type BufferedBuilder

type BufferedBuilder struct {
	GroupKey  flux.GroupKey
	Columns   []flux.ColMeta
	Buffers   []*arrow.TableBuffer
	Allocator memory.Allocator
}

BufferedBuilder is a table builder that constructs a BufferedTable with zero or more buffers.

func GetBufferedBuilder

func GetBufferedBuilder(key flux.GroupKey, cache *BuilderCache) (builder *BufferedBuilder, created bool)

GetBufferedBuilder is a convenience method for retrieving a BufferedBuilder from the BuilderCache.

func NewBufferedBuilder

func NewBufferedBuilder(key flux.GroupKey, mem memory.Allocator) *BufferedBuilder

NewBufferedBuilder constructs a new BufferedBuilder.

func (*BufferedBuilder) AppendBuffer

func (b *BufferedBuilder) AppendBuffer(cr flux.ColReader) error

AppendBuffer will append a new buffer to this table builder. It ensures the schemas are compatible and will backfill previous buffers with nil for new columns that didn't previously exist.

func (*BufferedBuilder) Release

func (b *BufferedBuilder) Release()

func (*BufferedBuilder) Table

func (b *BufferedBuilder) Table() (flux.Table, error)

type BufferedTable

type BufferedTable struct {
	GroupKey flux.GroupKey
	Columns  []flux.ColMeta
	Buffers  []flux.ColReader
	// contains filtered or unexported fields
}

BufferedTable represents a table of buffered column readers.

func (*BufferedTable) Cols

func (b *BufferedTable) Cols() []flux.ColMeta

func (*BufferedTable) Do

func (b *BufferedTable) Do(f func(flux.ColReader) error) error

func (*BufferedTable) Done

func (b *BufferedTable) Done()

func (*BufferedTable) Empty

func (b *BufferedTable) Empty() bool

func (*BufferedTable) Key

func (b *BufferedTable) Key() flux.GroupKey

type Builder

type Builder interface {
	// Table will construct a Table from the existing contents.
	// Invoking this method should reset the builder and all allocated
	// memory will be owned by the returned flux.Table.
	Table() (flux.Table, error)

	// Release will release the buffered contents from the builder.
	// This method is unnecessary if Table is called.
	Release()
}

Builder is the minimum interface for constructing a Table.

type BuilderCache

type BuilderCache struct {
	// New will be called to construct a new Builder
	// when a GroupKey that hasn't been seen before is
	// requested. The returned Builder should be empty.
	New func(key flux.GroupKey) Builder
	// contains filtered or unexported fields
}

BuilderCache hold a mapping of group keys to Builder. When a Builder is requested for a specific group key, the BuilderCache will return a Builder that is unique for that GroupKey.

func (*BuilderCache) DiscardTable

func (d *BuilderCache) DiscardTable(key flux.GroupKey)

func (*BuilderCache) ExpireTable

func (d *BuilderCache) ExpireTable(key flux.GroupKey)

func (*BuilderCache) ForEach

func (d *BuilderCache) ForEach(f func(key flux.GroupKey, builder Builder) error) error

func (*BuilderCache) Get

func (d *BuilderCache) Get(key flux.GroupKey, b interface{}) bool

Get retrieves the Builder for this group key. If one doesn't exist, it will invoke the New function and store it within the Builder. If the builder was newly created, this method returns true for the second parameter. The interface must be a pointer to the type that is created from the New method. This method will use reflection to set the value of the pointer.

func (*BuilderCache) Table

func (d *BuilderCache) Table(key flux.GroupKey) (flux.Table, error)

Table will remove a builder from the cache and construct a flux.Table from the buffered contents.

type SendFunc

type SendFunc func(flux.ColReader)

SendFunc is used to send a flux.ColReader to a table stream so it can be read by the table consumer.

type StreamWriter

type StreamWriter struct {
	// contains filtered or unexported fields
}

StreamWriter is the input end of a stream.

func (*StreamWriter) Cols

func (s *StreamWriter) Cols() []flux.ColMeta

func (*StreamWriter) Key

func (s *StreamWriter) Key() flux.GroupKey

func (*StreamWriter) UnsafeWrite

func (s *StreamWriter) UnsafeWrite(vs []array.Interface) error

UnsafeWrite will write the new buffer to the stream without validating that the resulting table is valid. This can be used to avoid the small performance hit that comes from validating the resulting table.

func (*StreamWriter) UnsafeWriteBuffer

func (s *StreamWriter) UnsafeWriteBuffer(cr flux.ColReader) error

UnsafeWriteBuffer will emit the given column reader to the stream. This does not validate that the column reader matches with the stream schema.

func (*StreamWriter) Write

func (s *StreamWriter) Write(vs []array.Interface) error

Write will write a new buffer to the stream using the given values. The group key and columns will be used for the emitted column reader.