table

Documentation

Index

• Constants
• func Copy(t flux.Table) (flux.BufferedTable, error)
• func Diff(want, got flux.TableIterator, opts ...DiffOption) string
• func Mask(tbl flux.Table, columns []string) flux.Table
• func Sort(tables flux.TableIterator) (flux.TableIterator, error)
• func Stream(key flux.GroupKey, cols []flux.ColMeta, ...) (flux.Table, error)
• func StreamWithContext(ctx context.Context, key flux.GroupKey, cols []flux.ColMeta, ...) (flux.Table, error)
• func Stringify(t flux.Table) string
• func Values(cr flux.ColReader, j int) array.Array
• type ArrowBuilder
  • func GetArrowBuilder(key flux.GroupKey, cache *table.BuilderCache) (builder *ArrowBuilder, created bool)
  • func NewArrowBuilder(key flux.GroupKey, mem memory.Allocator) *ArrowBuilder
  • func (a *ArrowBuilder) AddCol(c flux.ColMeta) (int, error)
  • func (a *ArrowBuilder) Buffer() (arrow.TableBuffer, error)
  • func (a *ArrowBuilder) CheckCol(c flux.ColMeta) (int, error)
  • func (a *ArrowBuilder) Cols() []flux.ColMeta
  • func (a *ArrowBuilder) Init(cols []flux.ColMeta)
  • func (a *ArrowBuilder) Key() flux.GroupKey
  • func (a *ArrowBuilder) Release()
  • func (a *ArrowBuilder) Reserve(n int)
  • func (a *ArrowBuilder) Resize(n int)
  • func (a *ArrowBuilder) Table() (flux.Table, error)
• type BufferedBuilder
  • func GetBufferedBuilder(key flux.GroupKey, cache *BuilderCache) (builder *BufferedBuilder, created bool)
  • func NewBufferedBuilder(key flux.GroupKey, mem memory.Allocator) *BufferedBuilder
• type Builder
• type BuilderCache
• type Chunk
  • func ChunkFromBuffer(buf arrow.TableBuffer) Chunk
  • func ChunkFromReader(cr flux.ColReader) Chunk
• type DiffOption
  • func DiffContext(n int) DiffOption
• type Iterator
• type SendFunc
• type StreamWriter
  • func (s *StreamWriter) Cols() []flux.ColMeta
  • func (s *StreamWriter) Key() flux.GroupKey
  • func (s *StreamWriter) UnsafeWrite(vs []array.Array) error
  • func (s *StreamWriter) UnsafeWriteBuffer(cr flux.ColReader) error
  • func (s *StreamWriter) Write(vs []array.Array) error

Constants

const BufferSize = table.BufferSize

Functions

func Copy

func Copy(t flux.Table) (flux.BufferedTable, error)

Copy returns a buffered copy of the table and consumes the input table. If the input table is already buffered, it "consumes" the input and returns the same table.

The buffered table can then be copied additional times using the BufferedTable.Copy method.

This method should be used sparingly if at all. It will retain each of the buffers of data coming out of a table so the entire table is materialized in memory. For large datasets, this could potentially cause a problem. The allocator is meant to catch when this happens and prevent it.

func Diff

func Diff(want, got flux.TableIterator, opts ...DiffOption) string

Diff will perform a diff between two table iterators. This will sort the tables within the table iterators and produce a diff of the full output.

func Mask

func Mask(tbl flux.Table, columns []string) flux.Table

Mask will return a no-copy Table that masks the given columns. If the columns are part of the group key, they will be removed from the key.

This function will not attempt any regrouping with other tables. This function should only be used when it is known that the group key will not conflict with others and the Table needs to have certain columns filtered either for display or other purposes.

func Sort

func Sort(tables flux.TableIterator) (flux.TableIterator, error)

Sort will read a TableIterator and produce another TableIterator where the keys are sorted.

This method will buffer all of the data since it needs to ensure all of the tables are read to avoid any deadlocks. Be careful using this method in performance sensitive areas.

func Stream

func Stream(key flux.GroupKey, cols []flux.ColMeta, f func(ctx context.Context, w *StreamWriter) error) (flux.Table, error)

Stream will call StreamWithContext with a background context.

func StreamWithContext

func StreamWithContext(ctx context.Context, key flux.GroupKey, cols []flux.ColMeta, f func(ctx context.Context, w *StreamWriter) error) (flux.Table, error)

StreamWithContext will create a table that streams column readers through the flux.Table. This method will return only after the function buffers the first column reader. This first column reader is used to identify the group key and columns for the entire table stream.

Implementors using this *must* return at least one table. If the function returns without returning at least one table, then an error will be returned. If the first table that is returned is empty, then this will return an empty table and further buffers will not be used.

func Stringify

func Stringify(t flux.Table) string

Stringify will read a table and turn it into a human-readable string.

func Values

func Values(cr flux.ColReader, j int) array.Array

Types

type ArrowBuilder

type ArrowBuilder struct {
	GroupKey  flux.GroupKey
	Columns   []flux.ColMeta
	Builders  []array.Builder
	Allocator memory.Allocator
}

ArrowBuilder is a Builder that uses arrow array builders as the underlying builder mechanism.

func GetArrowBuilder

func GetArrowBuilder(key flux.GroupKey, cache *table.BuilderCache) (builder *ArrowBuilder, created bool)

GetArrowBuilder is a convenience method for retrieving an ArrowBuilder from the BuilderCache.

func NewArrowBuilder

func NewArrowBuilder(key flux.GroupKey, mem memory.Allocator) *ArrowBuilder

NewArrowBuilder constructs a new ArrowBuilder.

func (*ArrowBuilder) AddCol

func (a *ArrowBuilder) AddCol(c flux.ColMeta) (int, error)

AddCol will add a column with the given metadata. If the column exists, an error is returned.

func (*ArrowBuilder) Buffer

func (a *ArrowBuilder) Buffer() (arrow.TableBuffer, error)

Buffer constructs an arrow.TableBuffer from the current builders.

func (*ArrowBuilder) CheckCol

func (a *ArrowBuilder) CheckCol(c flux.ColMeta) (int, error)

CheckCol will check if a column exists with the label and the same type. This will return an error if the column does not exist or has an incompatible type.

func (*ArrowBuilder) Cols

func (a *ArrowBuilder) Cols() []flux.ColMeta

func (*ArrowBuilder) Init

func (a *ArrowBuilder) Init(cols []flux.ColMeta)

Init will initialize the builders for this ArrowBuilder with the given columns. This can be used as an alternative to multiple calls to AddCol.

func (*ArrowBuilder) Key

func (a *ArrowBuilder) Key() flux.GroupKey

func (*ArrowBuilder) Release

func (a *ArrowBuilder) Release()

func (*ArrowBuilder) Reserve

func (a *ArrowBuilder) Reserve(n int)

Reserve calls Reserve on each of the builders in this builder.

func (*ArrowBuilder) Resize

func (a *ArrowBuilder) Resize(n int)

Resize calls Resize on each of the builders in this builder.

func (*ArrowBuilder) Table

func (a *ArrowBuilder) Table() (flux.Table, error)

Table constructs a flux.Table from the current builders.

type BufferedBuilder

type BufferedBuilder = table.BufferedBuilder

func GetBufferedBuilder

func GetBufferedBuilder(key flux.GroupKey, cache *BuilderCache) (builder *BufferedBuilder, created bool)

func NewBufferedBuilder

func NewBufferedBuilder(key flux.GroupKey, mem memory.Allocator) *BufferedBuilder

type Builder

type Builder = table.Builder

type BuilderCache

type BuilderCache = table.BuilderCache

type Chunk

type Chunk = table.Chunk

func ChunkFromBuffer

func ChunkFromBuffer(buf arrow.TableBuffer) Chunk

func ChunkFromReader

func ChunkFromReader(cr flux.ColReader) Chunk

type DiffOption

type DiffOption = table.DiffOption

func DiffContext

func DiffContext(n int) DiffOption

type Iterator

type Iterator = table.Iterator

type SendFunc

type SendFunc func(flux.ColReader)

SendFunc is used to send a flux.ColReader to a table stream so it can be read by the table consumer.

type StreamWriter

type StreamWriter struct {
	// contains filtered or unexported fields
}

StreamWriter is the input end of a stream.

func (*StreamWriter) Cols

func (s *StreamWriter) Cols() []flux.ColMeta

func (*StreamWriter) Key

func (s *StreamWriter) Key() flux.GroupKey

func (*StreamWriter) UnsafeWrite

func (s *StreamWriter) UnsafeWrite(vs []array.Array) error

UnsafeWrite will write the new buffer to the stream without validating that the resulting table is valid. This can be used to avoid the small performance hit that comes from validating the resulting table.

func (*StreamWriter) UnsafeWriteBuffer

func (s *StreamWriter) UnsafeWriteBuffer(cr flux.ColReader) error

UnsafeWriteBuffer will emit the given column reader to the stream. This does not validate that the column reader matches with the stream schema.

func (*StreamWriter) Write

func (s *StreamWriter) Write(vs []array.Array) error

Write will write a new buffer to the stream using the given values. The group key and columns will be used for the emitted column reader.