table

package
v0.108.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 15, 2021 License: MIT Imports: 12 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Copy added in v0.103.0 

func Copy(t flux.Table) (flux.BufferedTable, error)

Copy returns a buffered copy of the table and consumes the input table. If the input table is already buffered, it "consumes" the input and returns the same table.

The buffered table can then be copied additional times using the BufferedTable.Copy method.

This method should be used sparingly if at all. It will retain each of the buffers of data coming out of a table so the entire table is materialized in memory. For large datasets, this could potentially cause a problem. The allocator is meant to catch when this happens and prevent it.

func Diff added in v0.72.0 

func Diff(want, got flux.TableIterator, opts ...DiffOption) string

Diff will perform a diff between two table iterators. This will sort the tables within the table iterators and produce a diff of the full output.

func FromBuffer 

func FromBuffer(cr flux.ColReader) flux.Table

FromBuffer constructs a flux.Table from a single flux.ColReader.

func Mask added in v0.77.1 

func Mask(tbl flux.Table, columns []string) flux.Table

Mask will return a no-copy Table that masks the given columns. If the columns are part of the group key, they will be removed from the key.

This function will not attempt any regrouping with other tables. This function should only be used when it is known that the group key will not conflict with others and the Table needs to have certain columns filtered either for display or other purposes.

func Sort added in v0.74.0 

func Sort(tables flux.TableIterator) (flux.TableIterator, error)

Sort will read a TableIterator and produce another TableIterator where the keys are sorted.

This method will buffer all of the data since it needs to ensure all of the tables are read to avoid any deadlocks. Be careful using this method in performance sensitive areas.

func Stream 

func Stream(key flux.GroupKey, cols []flux.ColMeta, f func(ctx context.Context, w *StreamWriter) error) (flux.Table, error)

Stream will call StreamWithContext with a background context.

func StreamWithContext 

func StreamWithContext(ctx context.Context, key flux.GroupKey, cols []flux.ColMeta, f func(ctx context.Context, w *StreamWriter) error) (flux.Table, error)

StreamWithContext will create a table that streams column readers through the flux.Table. This method will return only after the function buffers the first column reader. This first column reader is used to identify the group key and columns for the entire table stream.

Implementors using this *must* return at least one table. If the function returns without returning at least one table, then an error will be returned. If the first table that is returned is empty, then this will return an empty table and further buffers will not be used.

func Stringify added in v0.74.0 

func Stringify(t flux.Table) string

Stringify will read a table and turn it into a human-readable string.

func Values 

func Values(cr flux.ColReader, j int) array.Interface

Values returns the array from the column reader as an array.Interface.

Types

type ArrowBuilder 

type ArrowBuilder struct {
	GroupKey  flux.GroupKey
	Columns   []flux.ColMeta
	Builders  []array.Builder
	Allocator memory.Allocator
}

ArrowBuilder is a Builder that uses arrow array builders as the underlying builder mechanism.

func GetArrowBuilder 

func GetArrowBuilder(key flux.GroupKey, cache *BuilderCache) (builder *ArrowBuilder, created bool)

GetArrowBuilder is a convenience method for retrieving an ArrowBuilder from the BuilderCache.

func NewArrowBuilder 

func NewArrowBuilder(key flux.GroupKey, mem memory.Allocator) *ArrowBuilder

NewArrowBuilder constructs a new ArrowBuilder.

func (*ArrowBuilder) AddCol 

func (a *ArrowBuilder) AddCol(c flux.ColMeta) (int, error)

AddCol will add a column with the given metadata. If the column exists, an error is returned.

func (*ArrowBuilder) Buffer added in v0.103.0 

func (a *ArrowBuilder) Buffer() (arrow.TableBuffer, error)

Buffer constructs an arrow.TableBuffer from the current builders.

func (*ArrowBuilder) CheckCol 

func (a *ArrowBuilder) CheckCol(c flux.ColMeta) (int, error)

CheckCol will check if a column exists with the label and the same type. This will return an error if the column does not exist or has an incompatible type.

func (*ArrowBuilder) Cols 

func (a *ArrowBuilder) Cols() []flux.ColMeta

func (*ArrowBuilder) Key 

func (a *ArrowBuilder) Key() flux.GroupKey

func (*ArrowBuilder) Release 

func (a *ArrowBuilder) Release()

func (*ArrowBuilder) Table 

func (a *ArrowBuilder) Table() (flux.Table, error)

Table constructs a flux.Table from the current builders.

type BufferedBuilder 

type BufferedBuilder struct {
	GroupKey  flux.GroupKey
	Columns   []flux.ColMeta
	Buffers   []*arrow.TableBuffer
	Allocator memory.Allocator
}

BufferedBuilder is a table builder that constructs a BufferedTable with zero or more buffers.

func GetBufferedBuilder 

func GetBufferedBuilder(key flux.GroupKey, cache *BuilderCache) (builder *BufferedBuilder, created bool)

GetBufferedBuilder is a convenience method for retrieving a BufferedBuilder from the BuilderCache.

func NewBufferedBuilder 

func NewBufferedBuilder(key flux.GroupKey, mem memory.Allocator) *BufferedBuilder

NewBufferedBuilder constructs a new BufferedBuilder.

func (*BufferedBuilder) AppendBuffer 

func (b *BufferedBuilder) AppendBuffer(cr flux.ColReader) error

AppendBuffer will append a new buffer to this table builder. It ensures the schemas are compatible and will backfill previous buffers with nil for new columns that didn't previously exist.

func (*BufferedBuilder) AppendTable added in v0.72.0 

func (b *BufferedBuilder) AppendTable(tbl flux.Table) error

AppendTable will append all of the table buffers inside of a table to this BufferedBuilder.

This method will take care of normalizing the schema in the case where there is an empty table with no buffers.

func (*BufferedBuilder) Release 

func (b *BufferedBuilder) Release()

func (*BufferedBuilder) Table 

func (b *BufferedBuilder) Table() (flux.Table, error)

type BufferedTable 

type BufferedTable struct {
	GroupKey flux.GroupKey
	Columns  []flux.ColMeta
	Buffers  []flux.ColReader
	// contains filtered or unexported fields
}

BufferedTable represents a table of buffered column readers.

func (*BufferedTable) Cols 

func (b *BufferedTable) Cols() []flux.ColMeta

func (*BufferedTable) Do 

func (b *BufferedTable) Do(f func(flux.ColReader) error) error

func (*BufferedTable) Done 

func (b *BufferedTable) Done()

func (*BufferedTable) Empty 

func (b *BufferedTable) Empty() bool

func (*BufferedTable) Key 

func (b *BufferedTable) Key() flux.GroupKey

type Builder 

type Builder interface {
	// Table will construct a Table from the existing contents.
	// Invoking this method should reset the builder and all allocated
	// memory will be owned by the returned flux.Table.
	Table() (flux.Table, error)

	// Release will release the buffered contents from the builder.
	// This method is unnecessary if Table is called.
	Release()
}

Builder is the minimum interface for constructing a Table.

type BuilderCache 

type BuilderCache struct {
	// New will be called to construct a new Builder
	// when a GroupKey that hasn't been seen before is
	// requested. The returned Builder should be empty.
	New func(key flux.GroupKey) Builder

	// Tables contains the cached builders.
	// This can be set before use to customize the
	// method for storing data. If this is null,
	// the default execute.GroupLookup is initialized
	// when the cache is first used.
	Tables KeyLookup
}

BuilderCache hold a mapping of group keys to Builder. When a Builder is requested for a specific group key, the BuilderCache will return a Builder that is unique for that GroupKey.

func (*BuilderCache) DiscardTable 

func (d *BuilderCache) DiscardTable(key flux.GroupKey)

func (*BuilderCache) ExpireTable 

func (d *BuilderCache) ExpireTable(key flux.GroupKey)

func (*BuilderCache) ForEach 

func (d *BuilderCache) ForEach(f func(key flux.GroupKey, builder Builder) error) error

func (*BuilderCache) Get 

func (d *BuilderCache) Get(key flux.GroupKey, b interface{}) bool

Get retrieves the Builder for this group key. If one doesn't exist, it will invoke the New function and store it within the Builder. If the builder was newly created, this method returns true for the second parameter. The interface must be a pointer to the type that is created from the New method. This method will use reflection to set the value of the pointer.

func (*BuilderCache) Table 

func (d *BuilderCache) Table(key flux.GroupKey) (flux.Table, error)

Table will remove a builder from the cache and construct a flux.Table from the buffered contents.

type DiffOption added in v0.74.0 

type DiffOption = table.DiffOption

func DiffContext added in v0.74.0 

func DiffContext(n int) DiffOption

type Iterator added in v0.74.0 

type Iterator = table.Iterator

type KeyLookup 

type KeyLookup interface {
	// Lookup will retrieve the value associated with the given key if it exists.
	Lookup(key flux.GroupKey) (interface{}, bool)

	// LookupOrCreate will retrieve the value associated with the given key or,
	// if it does not exist, will invoke the function to create one and set
	// it in the group lookup.
	LookupOrCreate(key flux.GroupKey, fn func() interface{}) interface{}

	// Set will set the value for the given key.
	// It will overwrite an existing value.
	Set(key flux.GroupKey, value interface{})

	// Delete will remove the key from this KeyLookup.
	// It will return the same thing as a call to Lookup.
	Delete(key flux.GroupKey) (v interface{}, found bool)

	// Range will iterate over all groups keys in a stable ordering.
	// Range must not be called within another call to Range.
	// It is safe to call Set/Delete while ranging.
	Range(f func(key flux.GroupKey, value interface{}))

	// Clear will clear the lookup and reset it to contain nothing.
	Clear()
}

KeyLookup is an interface for storing and retrieving items by their group key.

type SendFunc 

type SendFunc func(flux.ColReader)

SendFunc is used to send a flux.ColReader to a table stream so it can be read by the table consumer.

type StreamWriter 

type StreamWriter struct {
	// contains filtered or unexported fields
}

StreamWriter is the input end of a stream.

func (*StreamWriter) Cols 

func (s *StreamWriter) Cols() []flux.ColMeta

func (*StreamWriter) Key 

func (s *StreamWriter) Key() flux.GroupKey

func (*StreamWriter) UnsafeWrite 

func (s *StreamWriter) UnsafeWrite(vs []array.Interface) error

UnsafeWrite will write the new buffer to the stream without validating that the resulting table is valid. This can be used to avoid the small performance hit that comes from validating the resulting table.

func (*StreamWriter) UnsafeWriteBuffer 

func (s *StreamWriter) UnsafeWriteBuffer(cr flux.ColReader) error

UnsafeWriteBuffer will emit the given column reader to the stream. This does not validate that the column reader matches with the stream schema.

func (*StreamWriter) Write 

func (s *StreamWriter) Write(vs []array.Interface) error

Write will write a new buffer to the stream using the given values. The group key and columns will be used for the emitted column reader.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.