batch

Documentation

Index

• func SliceRecord(r arrow.Record, limit *Cap) (add *SlicedRecord, flush []arrow.Record, remaining *SlicedRecord)
• type Cap
  • func CappedAt(bytes, rows int64) *Cap
  • func (c *Cap) AddRows(rows int64)
  • func (c *Cap) AddSlice(record *SlicedRecord)
  • func (c *Cap) ReachedLimit() bool
  • func (c *Cap) Reset()
  • func (c *Cap) Rows() int64
• type SlicedRecord

Functions

func SliceRecord

func SliceRecord(r arrow.Record, limit *Cap) (add *SlicedRecord, flush []arrow.Record, remaining *SlicedRecord)

SliceRecord will return the SlicedRecord you can add to the batch given the restrictions provided (if any). The meaning of the returned values: - `add` is good to be added to the current batch that the caller is assembling - `flush` represents sliced arrow.Record that needs own batch to be flushed - `remaining` represents the overflow of the batch after `add` & `flush` are processed Note that the `limit` provided will not be updated.

Types

type Cap

type Cap struct {
	// contains filtered or unexported fields
}

func CappedAt

func CappedAt(bytes, rows int64) *Cap

func (*Cap) AddRows

func (c *Cap) AddRows(rows int64)

func (*Cap) AddSlice

func (c *Cap) AddSlice(record *SlicedRecord)

func (*Cap) ReachedLimit

func (c *Cap) ReachedLimit() bool

func (*Cap) Reset

func (c *Cap) Reset()

func (*Cap) Rows

func (c *Cap) Rows() int64

type SlicedRecord

type SlicedRecord struct {
	arrow.Record
	Bytes int64 // we need this as the util.TotalRecordSize will report the full size even for the sliced record
	// contains filtered or unexported fields
}