arrow

Documentation

Index

• type ArrowRowBuilder
  • func NewArrowRowBuilder(aw *ArrowWriter, chunkSize int) (*ArrowRowBuilder, error)
  • func (arb *ArrowRowBuilder) Release() error
  • func (arb *ArrowRowBuilder) WriteRow(row []any) error
• type ArrowWriter
  • func NewArrowIPCFileWriter(f *os.File, fieldNames []string, fieldTypes []arrow.DataType, nullable bool, ...) (*ArrowWriter, error)
  • func NewArrowIPCFileWriterWithSchema(f *os.File, schema *arrow.Schema, options ...ipc.Option) (*ArrowWriter, error)
  • func (aw *ArrowWriter) Close() error
  • func (aw *ArrowWriter) WriteChunk(record arrow.Record) error

Types

type ArrowRowBuilder

type ArrowRowBuilder struct {
	// contains filtered or unexported fields
}

func NewArrowRowBuilder

func NewArrowRowBuilder(aw *ArrowWriter, chunkSize int) (*ArrowRowBuilder, error)

NewArrowRowBuilder creates a new ArrowRowBuilder. The ArrowRowBuilder will write to the underlying ArrowWriter. ArrowRowBuilder is not threadsafe: it should only be used by one thread at a time, though multiple ArrowRowBuilders writing to a single ArrowWriter concurrently is possible This is done to enable fast, parallel accumulation of rows, delaying synchronization until enough rows are accumulated to write a chunk.

func (*ArrowRowBuilder) Release

func (arb *ArrowRowBuilder) Release() error

Release releases the ArrowRowBuilder. This must be called to ensure that all data is successfully written to the file. Release will write any remaining rows to the file, and so must be called before Close() on the underlying ArrowWriter.

func (*ArrowRowBuilder) WriteRow

func (arb *ArrowRowBuilder) WriteRow(row []any) error

WriteRow writes a row to the ArrowRowBuilder. The number of fields in the row must match the number of fields in the schema.

type ArrowWriter

type ArrowWriter struct {
	Schema *arrow.Schema
	// contains filtered or unexported fields
}

func NewArrowIPCFileWriter

func NewArrowIPCFileWriter(f *os.File, fieldNames []string, fieldTypes []arrow.DataType, nullable bool, options ...ipc.Option) (*ArrowWriter, error)

Create a new ArrowWriter. The number of fields in fieldNames and fieldTypes must match. The number of rows in each chunk is determined by chunkSize. The ArrowWriter will write to filePath. This writing operation is threadsafe.

func NewArrowIPCFileWriterWithSchema

func NewArrowIPCFileWriterWithSchema(f *os.File, schema *arrow.Schema, options ...ipc.Option) (*ArrowWriter, error)

func (*ArrowWriter) Close

func (aw *ArrowWriter) Close() error

Close closes the ArrowWriter. This must be called to ensure that all data is successfully written to the file.

func (*ArrowWriter) WriteChunk

func (aw *ArrowWriter) WriteChunk(record arrow.Record) error