arrow

README

Arrow Processor

To use the Arrow Processor use this data config:

data:
  processor:
    name: arrow

Params

Name	Supported Values	Description
time_selector	A top-level field name	Specifies the field to use for time

Example Dataspace Config

Paired with the Apache Arrow Flight Data Connector.

dataspaces:
  - from: spice
    name: sql
    identifiers:
      - name: number
    measurements:
      - name: gas_used
    data:
      connector:
        name: flight
        params:
          password: <api_key>
          sql: SELECT number, timestamp, gas_used FROM eth.recent_blocks ORDER BY number DESC
      processor:
        name: arrow
        params:
          time_selector: timestamp

Documentation

Index

• Constants
• type ArrowProcessor
  • func NewArrowProcessor() *ArrowProcessor
  • func (p *ArrowProcessor) GetRecord() (apache_arrow.Record, error)
  • func (p *ArrowProcessor) Init(params map[string]string, identifiers map[string]string, ...) error
  • func (p *ArrowProcessor) OnData(data []byte) ([]byte, error)
• type FieldInfo

Constants

const (
	ArrowProcessorName string = "arrow"
)

Types

type ArrowProcessor

type ArrowProcessor struct {
	// contains filtered or unexported fields
}

func NewArrowProcessor

func NewArrowProcessor() *ArrowProcessor

func (*ArrowProcessor) GetRecord

func (p *ArrowProcessor) GetRecord() (apache_arrow.Record, error)

func (*ArrowProcessor) Init

func (p *ArrowProcessor) Init(params map[string]string, identifiers map[string]string, measurements map[string]string, categories map[string]string, tags []string) error

func (*ArrowProcessor) OnData

func (p *ArrowProcessor) OnData(data []byte) ([]byte, error)

type FieldInfo

type FieldInfo struct {
	Index int
	Field apache_arrow.Field
}