bq

Documentation

Index

• type BatchInserter
  • func (b *BatchInserter) Run(rowCh <-chan interface{})
• type Table
  • func NewTableForStruct(project string, datasetName string, datasetLoc string, tableName string, ...) (*Table, error)
  • func (t *Table) Inserter() *bigquery.Inserter

Types

type BatchInserter

type BatchInserter struct {
	// BatchSize specifies how many rows (struct savers) to wait for before pushing a batch.
	BatchSize int
	// PushTimeout specifies how long to wait before pushing a batch even if the batch isn't at BatchSize yet.
	PushTimeout time.Duration
	Table       *Table
	// contains filtered or unexported fields
}

BatchInserter batches up bigquery struct savers, to reduce the number of calls to inserter.Put.

func (*BatchInserter) Run

func (b *BatchInserter) Run(rowCh <-chan interface{})

Run starts running the BatchInserter.

type Table

type Table struct {
	Schema bigquery.Schema
	Table  *bigquery.Table
}

Table is a wrapper around bigquery's Table which also stores the bigquery Schema. Storing the bigquery.Schema with the bigquery.Table is useful because the Schema is required for some bigquery operations (for example, bigquery.StructSaver based puts).

func NewTableForStruct

func NewTableForStruct(project string, datasetName string, datasetLoc string, tableName string, timePartitioning *bigquery.TimePartitioning, s interface{}) (*Table, error)

NewTableForStruct returns a new bigquery Table wrapper storing rows of type `s`, creating the bigquery table and dataset if necessary.

func (*Table) Inserter

func (t *Table) Inserter() *bigquery.Inserter

Inserter returns the underlying bigquery.Table's Inserter.