bigqueryio

Documentation

Overview

Package bigqueryio provides transformations and utilities to interact with Google BigQuery. See also: https://cloud.google.com/bigquery/docs.

Index

• func Query(s beam.Scope, project, q string, t reflect.Type, ...) beam.PCollection
• func Read(s beam.Scope, project, table string, t reflect.Type) beam.PCollection
• func UseStandardSQL() func(qo *QueryOptions) error
• func Write(s beam.Scope, project, table string, col beam.PCollection)
• type QualifiedTableName
  • func NewQualifiedTableName(s string) (QualifiedTableName, error)
  • func (qn QualifiedTableName) String() string
• type QueryOptions

Functions

func Query

func Query(s beam.Scope, project, q string, t reflect.Type, options ...func(*QueryOptions) error) beam.PCollection

Query executes a query. The output must have a schema compatible with the given type, t. It returns a PCollection<t>.

func Read

func Read(s beam.Scope, project, table string, t reflect.Type) beam.PCollection

Read reads all rows from the given table. The table must have a schema compatible with the given type, t, and Read returns a PCollection<t>. If the table has more rows than t, then Read is implicitly a projection.

func UseStandardSQL

func UseStandardSQL() func(qo *QueryOptions) error

UseStandardSQL enables BigQuery's Standard SQL dialect when executing a query.

func Write

func Write(s beam.Scope, project, table string, col beam.PCollection)

Write writes the elements of the given PCollection<T> to bigquery. T is required to be the schema type.

Types

type QualifiedTableName

type QualifiedTableName struct {
	// Project is the Google Cloud project ID.
	Project string `json:"project"`
	// Dataset is the dataset ID within the project.
	Dataset string `json:"dataset"`
	// Table is the table ID within the dataset.
	Table string `json:"table"`
}

QualifiedTableName is a fully qualified name of a bigquery table.

func NewQualifiedTableName

func NewQualifiedTableName(s string) (QualifiedTableName, error)

NewQualifiedTableName parses "<project>:<dataset>.<table>" into a QualifiedTableName.

func (QualifiedTableName) String

func (qn QualifiedTableName) String() string

String formats the qualified name as "<project>:<dataset>.<table>".

type QueryOptions

type QueryOptions struct {
	// UseStandardSQL enables BigQuery's Standard SQL dialect when executing a query.
	UseStandardSQL bool
}

QueryOptions represents additional options for executing a query.