bqext

README

The bqext package

This package contains extensions for the bigquery library, to facilitate various common operations, notably query processing.

Documentation

Overview

Package bqext includes generally useful abstractions for simplifying interactions with bigquery. Production utilities should go here, but test facilities should go in a separate bqtest package. TODO - rename bqext

Index

• type Table
  • func NewTable(project, dataset string, clientOpts ...option.ClientOption) (Table, error)
  • func (util *Table) QueryAndParse(q string, structPtr interface{}) error
  • func (util *Table) ResultQuery(query string, dryRun bool) *bigquery.Query

Types

type Table

type Table struct {
	BqClient *bigquery.Client
	Dataset  *bigquery.Dataset
}

Table provides extensions to the bigquery Dataset and Table objects to streamline common actions. It encapsulates the Client and Dataset to simplify methods. TODO(gfr) Should this be called DatasetExt ?

func NewTable

func NewTable(project, dataset string, clientOpts ...option.ClientOption) (Table, error)

NewTable creates a Table for a project. httpClient is used to inject mocks for the bigquery client. if httpClient is nil, a suitable default client is used. Additional bigquery ClientOptions may be optionally passed as final

clientOpts argument.  This is useful for testing credentials.

func (*Table) QueryAndParse

func (util *Table) QueryAndParse(q string, structPtr interface{}) error

QueryAndParse executes a query that should return a single row, with all struct fields that match query columns filled in. The caller must pass in the *address* of an appropriate struct. TODO - extend this to also handle multirow results, by passing slice of structs.

func (*Table) ResultQuery

func (util *Table) ResultQuery(query string, dryRun bool) *bigquery.Query

ResultQuery constructs a query with common QueryConfig settings for writing results to a table. Generally, may need to change WriteDisposition.