bqext

README

The bqext package

This package contains extensions for the bigquery library, to facilitate various common operations, notably query processing.

Documentation

Overview

Package bqext includes generally useful abstractions for simplifying interactions with bigquery. Production extensions should go here, but test facilities should go in a separate bqtest package.

Index

• type Dataset
  • func NewDataset(project, dataset string, clientOpts ...option.ClientOption) (Dataset, error)
  • func (dsExt *Dataset) DestQuery(query string, dest *bigquery.Table, disposition bigquery.TableWriteDisposition) *bigquery.Query
  • func (dsExt *Dataset) ExecDestQuery(q *bigquery.Query) (*bigquery.JobStatus, error)
  • func (dsExt Dataset) GetPartitionInfo(table string, partition string) (PartitionInfo, error)
  • func (dsExt *Dataset) QueryAndParse(q string, structPtr interface{}) error
  • func (dsExt *Dataset) ResultQuery(query string, dryRun bool) *bigquery.Query
• type PartitionInfo

Types

type Dataset

type Dataset struct {
	*bigquery.Dataset // Exposes Dataset API directly.
	BqClient          *bigquery.Client
}

Dataset provides extensions to the bigquery Dataset and Dataset objects to streamline common actions. It encapsulates the Client and Dataset to simplify methods. DEPRECATED - use bqiface version in go/dataset

func NewDataset

func NewDataset(project, dataset string, clientOpts ...option.ClientOption) (Dataset, error)

NewDataset creates a Dataset for a project. httpClient is used to inject mocks for the bigquery client. if httpClient is nil, a suitable default client is used. Additional bigquery ClientOptions may be optionally passed as final

clientOpts argument.  This is useful for testing credentials.

DEPRECATED - use bqiface version in go/dataset

func (*Dataset) DestQuery

func (dsExt *Dataset) DestQuery(query string, dest *bigquery.Table, disposition bigquery.TableWriteDisposition) *bigquery.Query

DestinationQuery constructs a query with common Config settings for writing results to a table. If dest is nil, then this will create a DryRun query. TODO - should disposition be an opts... field instead? DEPRECATED - use bqiface version in go/dataset

func (*Dataset) ExecDestQuery

func (dsExt *Dataset) ExecDestQuery(q *bigquery.Query) (*bigquery.JobStatus, error)

ExecDestQuery executes a destination or dryrun query, and returns status or error.

func (Dataset) GetPartitionInfo

func (dsExt Dataset) GetPartitionInfo(table string, partition string) (PartitionInfo, error)

GetPartitionInfo provides basic information about a partition. DEPRECATED - use bqiface version in go/dataset

func (*Dataset) QueryAndParse

func (dsExt *Dataset) QueryAndParse(q string, structPtr interface{}) error

QueryAndParse executes a query that should return a single row, with all struct fields that match query columns filled in. The caller must pass in the *address* of an appropriate struct. TODO - extend this to also handle multirow results, by passing slice of structs. DEPRECATED - use bqiface version in go/dataset

func (*Dataset) ResultQuery

func (dsExt *Dataset) ResultQuery(query string, dryRun bool) *bigquery.Query

ResultQuery constructs a query with common QueryConfig settings for writing results to a table. Generally, may need to change WriteDisposition. DEPRECATED - use bqiface version in go/dataset

type PartitionInfo

type PartitionInfo struct {
	PartitionID  string
	CreationTime time.Time
	LastModified time.Time
}

PartitionInfo provides basic information about a partition. DEPRECATED - use bqiface version in go/dataset