Documentation
¶
Overview ¶
Package bqext includes generally useful abstractions for simplifying interactions with bigquery. Production extensions should go here, but test facilities should go in a separate bqtest package.
Index ¶
- type Dataset
- func (dsExt *Dataset) Dedup(src string, dedupOn string, overwrite bool, project, dataset, table string) (*bigquery.JobStatus, error)
- func (dsExt *Dataset) DestinationQuery(query string, dest *bigquery.Table) *bigquery.Query
- func (dsExt Dataset) GetPartitionInfo(table string, partition string) (PartitionInfo, error)
- func (dsExt *Dataset) QueryAndParse(q string, structPtr interface{}) error
- func (dsExt *Dataset) ResultQuery(query string, dryRun bool) *bigquery.Query
- type PartitionInfo
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Dataset ¶ added in v0.1.2
Dataset provides extensions to the bigquery Dataset and Dataset objects to streamline common actions. It encapsulates the Client and Dataset to simplify methods.
func NewDataset ¶ added in v0.1.2
func NewDataset(project, dataset string, clientOpts ...option.ClientOption) (Dataset, error)
NewDataset creates a Dataset for a project. httpClient is used to inject mocks for the bigquery client. if httpClient is nil, a suitable default client is used. Additional bigquery ClientOptions may be optionally passed as final
clientOpts argument. This is useful for testing credentials.
func (*Dataset) Dedup ¶ added in v0.1.2
func (dsExt *Dataset) Dedup(src string, dedupOn string, overwrite bool, project, dataset, table string) (*bigquery.JobStatus, error)
Dedup executes a query that dedups and writes to an appropriate partition. src is relative to the project:dataset of dsExt. dedupOn names the field to be used for dedupping. project, dataset, table specify the table to write into. NOTE: destination table must include the partition suffix. This avoids accidentally overwriting TODAY's partition.
func (*Dataset) DestinationQuery ¶ added in v0.1.2
DestinationQuery constructs a query with common Config settings for writing results to a table. Generally, may need to change WriteDisposition.
func (Dataset) GetPartitionInfo ¶ added in v0.1.2
func (dsExt Dataset) GetPartitionInfo(table string, partition string) (PartitionInfo, error)
GetPartitionInfo provides basic information about a partition.
func (*Dataset) QueryAndParse ¶ added in v0.1.2
QueryAndParse executes a query that should return a single row, with all struct fields that match query columns filled in. The caller must pass in the *address* of an appropriate struct. TODO - extend this to also handle multirow results, by passing slice of structs.