bigquery

package
v0.0.0-...-816697b Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 27, 2017 License: Apache-2.0, Apache-2.0 Imports: 12 Imported by: 0

Documentation

Overview

Package bigquery provides a client for the BigQuery service.

Note: This package is a work-in-progress. Backwards-incompatible changes should be expected.

Index

Examples

Constants

View Source 
const (
	BatchPriority       = "BATCH"
	InteractivePriority = "INTERACTIVE"
)
View Source 
const Scope = "https://www.googleapis.com/auth/bigquery"

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client may be used to perform BigQuery operations.

func NewClient 

func NewClient(ctx context.Context, projectID string, opts ...option.ClientOption) (*Client, error)

NewClient constructs a new Client which can perform BigQuery operations. Operations performed via the client are billed to the specified GCP project.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	_ = client // TODO: Use client.
}

Output:

func (*Client) Copy 

func (c *Client) Copy(ctx context.Context, dst Destination, src Source, options ...Option) (*Job, error)

Copy starts a BigQuery operation to copy data from a Source to a Destination.

func (*Client) CreateTable deprecated 

func (c *Client) CreateTable(ctx context.Context, projectID, datasetID, tableID string, options ...CreateTableOption) (*Table, error)

CreateTable creates a table in the BigQuery service and returns a handle to it.

Deprecated: use Table.Create instead.

func (*Client) Dataset 

func (c *Client) Dataset(id string) *Dataset

Dataset creates a handle to a BigQuery dataset in the client's project.

Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	ds := client.Dataset("my-dataset")
	fmt.Println(ds)
}

Output:

func (*Client) DatasetInProject 

func (c *Client) DatasetInProject(projectID, datasetID string) *Dataset

DatasetInProject creates a handle to a BigQuery dataset in the specified project.

Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	ds := client.DatasetInProject("their-project-id", "their-dataset")
	fmt.Println(ds)
}

Output:

func (*Client) Datasets 

func (c *Client) Datasets(ctx context.Context) *DatasetIterator

Datasets returns an iterator over the datasets in the Client's project.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	it := client.Datasets(ctx)
	_ = it // TODO: iterate using Next or iterator.Pager.
}

Output:

func (*Client) DatasetsInProject 

func (c *Client) DatasetsInProject(ctx context.Context, projectID string) *DatasetIterator

DatasetsInProject returns an iterator over the datasets in the provided project.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	it := client.DatasetsInProject(ctx, "their-project-id")
	_ = it // TODO: iterate using Next or iterator.Pager.
}

Output:

func (*Client) JobFromID 

func (c *Client) JobFromID(ctx context.Context, id string) (*Job, error)

JobFromID creates a Job which refers to an existing BigQuery job. The job need not have been created by this package. For example, the job may have been created in the BigQuery console.

Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func getJobID() string { return "" }

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	jobID := getJobID() // Get a job ID using Job.ID, the console or elsewhere.
	job, err := client.JobFromID(ctx, jobID)
	if err != nil {
		// TODO: Handle error.
	}
	fmt.Println(job)
}

Output:

func (*Client) NewGCSReference 

func (c *Client) NewGCSReference(uri ...string) *GCSReference

NewGCSReference constructs a reference to one or more Google Cloud Storage objects, which together constitute a data source or destination. In the simple case, a single URI in the form gs://bucket/object may refer to a single GCS object. Data may also be split into mutiple files, if multiple URIs or URIs containing wildcards are provided. Each URI may contain one '*' wildcard character, which (if present) must come after the bucket name. For more information about the treatment of wildcards and multiple URIs, see https://cloud.google.com/bigquery/exporting-data-from-bigquery#exportingmultiple

func (*Client) OpenTable deprecated 

func (c *Client) OpenTable(projectID, datasetID, tableID string) *Table

OpenTable creates a handle to an existing BigQuery table. If the table does not already exist, subsequent uses of the *Table will fail.

Deprecated: use Client.DatasetInProject.Table instead.

func (*Client) Query 

func (c *Client) Query(q string) *Query

Query creates a query with string q. You may optionally set DefaultProjectID and DefaultDatasetID on the returned query before using it.

func (*Client) Read deprecated 

func (c *Client) Read(ctx context.Context, src ReadSource, options ...ReadOption) (*Iterator, error)

Read fetches data from a ReadSource and returns the data via an Iterator.

Deprecated: use Query.Read, Job.Read or Table.Read instead.

func (*Client) Table deprecated 

func (c *Client) Table(projectID, datasetID, tableID string) *Table

Table creates a handle to a BigQuery table.

Use this method to reference a table in a project other than that of the Client.

Deprecated: use Client.DatasetInProject.Table instead.

type Compression 

type Compression string

Compression is the type of compression to apply when writing data to Google Cloud Storage. 

const (
	None Compression = "NONE"
	Gzip Compression = "GZIP"
)

type CreateTableOption 

type CreateTableOption interface {
	// contains filtered or unexported methods
}

A CreateTableOption is an optional argument to CreateTable.

func TableExpiration 

func TableExpiration(exp time.Time) CreateTableOption

TableExpiration returns a CreateTableOption that will cause the created table to be deleted after the expiration time.

func UseStandardSQL 

func UseStandardSQL() CreateTableOption

UseStandardSQL returns a CreateTableOption to set the table to use standard SQL. The default setting is false (using legacy SQL).

func ViewQuery 

func ViewQuery(query string) CreateTableOption

ViewQuery returns a CreateTableOption that causes the created table to be a virtual table defined by the supplied query. For more information see: https://cloud.google.com/bigquery/querying-data#views

type DataFormat 

type DataFormat string
const (
	CSV             DataFormat = "CSV"
	Avro            DataFormat = "AVRO"
	JSON            DataFormat = "NEWLINE_DELIMITED_JSON"
	DatastoreBackup DataFormat = "DATASTORE_BACKUP"
)

type Dataset 

type Dataset struct {
	// contains filtered or unexported fields
}

Dataset is a reference to a BigQuery dataset.

func (*Dataset) Create 

func (d *Dataset) Create(ctx context.Context) error

Create creates a dataset in the BigQuery service. An error will be returned if the dataset already exists.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	if err := client.Dataset("new-dataset").Create(ctx); err != nil {
		// TODO: Handle error.
	}
}

Output:

func (*Dataset) ListTables deprecated 

func (d *Dataset) ListTables(ctx context.Context) ([]*Table, error)

ListTables returns a list of all the tables contained in the Dataset.

Deprecated: use Dataset.Tables instead.

func (*Dataset) Table 

func (d *Dataset) Table(tableID string) *Table

Table creates a handle to a BigQuery table in the dataset. To determine if a table exists, call Table.Metadata. If the table does not already exist, use Table.Create to create it.

Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	// Table creates a reference to the table. It does not create the actual
	// table in BigQuery; to do so, use Table.Create.
	t := client.Dataset("my-dataset").Table("my-table")
	fmt.Println(t)
}

Output:

func (*Dataset) Tables 

func (d *Dataset) Tables(ctx context.Context) *TableIterator

Tables returns an iterator over the tables in the Dataset.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	it := client.Dataset("my-dataset").Tables(ctx)
	_ = it // TODO: iterate using Next or iterator.Pager.
}

Output:

type DatasetIterator 

type DatasetIterator struct {
	// ListHidden causes hidden datasets to be listed when set to true.
	ListHidden bool

	// Filter restricts the datasets returned by label. The filter syntax is described in
	// https://cloud.google.com/bigquery/docs/labeling-datasets#filtering_datasets_using_labels
	Filter string
	// contains filtered or unexported fields
}

DatasetIterator iterates over the datasets in a project.

func (*DatasetIterator) Next 

func (it *DatasetIterator) Next() (*Dataset, error)
Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
	"google.golang.org/api/iterator"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	it := client.Datasets(ctx)
	for {
		ds, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		fmt.Println(ds)
	}
}

Output:

func (*DatasetIterator) PageInfo 

func (it *DatasetIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

type Destination 

type Destination interface {
	// contains filtered or unexported methods
}

A Destination is a destination of data for the Copy function.

type Encoding 

type Encoding string

Encoding specifies the character encoding of data to be loaded into BigQuery. See https://cloud.google.com/bigquery/docs/reference/v2/jobs#configuration.load.encoding for more details about how this is used. 

const (
	UTF_8      Encoding = "UTF-8"
	ISO_8859_1 Encoding = "ISO-8859-1"
)

type Error 

type Error struct {
	// Mirrors bq.ErrorProto, but drops DebugInfo
	Location, Message, Reason string
}

An Error contains detailed information about a failed bigquery operation.

func (Error) Error 

func (e Error) Error() string

type FieldSchema 

type FieldSchema struct {
	// The field name.
	// Must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_),
	// and must start with a letter or underscore.
	// The maximum length is 128 characters.
	Name string

	// A description of the field. The maximum length is 16,384 characters.
	Description string

	// Whether the field may contain multiple values.
	Repeated bool
	// Whether the field is required.  Ignored if Repeated is true.
	Required bool

	// The field data type.  If Type is Record, then this field contains a nested schema,
	// which is described by Schema.
	Type FieldType
	// Describes the nested schema if Type is set to Record.
	Schema Schema
}

type FieldType 

type FieldType string
const (
	StringFieldType    FieldType = "STRING"
	IntegerFieldType   FieldType = "INTEGER"
	FloatFieldType     FieldType = "FLOAT"
	BooleanFieldType   FieldType = "BOOLEAN"
	TimestampFieldType FieldType = "TIMESTAMP"
	RecordFieldType    FieldType = "RECORD"
)

type GCSReference 

type GCSReference struct {

	// FieldDelimiter is the separator for fields in a CSV file, used when loading or exporting data.
	// The default is ",".
	FieldDelimiter string

	// The number of rows at the top of a CSV file that BigQuery will skip when loading the data.
	SkipLeadingRows int64

	// SourceFormat is the format of the GCS data to be loaded into BigQuery.
	// Allowed values are: CSV, JSON, DatastoreBackup.  The default is CSV.
	SourceFormat DataFormat
	// Only used when loading data.
	Encoding Encoding

	// Quote is the value used to quote data sections in a CSV file.
	// The default quotation character is the double quote ("), which is used if both Quote and ForceZeroQuote are unset.
	// To specify that no character should be interpreted as a quotation character, set ForceZeroQuote to true.
	// Only used when loading data.
	Quote          string
	ForceZeroQuote bool

	// DestinationFormat is the format to use when writing exported files.
	// Allowed values are: CSV, Avro, JSON.  The default is CSV.
	// CSV is not supported for tables with nested or repeated fields.
	DestinationFormat DataFormat
	// Only used when writing data.  Default is None.
	Compression Compression
	// contains filtered or unexported fields
}

GCSReference is a reference to one or more Google Cloud Storage objects, which together constitute an input or output to a BigQuery operation.

type Iterator 

type Iterator struct {
	// contains filtered or unexported fields
}

Iterator provides access to the result of a BigQuery lookup. Next must be called before the first call to Get.

func (*Iterator) Err 

func (it *Iterator) Err() error

Err returns the last error encountered by Next, or nil for no error.

func (*Iterator) Get 

func (it *Iterator) Get(dst interface{}) error

Get loads the current row into dst, which must implement ValueLoader.

func (*Iterator) Next 

func (it *Iterator) Next(ctx context.Context) bool

Next advances the Iterator to the next row, making that row available via the Get method. Next must be called before the first call to Get or Schema, and blocks until data is available. Next returns false when there are no more rows available, either because the end of the output was reached, or because there was an error (consult the Err method to determine which).

func (*Iterator) Schema 

func (it *Iterator) Schema() (Schema, error)

Schema returns the schema of the result rows.

type Job 

type Job struct {
	// contains filtered or unexported fields
}

A Job represents an operation which has been submitted to BigQuery for processing.

func (*Job) Cancel 

func (j *Job) Cancel(ctx context.Context) error

Cancel requests that a job be cancelled. This method returns without waiting for cancellation to take effect. To check whether the job has terminated, use Job.Status. Cancelled jobs may still incur costs.

func (*Job) ID 

func (j *Job) ID() string

func (*Job) Read 

func (j *Job) Read(_ context.Context, options ...ReadOption) (*Iterator, error)

Read fetches the results of a query job.

func (*Job) Status 

func (j *Job) Status(ctx context.Context) (*JobStatus, error)

Status returns the current status of the job. It fails if the Status could not be determined.

type JobStatus 

type JobStatus struct {
	State State

	// All errors encountered during the running of the job.
	// Not all Errors are fatal, so errors here do not necessarily mean that the job has completed or was unsuccessful.
	Errors []*Error
	// contains filtered or unexported fields
}

JobStatus contains the current State of a job, and errors encountered while processing that job.

func (*JobStatus) Done 

func (s *JobStatus) Done() bool

Done reports whether the job has completed. After Done returns true, the Err method will return an error if the job completed unsuccesfully.

func (*JobStatus) Err 

func (s *JobStatus) Err() error

Err returns the error that caused the job to complete unsuccesfully (if any).

type MultiError 

type MultiError []error

A MultiError contains multiple related errors.

func (MultiError) Error 

func (m MultiError) Error() string

type Option 

type Option interface {
	// contains filtered or unexported methods
}

An Option is an optional argument to Copy.

func AllowJaggedRows 

func AllowJaggedRows() Option

AllowJaggedRows returns an Option that causes missing trailing optional columns to be tolerated in CSV data. Missing values are treated as nulls.

func AllowLargeResults 

func AllowLargeResults() Option

AllowLargeResults returns an Option that allows the query to produce arbitrarily large result tables. The destination must be a table. When using this option, queries will take longer to execute, even if the result set is small. For additional limitations, see https://cloud.google.com/bigquery/querying-data#largequeryresults

func AllowQuotedNewlines 

func AllowQuotedNewlines() Option

AllowQuotedNewlines returns an Option that allows quoted data sections containing newlines in CSV data.

func CreateDisposition 

func CreateDisposition(disp TableCreateDisposition) Option

func DestinationSchema 

func DestinationSchema(schema Schema) Option

DestinationSchema returns an Option that specifies the schema to use when loading data into a new table. A DestinationSchema Option must be supplied when loading data from Google Cloud Storage into a non-existent table. Caveat: DestinationSchema is not required if the data being loaded is a datastore backup. schema must not be nil.

func DisableFlattenedResults 

func DisableFlattenedResults() Option

DisableFlattenedResults returns an Option that prevents results being flattened. If this Option is not used, results from nested and repeated fields are flattened. DisableFlattenedResults implies AllowLargeResults For more information, see https://cloud.google.com/bigquery/docs/data#nested

func DisableHeader 

func DisableHeader() Option

DisableHeader returns an Option that disables the printing of a header row in exported data.

func DisableQueryCache 

func DisableQueryCache() Option

DisableQueryCache returns an Option that prevents results being fetched from the query cache. If this Option is not used, results are fetched from the cache if they are available. The query cache is a best-effort cache that is flushed whenever tables in the query are modified. Cached results are only available when TableID is unspecified in the query's destination Table. For more information, see https://cloud.google.com/bigquery/querying-data#querycaching

func IgnoreUnknownValues 

func IgnoreUnknownValues() Option

IgnoreUnknownValues returns an Option that causes values not matching the schema to be tolerated. Unknown values are ignored. For CSV this ignores extra values at the end of a line. For JSON this ignores named values that do not match any column name. If this Option is not used, records containing unknown values are treated as bad records. The MaxBadRecords Option can be used to customize how bad records are handled.

func JobID 

func JobID(ID string) Option

JobID returns an Option that sets the job ID of a BigQuery job. If this Option is not used, a job ID is generated automatically.

func JobPriority 

func JobPriority(priority string) Option

JobPriority returns an Option that causes a query to be scheduled with the specified priority. The default priority is InteractivePriority. For more information, see https://cloud.google.com/bigquery/querying-data#batchqueries

func MaxBadRecords 

func MaxBadRecords(n int64) Option

MaxBadRecords returns an Option that sets the maximum number of bad records that will be ignored. If this maximum is exceeded, the operation will be unsuccessful.

func MaxBillingTier 

func MaxBillingTier(tier int) Option

MaxBillingTier returns an Option that sets the maximum billing tier for a Query. Queries that have resource usage beyond this tier will fail (without incurring a charge). If this Option is not used, the project default will be used.

func MaxBytesBilled 

func MaxBytesBilled(bytes int64) Option

MaxBytesBilled returns an Option that limits the number of bytes billed for this job. Queries that would exceed this limit will fail (without incurring a charge). If this Option is not used, or bytes is < 1, the project default will be used.

func QueryUseStandardSQL 

func QueryUseStandardSQL() Option

QueryUseStandardSQL returns an Option that set the query to use standard SQL. The default setting is false (using legacy SQL).

func WriteDisposition 

func WriteDisposition(disp TableWriteDisposition) Option

type PutMultiError 

type PutMultiError []RowInsertionError

PutMultiError contains an error for each row which was not successfully inserted into a BigQuery table.

func (PutMultiError) Error 

func (pme PutMultiError) Error() string

type Query 

type Query struct {
	// The query to execute. See https://cloud.google.com/bigquery/query-reference for details.
	Q string

	// DefaultProjectID and DefaultDatasetID specify the dataset to use for unqualified table names in the query.
	// If DefaultProjectID is set, DefaultDatasetID must also be set.
	DefaultProjectID string
	DefaultDatasetID string
	// contains filtered or unexported fields
}

Query represents a query to be executed. Use Client.Query to create a query.

func (*Query) Read 

func (q *Query) Read(ctx context.Context, options ...ReadOption) (*Iterator, error)

Read submits a query for execution and returns the results via an Iterator.

Read uses a temporary table to hold the results of the query job.

For more control over how a query is performed, don't use this method but instead pass the Query as a Source to Client.Copy, and call Read on the resulting Job.

type ReadOption 

type ReadOption interface {
	// contains filtered or unexported methods
}

A ReadOption is an optional argument to Read.

func RecordsPerRequest 

func RecordsPerRequest(n int64) ReadOption

RecordsPerRequest returns a ReadOption that sets the number of records to fetch per request when streaming data from BigQuery.

func StartIndex 

func StartIndex(i uint64) ReadOption

StartIndex returns a ReadOption that sets the zero-based index of the row to start reading from.

type ReadSource 

type ReadSource interface {
	// contains filtered or unexported methods
}

A ReadSource is a source of data for the Read function.

type RowInsertionError 

type RowInsertionError struct {
	InsertID string // The InsertID associated with the affected row.
	RowIndex int    // The 0-based index of the affected row in the batch of rows being inserted.
	Errors   MultiError
}

RowInsertionError contains all errors that occurred when attempting to insert a row.

func (*RowInsertionError) Error 

func (e *RowInsertionError) Error() string

type Schema 

type Schema []*FieldSchema

Schema describes the fields in a table or query result.

func InferSchema 

func InferSchema(st interface{}) (Schema, error)

InferSchema tries to derive a BigQuery schema from the supplied struct value. NOTE: All fields in the returned Schema are configured to be required, unless the corresponding field in the supplied struct is a slice or array. It is considered an error if the struct (including nested structs) contains any exported fields that are pointers or one of the following types: map, interface, complex64, complex128, func, chan. In these cases, an error will be returned. Future versions may handle these cases without error.

Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
)

func main() {
	type Item struct {
		Name  string
		Size  float64
		Count int
	}
	schema, err := bigquery.InferSchema(Item{})
	if err != nil {
		fmt.Println(err)
		// TODO: Handle error.
	}
	for _, fs := range schema {
		fmt.Println(fs.Name, fs.Type)
	}
}

Output:

Name STRING
Size FLOAT
Count INTEGER

type Source 

type Source interface {
	// contains filtered or unexported methods
}

A Source is a source of data for the Copy function.

type State 

type State int

State is one of a sequence of states that a Job progresses through as it is processed. 

const (
	Pending State = iota
	Running
	Done
)

type Table 

type Table struct {
	// ProjectID, DatasetID and TableID may be omitted if the Table is the destination for a query.
	// In this case the result will be stored in an ephemeral table.
	ProjectID string
	DatasetID string
	// TableID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_).
	// The maximum length is 1,024 characters.
	TableID string
	// contains filtered or unexported fields
}

A Table is a reference to a BigQuery table.

func (*Table) Create 

func (t *Table) Create(ctx context.Context, options ...CreateTableOption) error

Create creates a table in the BigQuery service.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	t := client.Dataset("my-dataset").Table("new-table")
	if err := t.Create(ctx); err != nil {
		// TODO: Handle error.
	}
}

Output:

func (*Table) Delete 

func (t *Table) Delete(ctx context.Context) error

Delete deletes the table.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	if err := client.Dataset("my-dataset").Table("my-table").Delete(ctx); err != nil {
		// TODO: Handle error.
	}
}

Output:

func (*Table) FullyQualifiedName 

func (t *Table) FullyQualifiedName() string

FullyQualifiedName returns the ID of the table in projectID:datasetID.tableID format.

func (*Table) Metadata 

func (t *Table) Metadata(ctx context.Context) (*TableMetadata, error)

Metadata fetches the metadata for the table.

Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	md, err := client.Dataset("my-dataset").Table("my-table").Metadata(ctx)
	if err != nil {
		// TODO: Handle error.
	}
	fmt.Println(md)
}

Output:

func (*Table) NewUploader 

func (t *Table) NewUploader(opts ...UploadOption) *Uploader

NewUploader returns an *Uploader that can be used to append rows to t.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	u := client.Dataset("my-dataset").Table("my-table").NewUploader()
	_ = u // TODO: Use u.
}

Output:

func (*Table) Patch 

func (t *Table) Patch() *TableMetadataPatch

Patch returns a *TableMetadataPatch, which can be used to modify specific Table metadata fields. In order to apply the changes, the TableMetadataPatch's Apply method must be called.

func (*Table) Read 

func (t *Table) Read(_ context.Context, options ...ReadOption) (*Iterator, error)

Read fetches the contents of the table.

type TableCreateDisposition 

type TableCreateDisposition string

CreateDisposition specifies the circumstances under which destination table will be created. Default is CreateIfNeeded. 

const (
	// The table will be created if it does not already exist.  Tables are created atomically on successful completion of a job.
	CreateIfNeeded TableCreateDisposition = "CREATE_IF_NEEDED"

	// The table must already exist and will not be automatically created.
	CreateNever TableCreateDisposition = "CREATE_NEVER"
)

type TableIterator 

type TableIterator struct {
	// contains filtered or unexported fields
}

A TableIterator is an iterator over Tables.

func (*TableIterator) Next 

func (it *TableIterator) Next() (*Table, error)

Next returns the next result. Its second return value is Done if there are no more results. Once Next returns Done, all subsequent calls will return Done.

Example 
package main

import (
	"fmt"

	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
	"google.golang.org/api/iterator"
)

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	it := client.Dataset("my-dataset").Tables(ctx)
	for {
		t, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			// TODO: Handle error.
		}
		fmt.Println(t)
	}
}

Output:

func (*TableIterator) PageInfo 

func (it *TableIterator) PageInfo() *iterator.PageInfo

PageInfo supports pagination. See the google.golang.org/api/iterator package for details.

type TableMetadata 

type TableMetadata struct {
	Description string // The user-friendly description of this table.
	Name        string // The user-friendly name for this table.
	Schema      Schema
	View        string

	ID   string // An opaque ID uniquely identifying the table.
	Type TableType

	// The time when this table expires. If not set, the table will persist
	// indefinitely. Expired tables will be deleted and their storage reclaimed.
	ExpirationTime time.Time

	CreationTime     time.Time
	LastModifiedTime time.Time

	// The size of the table in bytes.
	// This does not include data that is being buffered during a streaming insert.
	NumBytes int64

	// The number of rows of data in this table.
	// This does not include data that is being buffered during a streaming insert.
	NumRows uint64
}

TableMetadata contains information about a BigQuery table.

type TableMetadataPatch 

type TableMetadataPatch struct {
	// contains filtered or unexported fields
}

TableMetadataPatch represents a set of changes to a table's metadata.

func (*TableMetadataPatch) Apply 

func (p *TableMetadataPatch) Apply(ctx context.Context) (*TableMetadata, error)

Apply applies the patch operation.

func (*TableMetadataPatch) Description 

func (p *TableMetadataPatch) Description(desc string)

Description sets the table description.

func (*TableMetadataPatch) Name 

func (p *TableMetadataPatch) Name(name string)

Name sets the table name.

type TableType 

type TableType string

TableType is the type of table. 

const (
	RegularTable TableType = "TABLE"
	ViewTable    TableType = "VIEW"
)

type TableWriteDisposition 

type TableWriteDisposition string

TableWriteDisposition specifies how existing data in a destination table is treated. Default is WriteAppend. 

const (
	// Data will be appended to any existing data in the destination table.
	// Data is appended atomically on successful completion of a job.
	WriteAppend TableWriteDisposition = "WRITE_APPEND"

	// Existing data in the destination table will be overwritten.
	// Data is overwritten atomically on successful completion of a job.
	WriteTruncate TableWriteDisposition = "WRITE_TRUNCATE"

	// Writes will fail if the destination table already contains data.
	WriteEmpty TableWriteDisposition = "WRITE_EMPTY"
)

type Tables 

type Tables []*Table

Tables is a group of tables. The tables may belong to differing projects or datasets.

type UploadOption 

type UploadOption interface {
	// contains filtered or unexported methods
}

An UploadOption is an optional argument to NewUploader.

func SkipInvalidRows 

func SkipInvalidRows() UploadOption

SkipInvalidRows returns an UploadOption that causes rows containing invalid data to be silently ignored. The default value is false, which causes the entire request to fail, if there is an attempt to insert an invalid row.

func TableTemplateSuffix 

func TableTemplateSuffix(suffix string) UploadOption

A TableTemplateSuffix allows Uploaders to create tables automatically.

Experimental: this option is experimental and may be modified or removed in future versions, regardless of any other documented package stability guarantees.

When you specify a suffix, the table you upload data to will be used as a template for creating a new table, with the same schema, called <table> + <suffix>.

More information is available at https://cloud.google.com/bigquery/streaming-data-into-bigquery#template-tables

func UploadIgnoreUnknownValues 

func UploadIgnoreUnknownValues() UploadOption

UploadIgnoreUnknownValues returns an UploadOption that causes values not matching the schema to be ignored. If this option is not used, records containing such values are treated as invalid records.

type Uploader 

type Uploader struct {
	// contains filtered or unexported fields
}

An Uploader does streaming inserts into a BigQuery table. It is safe for concurrent use.

func (*Uploader) Put 

func (u *Uploader) Put(ctx context.Context, src interface{}) error

Put uploads one or more rows to the BigQuery service. src must implement ValueSaver or be a slice of ValueSavers. Put returns a PutMultiError if one or more rows failed to be uploaded. The PutMultiError contains a RowInsertionError for each failed row.

Example 
package main

import (
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

type Item struct {
	Name  string
	Size  float64
	Count int
}

// Save implements the ValueSaver interface.
func (i *Item) Save() (map[string]bigquery.Value, string, error) {
	return map[string]bigquery.Value{
		"Name":  i.Name,
		"Size":  i.Size,
		"Count": i.Count,
	}, "", nil
}

func main() {
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
		// TODO: Handle error.
	}
	u := client.Dataset("my-dataset").Table("my-table").NewUploader()
	// Item implements the ValueSaver interface.
	items := []*Item{
		{Name: "n1", Size: 32.6, Count: 7},
		{Name: "n2", Size: 4, Count: 2},
		{Name: "n3", Size: 101.5, Count: 1},
	}
	if err := u.Put(ctx, items); err != nil {
		// TODO: Handle error.
	}
}

Output:

type Value 

type Value interface{}

Value stores the contents of a single cell from a BigQuery result.

type ValueList 

type ValueList []Value

ValueList converts a []Value to implement ValueLoader.

func (*ValueList) Load 

func (vs *ValueList) Load(v []Value) error

Load stores a sequence of values in a ValueList.

type ValueLoader 

type ValueLoader interface {
	Load(v []Value) error
}

ValueLoader stores a slice of Values representing a result row from a Read operation. See Iterator.Get for more information.

type ValueSaver 

type ValueSaver interface {
	// Save returns a row to be inserted into a BigQuery table, represented
	// as a map from field name to Value.
	// If insertID is non-empty, BigQuery will use it to de-duplicate
	// insertions of this row on a best-effort basis.
	Save() (row map[string]Value, insertID string, err error)
}

A ValueSaver returns a row of data to be inserted into a table.

type ValuesSaver 

type ValuesSaver struct {
	Schema Schema

	// If non-empty, BigQuery will use InsertID to de-duplicate insertions
	// of this row on a best-effort basis.
	InsertID string

	Row []Value
}

ValuesSaver implements ValueSaver for a slice of Values.

func (*ValuesSaver) Save 

func (vls *ValuesSaver) Save() (map[string]Value, string, error)

Save implements ValueSaver

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.