bqutil

Documentation

Overview

Package bqutil provides utility functions to interact with BigQuery.

Index

• Constants
• func DatasetForProject(luciProject string) (string, error)
• func FatalError(err error) bool
• func MarshalStructPB(s *structpb.Struct) (string, error)
• func ProjectForDataset(dataset string) (string, error)
• func TestVariantIdentifier(testID string, variant *rdbpb.Variant) (*bqpb.TestVariantIdentifier, error)
• func VariantJSON(variant *rdbpb.Variant) (string, error)
• type Inserter
  • func NewInserter(table *bigquery.Table, batchSize int) *Inserter
  • func (i *Inserter) Put(ctx context.Context, rows []*bq.Row) error
  • func (i *Inserter) PutWithRetries(ctx context.Context, rows []*bq.Row) error

Constants

const InternalDatasetID = "internal"

InternalDatasetID is the name of the BigQuery dataset which is intended for internal service use only.

Functions

func DatasetForProject

func DatasetForProject(luciProject string) (string, error)

DatasetForProject returns the name of the BigQuery dataset that contains the given project's data, in the LUCI Analysis GCP project.

func FatalError

func FatalError(err error) bool

FatalError returns true if the error is a known fatal error.

func MarshalStructPB

func MarshalStructPB(s *structpb.Struct) (string, error)

MarshalStructPB serialises a structpb.Struct as a JSONPB.

func ProjectForDataset

func ProjectForDataset(dataset string) (string, error)

ProjectForDataset returns the name of the LUCI Project that corresponds to the given BigQuery dataset.

func TestVariantIdentifier

func TestVariantIdentifier(testID string, variant *rdbpb.Variant) (*bqpb.TestVariantIdentifier, error)

TestVariantIdentifier constructs a BigQuery-format TestVariantIdentifier from a flat ResultDB test ID and variant combination.

func VariantJSON

func VariantJSON(variant *rdbpb.Variant) (string, error)

VariantJSON returns the JSON equivalent for a variant. Each key in the variant is mapped to a top-level key in the JSON object. e.g. `{"builder":"linux-rel","os":"Ubuntu-18.04"}`

Types

type Inserter

type Inserter struct {
	// contains filtered or unexported fields
}

Inserter provides methods to insert rows into a BigQuery table.

func NewInserter

func NewInserter(table *bigquery.Table, batchSize int) *Inserter

NewInserter initialises a new inserter.

func (*Inserter) Put

func (i *Inserter) Put(ctx context.Context, rows []*bq.Row) error

Put inserts the given rows into BigQuery.

func (*Inserter) PutWithRetries

func (i *Inserter) PutWithRetries(ctx context.Context, rows []*bq.Row) error

PutWithRetries puts rows into BigQuery. Retries on transient errors.