bqexporter

Documentation

Overview

Package bqexporter handles the export of test variant analysis results to BigQuery.

Index

• func MergeTables(ctx context.Context, gcpProject string) (retErr error)
• type Client
  • func NewClient(ctx context.Context, projectID string) (s *Client, reterr error)
  • func (s *Client) Close() (reterr error)
  • func (s *Client) Insert(ctx context.Context, rows []*bqpb.TestVariantBranchRow) error
• type Exporter
  • func NewExporter(client InsertClient) *Exporter
  • func (e *Exporter) ExportTestVariantBranches(ctx context.Context, rowInputs RowInputs) error
• type FakeClient
  • func NewFakeClient() *FakeClient
  • func (fc *FakeClient) Insert(ctx context.Context, rows []*bqp.TestVariantBranchRow) error
• type InsertClient
• type PartialBigQueryRow
  • func ToPartialBigQueryRow(tvb *testvariantbranch.Entry, segments []analyzer.Segment) (PartialBigQueryRow, error)
  • func (r PartialBigQueryRow) Complete(commitTimestamp time.Time) *bqpb.TestVariantBranchRow
• type RowInputs

Functions

func MergeTables

func MergeTables(ctx context.Context, gcpProject string) (retErr error)

MergeTables is the entry point of the merge-test-variant-branches cron job. It runs use DML merge to merge the data from test_variant_segment_updates table to test_variant_segments table.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client provides methods to export clustered failures to BigQuery via the BigQuery Write API.

func NewClient

func NewClient(ctx context.Context, projectID string) (s *Client, reterr error)

NewClient creates a new client for exporting test variant branches via the BigQuery Write API. projectID is the project ID of the GCP project.

func (*Client) Close

func (s *Client) Close() (reterr error)

Close releases resources held by the client.

func (*Client) Insert

func (s *Client) Insert(ctx context.Context, rows []*bqpb.TestVariantBranchRow) error

Insert inserts the given rows in BigQuery.

type Exporter

type Exporter struct {
	// contains filtered or unexported fields
}

Exporter provides methods to export test variant branches to BigQuery.

func NewExporter

func NewExporter(client InsertClient) *Exporter

NewExporter instantiates a new Exporter. The given client is used to insert rows into BigQuery.

func (*Exporter) ExportTestVariantBranches

func (e *Exporter) ExportTestVariantBranches(ctx context.Context, rowInputs RowInputs) error

ExportTestVariantBranches exports test variant branches to BigQuery.

type FakeClient

type FakeClient struct {
	Insertions []*bqp.TestVariantBranchRow
}

FakeClient represents a fake implementation of the test variant branch exporter, for testing.

func NewFakeClient

func NewFakeClient() *FakeClient

NewFakeClient creates a new FakeClient for exporting test variant branches.

func (*FakeClient) Insert

func (fc *FakeClient) Insert(ctx context.Context, rows []*bqp.TestVariantBranchRow) error

Insert inserts the given rows in BigQuery.

type InsertClient

type InsertClient interface {
	// Insert inserts the given rows into BigQuery.
	Insert(ctx context.Context, rows []*bqpb.TestVariantBranchRow) error
}

InsertClient defines an interface for inserting TestVariantBranchRow into BigQuery.

type PartialBigQueryRow

type PartialBigQueryRow struct {
	// contains filtered or unexported fields
}

PartialBigQueryRow represents a partially constructed BigQuery export row. Call Complete(...) on the row to finish its construction.

func ToPartialBigQueryRow

func ToPartialBigQueryRow(tvb *testvariantbranch.Entry, segments []analyzer.Segment) (PartialBigQueryRow, error)

ToPartialBigQueryRow starts building a BigQuery TestVariantBranchRow. All fields except those dependent on the commit timestamp, (i.e. Version and HasRecentUnexpectedResults) are populated.

segments is the logical segments on the TestVariantBranch. Segments are sorted by commit position (most recent first).

To support re-use of the *testvariantbranch.Entry buffer, no reference to tvb or inputBufferSegments or their fields (except immutable strings) will be retained by this method or its result. (All data will be copied.)

func (PartialBigQueryRow) Complete

func (r PartialBigQueryRow) Complete(commitTimestamp time.Time) *bqpb.TestVariantBranchRow

Complete finishes creating the BigQuery export row, returning it.

type RowInputs

type RowInputs struct {
	Rows            []PartialBigQueryRow
	CommitTimestamp time.Time
}

RowInputs is the contains the rows to be exported to BigQuery, together with the Spanner commit timestamp.