bqexporter

package
v0.0.0-...-864d242 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 3, 2024 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Overview

Package bqexporter exports status data to BigQuery.

Package bqexporter exports status data to BigQuery.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ExportStatus

func ExportStatus(ctx context.Context) error

ExportStatus exports status data to BigQuery. It queries Spanner and BigQuery and exports rows in Spanner that are not in BigQuery. This is run from a cron job and only 1 instance may be run at one time. Otherwise, we may run into the issue of inserting duplicated rows into BigQuery.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client provides methods to export statuses to BigQuery via the BigQuery Write API.

func NewClient

func NewClient(ctx context.Context, projectID string) (s *Client, reterr error)

NewClient creates a new client for exporting statuses via the BigQuery Write API.

func (*Client) Close

func (c *Client) Close() (reterr error)

Close releases resources held by the client.

func (*Client) EnsureSchema

func (c *Client) EnsureSchema(ctx context.Context) error

func (*Client) InsertStatusRows

func (c *Client) InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error

InsertStatusRows inserts the given rows in BigQuery.

func (*Client) ReadMostRecentCreateTime

func (c *Client) ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)

ReadMostRecentCreateTime reads the most recent create_time from `luci-tree-status.internal.statuses` table. If the table is empty, it will returns time.Time{}.

type ExportClient

type ExportClient interface {
	EnsureSchema(ctx context.Context) error
	InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error
	ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)
}

type FakeClient

type FakeClient struct {
	Insertions []*bqpb.StatusRow
}

FakeClient represents a fake implementation of the test variant branch exporter, for testing.

func NewFakeClient

func NewFakeClient() *FakeClient

NewFakeClient creates a new FakeClient for exporting test variant branches.

func (*FakeClient) EnsureSchema

func (fc *FakeClient) EnsureSchema(ctx context.Context) error

func (*FakeClient) InsertStatusRows

func (fc *FakeClient) InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error

func (*FakeClient) ReadMostRecentCreateTime

func (fc *FakeClient) ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL