bqexporter

Documentation

Overview

Package bqexporter exports status data to BigQuery.

Package bqexporter exports status data to BigQuery.

Index

• func ExportStatus(ctx context.Context) error
• type Client
  • func NewClient(ctx context.Context, projectID string) (s *Client, reterr error)
  • func (c *Client) Close() (reterr error)
  • func (c *Client) EnsureSchema(ctx context.Context) error
  • func (c *Client) InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error
  • func (c *Client) ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)
• type ExportClient
• type FakeClient
  • func NewFakeClient() *FakeClient
  • func (fc *FakeClient) EnsureSchema(ctx context.Context) error
  • func (fc *FakeClient) InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error
  • func (fc *FakeClient) ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)

Functions

func ExportStatus

func ExportStatus(ctx context.Context) error

ExportStatus exports status data to BigQuery. It queries Spanner and BigQuery and exports rows in Spanner that are not in BigQuery. This is run from a cron job and only 1 instance may be run at one time. Otherwise, we may run into the issue of inserting duplicated rows into BigQuery.

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client provides methods to export statuses to BigQuery via the BigQuery Write API.

func NewClient

func NewClient(ctx context.Context, projectID string) (s *Client, reterr error)

NewClient creates a new client for exporting statuses via the BigQuery Write API.

func (*Client) Close

func (c *Client) Close() (reterr error)

Close releases resources held by the client.

func (*Client) EnsureSchema

func (c *Client) EnsureSchema(ctx context.Context) error

func (*Client) InsertStatusRows

func (c *Client) InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error

InsertStatusRows inserts the given rows in BigQuery.

func (*Client) ReadMostRecentCreateTime

func (c *Client) ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)

ReadMostRecentCreateTime reads the most recent create_time from `luci-tree-status.internal.statuses` table. If the table is empty, it will returns time.Time{}.

type ExportClient

type ExportClient interface {
	EnsureSchema(ctx context.Context) error
	InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error
	ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)
}

type FakeClient

type FakeClient struct {
	Insertions []*bqpb.StatusRow
}

FakeClient represents a fake implementation of the test variant branch exporter, for testing.

func NewFakeClient

func NewFakeClient() *FakeClient

NewFakeClient creates a new FakeClient for exporting test variant branches.

func (*FakeClient) EnsureSchema

func (fc *FakeClient) EnsureSchema(ctx context.Context) error

func (*FakeClient) InsertStatusRows

func (fc *FakeClient) InsertStatusRows(ctx context.Context, rows []*bqpb.StatusRow) error

func (*FakeClient) ReadMostRecentCreateTime

func (fc *FakeClient) ReadMostRecentCreateTime(ctx context.Context) (time.Time, error)