bigquery

Documentation

Index

• type BqManager
  • func NewBigQueryManager(projectId string, isLocalEnv bool) *BqManager
  • func (bq *BqManager) CloseAll(ctx context.Context)
  • func (bq *BqManager) GetBigQueryWriteStream(ctx context.Context, datasetId, tableId string, message proto.Message, ...) (*managedwriter.ManagedStream, error)

Types

type BqManager

type BqManager struct {
	// contains filtered or unexported fields
}

BqManager manages BigQuery clients and multiple managed streams for different tables.

func NewBigQueryManager

func NewBigQueryManager(projectId string, isLocalEnv bool) *BqManager

NewBigQueryManager initializes and returns a new BqManager.

Parameters: - projectId: The GCP project ID. - isLocalEnv: A boolean indicating if the environment is local (used to determine authentication method).

Returns: - A pointer to the initialized BqManager.

func (*BqManager) CloseAll

func (bq *BqManager) CloseAll(ctx context.Context)

CloseAll closes all the BigQuery managed streams and the client.

This method ensures that both the managed streams and the client are closed gracefully. It handles cases where the stream or client might already be closed and suppresses logging for io.EOF errors.

Parameters: - ctx: The context for managing request deadlines, cancellation signals, and other request-scoped values.

func (*BqManager) GetBigQueryWriteStream

func (bq *BqManager) GetBigQueryWriteStream(ctx context.Context, datasetId, tableId string, message proto.Message, refreshStreamCache bool) (*managedwriter.ManagedStream, error)

GetBigQueryWriteStream initializes and returns the BigQuery managed stream for the given table and dataset.

This method manages stream caching, handles refreshing of streams, and initializes new streams if necessary.

Parameters: - ctx: The context for managing request deadlines, cancellation signals, and other request-scoped values. - datasetId: The ID of the BigQuery dataset. - tableId: The ID of the BigQuery table. - message: A Protobuf message to extract the schema descriptor. - refreshStreamCache: A boolean flag indicating if the stream should be refreshed.

Returns: - A pointer to the initialized BigQuery managed stream. - An error if the stream initialization fails.