query

Documentation

Index

• type BigqueryExecutor
  • func NewBigqueryExecutor(client bqiface.Client, store protocol.BigqueryJobStore, ...) *BigqueryExecutor
  • func (qe *BigqueryExecutor) Run(profile *job.Profile, query string, queryType job.QueryType) ([]protocol.Row, error)
• type SQLExpressionFactory
  • func NewSQLExpressionFactory(metadataStore protocol.MetadataStore) *SQLExpressionFactory
  • func (p *SQLExpressionFactory) CreatePartitionExpression(urn string) (string, error)

Types

type BigqueryExecutor

type BigqueryExecutor struct {
	// contains filtered or unexported fields
}

BigqueryExecutor to execute query in bigquery data warehouse

func NewBigqueryExecutor

func NewBigqueryExecutor(client bqiface.Client,
	store protocol.BigqueryJobStore,
	profileStore protocol.ProfileStore,
	statsFactory stats.ClientBuilder) *BigqueryExecutor

NewBigqueryExecutor constructor of BigqueryExecutor

func (*BigqueryExecutor) Run

func (qe *BigqueryExecutor) Run(profile *job.Profile, query string, queryType job.QueryType) ([]protocol.Row, error)

Run executes query and log the bigquery job progress and information to a storage

type SQLExpressionFactory

type SQLExpressionFactory struct {
	// contains filtered or unexported fields
}

SQLExpressionFactory create custom SQL expression

func NewSQLExpressionFactory

func NewSQLExpressionFactory(metadataStore protocol.MetadataStore) *SQLExpressionFactory

func (*SQLExpressionFactory) CreatePartitionExpression

func (p *SQLExpressionFactory) CreatePartitionExpression(urn string) (string, error)

CreatePartitionExpression create sql expression of __PARTITION__ macros that used on profile group only supported table with DATE and TIMESTAMP field partition or default _PARTITIONTIME partition field Here is the example of generated expression: When the data type is DATE

• time partitioning DAY date_column

• time partitioning MONTH DATE_TRUNC(date_column, MONTH)

• time partitioning YEAR DATE_TRUNC(date_column, YEAR)

* time partitioning HOUR is not supported

When the data type is TIMESTAMP

• time partitioning DAY DATE(ts_column, "UTC")

• timepartioning HOUR TIMESTAMP_TRUNC(ts_column, HOUR, "UTC")

• timepartitioning MONTH DATE_TRUNC(DATE(ts_column, "UTC"),MONTH)

• timepartitioning YEAR DATE_TRUNC(DATE(ts_column, "UTC"),YEAR)