athenadriver

package
v1.1.15 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 4, 2024 License: MIT Imports: 29 Imported by: 20

Documentation

Overview

Package athenadriver is a fully-featured Go database/sql driver for Amazon AWS Athena developed at Uber Technologies Inc.

It provides a hassle-free way of querying AWS Athena database with Go standard library. It not only provides basic features of Athena Go SDK, but addresses some of its limitation, improves and extends it.Except the basic features provided by Go database/sql like error handling, database pool and reconnection, athenadriver supports the following features out of box:

  • Support multiple AWS authorization methods
  • Full support of Athena Basic Data Types
  • Full support of Athena Advanced Type for queries with Geospatial identifiers, ML and UDFs
  • Full support of ALL Athena Query Statements, including DDL, DML and UTILITY
  • Support newly added INSERT INTO...VALUES
  • Full support of Athena Basic Data Types
  • Athena workgroup and tagging support including remote workgroup creation
  • Go sql's Prepared statement support
  • Go sql's DB.Exec() and db.ExecContext() support
  • Query cancelling support
  • Mask columns with specific values
  • Database missing value handling
  • Read-Only mode

Amazon Athena is an interactive query service that lets you use standard SQL to analyze data directly in Amazon S3. You can point Athena at your data in Amazon S3 and run ad-hoc queries and get results in seconds. Athena is serverless, so there is no infrastructure to set up or manage. You pay only for the queries you run. Athena scales automatically—executing queries in parallel—so results are fast, even with large datasets and complex queries. Author: Henry Fuheng Wu (wufuheng@gmail.com, henry.wu@uber.com)

Index

Constants

View Source 
const (
	// DriverName is the Name of this DB driver.
	DriverName = "awsathena"

	// DefaultBytesScannedCutoffPerQuery is 1G for every user.
	DefaultBytesScannedCutoffPerQuery = 1024 * 1024 * 1024

	// DefaultDBName is the default database name in Athena.
	DefaultDBName = "default"

	// DefaultWGName is the default workgroup name in Athena
	DefaultWGName = "primary"

	// DefaultRegion is the default region in Athena.
	DefaultRegion = "us-east-1"

	// TimestampUniXFormat is from https://docs.aws.amazon.com/athena/latest/ug/data-types.html.
	// https://stackoverflow.com/questions/20530327/origin-of-mon-jan-2-150405-mst-2006-in-golang
	// RFC3339 is not supported by AWS Athena. It uses session timezone!.
	TimestampUniXFormat = "2006-01-02 15:04:05.000"

	// ZeroDateTimeString is the invalid or zero result for a time.Time
	ZeroDateTimeString = "0001-01-01 00:00:00 +0000 UTC"

	// DateUniXFormat comes along the same way as TimestampUniXFormat.
	DateUniXFormat = "2006-01-02"

	// MetricsKey is the key for Metrics in context
	MetricsKey = TContextKey("MetricsKey")

	// LoggerKey is the key for Logger in context
	LoggerKey = TContextKey("LoggerKey")

	// DummyRegion is used when AWS CLI Config is used, ie AWS_SDK_LOAD_CONFIG is set
	DummyRegion = "dummy"

	// DummyAccessID is used when AWS CLI Config is used, ie AWS_SDK_LOAD_CONFIG is set
	DummyAccessID = "dummy"

	// DummySecretAccessKey is used when AWS CLI Config is used, ie AWS_SDK_LOAD_CONFIG is set
	DummySecretAccessKey = "dummy"
)
View Source 
const (
	// DDLQueryTimeout is DDL query timeout 600 minutes(unit second).
	DDLQueryTimeout = 600 * 60

	// DMLQueryTimeout is DML query timeout 30 minutes(unit second).
	DMLQueryTimeout = 30 * 60

	// PoolInterval is the interval between two status checks(unit second).
	PoolInterval = 3

	// The maximum allowed query string length is 262144 bytes,
	// where the strings are encoded in UTF-8.
	// This is not an adjustable quota. (unit bytes)
	MAXQueryStringLength = 262144
)

https://docs.aws.amazon.com/athena/latest/ug/service-limits.html

View Source 
const (
	// DebugLevel logs are typically voluminous, and are usually disabled in
	// production.
	DebugLevel = zap.DebugLevel
	// InfoLevel is the default logging priority.
	InfoLevel = zap.InfoLevel
	// WarnLevel logs are more important than Info, but don't need individual
	// human review.
	WarnLevel = zap.WarnLevel
	// ErrorLevel logs are high-priority. If an application is running smoothly,
	// it shouldn't generate any error-level logs.
	// DPanicLevel, PanicLevel and FatalLevel are not allowed in this package
	// to avoid terminating the whole process
	ErrorLevel = zap.ErrorLevel
)
View Source 
const DriverVersion = "1.1.15"

DriverVersion is athenadriver's version

View Source 
const PCGetDriverVersion = "get_driver_version"

PCGetDriverVersion is the pseudo command to get the version of athenadriver

View Source 
const PCGetQID = "get_query_id"

PCGetQID is the pseudo command of getting query execution id of an SQL

View Source 
const PCGetQIDStatus = "get_query_id_status"

PCGetQIDStatus is the pseudo command of getting status of a query execution id

View Source 
const PCStopQID = "stop_query_id"

PCStopQID is the pseudo command to stop a query execution id

View Source 
const Version = "0.1.0"

Version of the library.

Variables

View Source 
var (
	ErrInvalidQuery                 = errors.New("query is not valid")
	ErrConfigInvalidConfig          = errors.New("driver config is invalid")
	ErrConfigOutputLocation         = errors.New("output location must starts with s3")
	ErrConfigRegion                 = errors.New("region is required")
	ErrConfigWGPointer              = errors.New("workgroup pointer is nil")
	ErrConfigAccessIDRequired       = errors.New("AWS access ID is required")
	ErrConfigAccessKeyRequired      = errors.New("AWS access Key is required")
	ErrQueryUnknownType             = errors.New("query parameter type is unknown")
	ErrQueryBufferOF                = errors.New("query buffer overflow")
	ErrQueryTimeout                 = errors.New("query timeout")
	ErrAthenaTransactionUnsupported = errors.New("Athena doesn't support transaction statements")
	ErrAthenaNilDatum               = errors.New("*athena.Datum must not be nil")
	ErrAthenaNilAPI                 = errors.New("athenaAPI must not be nil")
	ErrTestMockGeneric              = errors.New("some_mock_error_for_test")
	ErrTestMockFailedByAthena       = errors.New("the reason why Athena failed the query")
	ErrServiceLimitOverride         = fmt.Errorf("service limit override must be greater than %d", PoolInterval)
)

Various errors the driver might return. Can change between driver versions.

View Source 
var AthenaColumnTypes = [...]string{"tinyint", "smallint", "integer", "bigint", "float", "real", "double",
	"json", "char", "varchar", "varbinary", "row", "string", "binary",
	"struct", "interval year to month", "interval day to second", "decimal",
	"ipaddress", "array", "map", "unknown", "boolean", "date", "time", "time with time zone",
	"timestamp with time zone", "timestamp", "weird_type"}

AthenaColumnTypes is a fixed array of Athena Column Types. An array isn't immutable by nature; you can't make it constant.

View Source 
var OutputFormats = [...]string{"csv", "html", "markdown", "table"}

OutputFormats are all the formats we can choose to print query result

View Source 
var OutputStyles = [...]string{"StyleDefault", "StyleBold", "StyleColoredBright", "StyleColoredDark",
	"StyleColoredBlackOnBlueWhite", "StyleColoredBlackOnCyanWhite", "StyleColoredBlackOnGreenWhite",
	"StyleColoredBlackOnMagentaWhite", "StyleColoredBlackOnYellowWhite", "StyleColoredBlackOnRedWhite",
	"StyleColoredBlueWhiteOnBlack", "StyleColoredCyanWhiteOnBlack", "StyleColoredGreenWhiteOnBlack",
	"StyleColoredMagentaWhiteOnBlack", "StyleColoredRedWhiteOnBlack", "StyleColoredYellowWhiteOnBlack",
	"StyleDouble", "StyleLight", "StyleRounded",
}

OutputStyles are all the styles we can choose to print query result

Functions

func ColsRowsToCSV 

func ColsRowsToCSV(rows *sql.Rows) string

ColsRowsToCSV is a convenient function to convert columns and rows of sql.Rows to CSV format.

func ColsToCSV 

func ColsToCSV(rows *sql.Rows) string

ColsToCSV is a convenient function to convert columns of sql.Rows to CSV format.

func GetDefaultWGConfig 

func GetDefaultWGConfig() *athena.WorkGroupConfiguration

GetDefaultWGConfig to create a default WorkGroupConfiguration.

func GetFromEnvVal 

func GetFromEnvVal(keys []string) string

GetFromEnvVal is to get environmental variable value by keys. The return value is from whichever key is set according to the order in the slice.

func GetTableNamesInQuery added in v1.1.5 

func GetTableNamesInQuery(query string) map[string]bool

GetTableNamesInQuery is a pessimistic function to return tables involved in query in format of DB.TABLE https://regoio.herokuapp.com/ https://golang.org/pkg/regexp/syntax/

func GetTidySQL added in v1.1.5 

func GetTidySQL(query string) string

GetTidySQL is to return a tidy SQL string

func IsQID added in v1.1.5 

func IsQID(q string) bool

IsQID is to check if a query string is a Query ID the hexadecimal Athena query ID like a44f8e61-4cbb-429a-b7ab-bea2c4a5caed https://aws.amazon.com/premiumsupport/knowledge-center/access-download-athena-query-results/

func NewWGConfig added in v1.1.1 

func NewWGConfig(bytesScannedCutoffPerQuery int64,
	enforceWorkGroupConfiguration bool,
	publishCloudWatchMetricsEnabled bool,
	requesterPaysEnabled bool,
	resultConfiguration *athena.ResultConfiguration) *athena.WorkGroupConfiguration

NewWGConfig to create a WorkGroupConfiguration.

func PrettyPrintCSV added in v1.1.11 

func PrettyPrintCSV(rows *sql.Rows)

PrettyPrintCSV is to print rows in CSV format with default style

func PrettyPrintFancy added in v1.1.11 

func PrettyPrintFancy(rows *sql.Rows)

PrettyPrintFancy is to print rows in table format with fancy style

func PrettyPrintMD added in v1.1.11 

func PrettyPrintMD(rows *sql.Rows)

PrettyPrintMD is to print rows in markdown format with default style

func PrettyPrintSQLColsRows added in v1.1.7 

func PrettyPrintSQLColsRows(rows *sql.Rows, style string, render string, page int)

PrettyPrintSQLColsRows is to print rows beautifully with header

func PrettyPrintSQLRows added in v1.1.7 

func PrettyPrintSQLRows(rows *sql.Rows, style string, render string, page int)

PrettyPrintSQLRows is to print rows beautifully

func RowsToCSV 

func RowsToCSV(rows *sql.Rows) string

RowsToCSV is to convert rows of sql.Rows to CSV format.

Types

type AthenaCache added in v1.1.11 

type AthenaCache interface {
	// SetQID is to put query -> QIDMetaData into cache
	SetQID(query string, data QIDMetaData)

	// GetQID is to get QIDMetaData from cache by query string
	GetQID(query string) QIDMetaData

	// GetQuery is to get query string from cache by QID
	GetQuery(QID string) string
}

AthenaCache is for Cached Query

type AthenaResult 

type AthenaResult struct {
	// contains filtered or unexported fields
}

AthenaResult is the result of an Athena query execution.

func (AthenaResult) LastInsertId 

func (a AthenaResult) LastInsertId() (int64, error)

LastInsertId returns the database's auto-generated ID after, for example, an INSERT into a table with primary key. For Athena, it is nil as Athena Go SDK doesn't support it.

func (AthenaResult) RowsAffected 

func (a AthenaResult) RowsAffected() (int64, error)

RowsAffected returns the number of rows affected by the query.

type AthenaTime 

type AthenaTime struct {
	Time  time.Time
	Valid bool
}

AthenaTime represents a time.Time value that can be null. The AthenaTime supports Athena's Date, Time and Timestamp data types, with or without time zone.

type Config 

type Config struct {
	// contains filtered or unexported fields
}

Config is for AWS Athena Driver Config. Be noted this is different from aws.Config.

func NewConfig 

func NewConfig(s string) (*Config, error)

NewConfig is to create Config from a string.

func NewDefaultConfig 

func NewDefaultConfig(outputBucket string, region string, accessID string,
	secretAccessKey string) (*Config, error)

NewDefaultConfig is to new a Config with some default values.

func NewNoOpsConfig 

func NewNoOpsConfig() *Config

NewNoOpsConfig is to create a noop version of driver Config WITHOUT credentials.

func (*Config) CheckColumnMasked 

func (c *Config) CheckColumnMasked(columnName string) (string, bool)

CheckColumnMasked is to check if a specific column has been masked by some value. https://stackoverflow.com/questions/30285169/replace-the-empty-or-null-value-with-specific-value-in-hive-query-result/30289503

func (*Config) GetAWSProfile added in v1.1.5 

func (c *Config) GetAWSProfile() string

GetAWSProfile is to get the credential provider name manually set by user

func (*Config) GetAccessID 

func (c *Config) GetAccessID() string

GetAccessID is a getter of AWS Access ID. It will try to get access ID from:

  1. string stored in c.values
  2. environmental variable ${AWS_ACCESS_KEY_ID} or ${AWS_ACCESS_KEY}

func (*Config) GetDB 

func (c *Config) GetDB() string

GetDB is getter of DB.

func (*Config) GetOutputBucket 

func (c *Config) GetOutputBucket() string

GetOutputBucket is getter of OutputBucket.

func (*Config) GetRegion 

func (c *Config) GetRegion() string

GetRegion is getter of Region.

func (*Config) GetResultPollIntervalSeconds added in v1.1.15 

func (c *Config) GetResultPollIntervalSeconds() time.Duration

GetResultPollIntervalSeconds is getter of resultPollIntervalSeconds.

func (*Config) GetSecretAccessKey 

func (c *Config) GetSecretAccessKey() string

GetSecretAccessKey is a getter of AWS Access Key.

func (*Config) GetServiceLimitOverride added in v1.1.13 

func (c *Config) GetServiceLimitOverride() *ServiceLimitOverride

GetServiceLimitOverride is to get the ServiceLimitOverride manually set by a user

func (*Config) GetSessionToken 

func (c *Config) GetSessionToken() string

GetSessionToken is a getter of AWS Session Token.

func (*Config) GetUser 

func (c *Config) GetUser() string

GetUser is getter of User.

func (*Config) GetWorkgroup 

func (c *Config) GetWorkgroup() Workgroup

GetWorkgroup is getter of Workgroup.

func (*Config) IsLoggingEnabled 

func (c *Config) IsLoggingEnabled() bool

IsLoggingEnabled is to check if driver level logging enabled.

func (*Config) IsMetricsEnabled 

func (c *Config) IsMetricsEnabled() bool

IsMetricsEnabled is to check if driver level metrics enabled.

func (*Config) IsMissingAsDefault 

func (c *Config) IsMissingAsDefault() bool

IsMissingAsDefault return true if missing value is set to be returned as default data.

func (*Config) IsMissingAsEmptyString 

func (c *Config) IsMissingAsEmptyString() bool

IsMissingAsEmptyString return true if missing value is set to be returned as empty string.

func (*Config) IsMissingAsNil added in v1.1.15 

func (c *Config) IsMissingAsNil() bool

IsMissingAsNil return true if missing value is set to be returned as nil.

func (*Config) IsMoneyWise added in v1.1.4 

func (c *Config) IsMoneyWise() bool

IsMoneyWise is to check if we are in the moneywise mode

func (*Config) IsReadOnly 

func (c *Config) IsReadOnly() bool

IsReadOnly is to check if only SELECT/SHOW/DESC are allowed

func (*Config) IsWGRemoteCreationAllowed 

func (c *Config) IsWGRemoteCreationAllowed() bool

IsWGRemoteCreationAllowed is to check if we are allowed to create workgroup with API from client.

func (*Config) SafeStringify 

func (c *Config) SafeStringify() string

SafeStringify is a secure version of Stringify(), with security information masked with *.

func (*Config) SetAWSProfile added in v1.1.5 

func (c *Config) SetAWSProfile(profile string)

SetAWSProfile is to manually set the credential provider https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html

func (*Config) SetAccessID 

func (c *Config) SetAccessID(o string) error

SetAccessID is a setter of AWS Access ID.

func (*Config) SetDB 

func (c *Config) SetDB(o string)

SetDB is a setter of DB.

func (*Config) SetLogging 

func (c *Config) SetLogging(b bool)

SetLogging is to set if driver level logging enabled.

func (*Config) SetMaskedColumnValue 

func (c *Config) SetMaskedColumnValue(columnName string, value string)

SetMaskedColumnValue is to set masked value for some column.

func (*Config) SetMetrics 

func (c *Config) SetMetrics(b bool)

SetMetrics is to set if driver level logging enabled.

func (*Config) SetMissingAsDefault 

func (c *Config) SetMissingAsDefault(b bool)

SetMissingAsDefault is to set if missing value is returned as default data.

func (*Config) SetMissingAsEmptyString 

func (c *Config) SetMissingAsEmptyString(b bool)

SetMissingAsEmptyString is to set if missing value is returned as empty string.

func (*Config) SetMissingAsNil added in v1.1.15 

func (c *Config) SetMissingAsNil(b bool)

SetMissingAsNil is to set if missing value is returned as nil.

func (*Config) SetMoneyWise added in v1.1.4 

func (c *Config) SetMoneyWise(b bool)

SetMoneyWise is to set if we are in the moneywise mode

func (*Config) SetOutputBucket 

func (c *Config) SetOutputBucket(o string) error

SetOutputBucket is to set S3 bucket for result set. On March 1, 2018, we updated our naming conventions for S3 buckets in the US East (N. Virginia) Region to match the naming conventions that we use in all other worldwide AWS Regions. Amazon S3 no longer supports creating bucket names that contain uppercase letters or underscores. https://docs.aws.amazon.com/AmazonS3/latest/dev/BucketRestrictions.html#bucketnamingrules

func (*Config) SetReadOnly 

func (c *Config) SetReadOnly(b bool)

SetReadOnly is to set if only SELECT/SHOW/DESC are allowed

func (*Config) SetRegion 

func (c *Config) SetRegion(o string) error

SetRegion is to set region.

func (*Config) SetResultPollIntervalSeconds added in v1.1.15 

func (c *Config) SetResultPollIntervalSeconds(n int)

SetResultPollIntervalSeconds is a setter of Overriding poll interval.

func (*Config) SetSecretAccessKey 

func (c *Config) SetSecretAccessKey(o string) error

SetSecretAccessKey is a setter of AWS Access Key.

func (*Config) SetServiceLimitOverride added in v1.1.13 

func (c *Config) SetServiceLimitOverride(serviceLimitOverride ServiceLimitOverride)

SetServiceLimitOverride is to set values from a ServiceLimitOverride

func (*Config) SetSessionToken 

func (c *Config) SetSessionToken(o string)

SetSessionToken is a setter of AWS Session Token.

func (*Config) SetUser 

func (c *Config) SetUser(o string)

SetUser is a setter of User.

func (*Config) SetWGRemoteCreationAllowed 

func (c *Config) SetWGRemoteCreationAllowed(b bool)

SetWGRemoteCreationAllowed is to set if we are allowed to create workgroup with API from client.

func (*Config) SetWorkGroup 

func (c *Config) SetWorkGroup(w *Workgroup) error

SetWorkGroup is a setter of WorkGroup.

func (*Config) String 

func (c *Config) String() string

String is to return the string form of DSN.

func (*Config) Stringify 

func (c *Config) Stringify() string

Stringify is to return the string form of DSN like JSON.stringify(). Please refer to: https://www.w3schools.com/js/js_json_stringify.asp

type Connection 

type Connection struct {
	// contains filtered or unexported fields
}

Connection is a connection to AWS Athena. It is not used concurrently by multiple goroutines. Connection is assumed to be stateful.

func (*Connection) Begin 

func (c *Connection) Begin() (driver.Tx, error)

Begin is from Conn interface, but no implementation for AWS Athena.

func (*Connection) BeginTx 

func (c *Connection) BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error)

BeginTx is to replace Begin as it is deprecated.

func (*Connection) CheckNamedValue 

func (c *Connection) CheckNamedValue(nv *driver.NamedValue) (err error)

CheckNamedValue is to implement interface driver.NamedValueChecker.

func (*Connection) Close 

func (c *Connection) Close() error

Close is from Conn interface, but no implementation for AWS Athena. Because the sql package maintains a free pool of connections and only calls Close when there's a surplus of idle connections, it shouldn't be necessary for drivers to do their own connection caching.

func (*Connection) ExecContext 

func (c *Connection) ExecContext(ctx context.Context, query string, namedArgs []driver.NamedValue) (driver.Result, error)

ExecContext executes a query that doesn't return rows, such as an INSERT or UPDATE.

func (*Connection) Ping 

func (c *Connection) Ping(ctx context.Context) error

Ping implements driver.Pinger interface. Ping is a good first step in a health check: If the Ping succeeds, make a simple query, then make a complex query which depends on proper DB scheme. This will make troubleshooting simpler as the error now is: "We've got network connectivity, we can Ping the DB, so we have valid credentials for a SELECT xxx; but ...".

func (*Connection) Prepare 

func (c *Connection) Prepare(query string) (driver.Stmt, error)

Prepare is inherited from Conn interface.

func (*Connection) QueryContext 

func (c *Connection) QueryContext(ctx context.Context, query string, namedArgs []driver.NamedValue) (driver.Rows, error)

QueryContext is implemented to be called by `DB.Query` (QueryerContext interface).

"QueryerContext is an optional interface that may be implemented by a Conn. If a Conn does not implement QueryerContext, the sql package's DB.Query will fall back to Queryer; if the Conn does not implement Queryer either, DB.Query will first prepare a query, execute the statement, and then close the statement."

With QueryContext implemented, we don't need Queryer. QueryerContext must honor the context timeout and return when the context is canceled.

type DriverTracer 

type DriverTracer struct {
	// contains filtered or unexported fields
}

DriverTracer is supported in athenadriver builtin.

func NewDefaultObservability added in v1.1.14 

func NewDefaultObservability(config *Config) *DriverTracer

NewDefaultObservability is to create an observability object with logger and scope as default(noops object).

func NewNoOpsObservability added in v1.1.14 

func NewNoOpsObservability() *DriverTracer

NewNoOpsObservability is for testing purpose.

func NewObservability added in v1.1.14 

func NewObservability(config *Config, logger *zap.Logger,
	scope tally.Scope) *DriverTracer

NewObservability is to create an observability object.

func (*DriverTracer) Config 

func (c *DriverTracer) Config() *Config

Config is to get c.config

func (*DriverTracer) Log 

func (c *DriverTracer) Log(lvl zapcore.Level, msg string, fields ...zap.Field)

Log is to log with zap.logger with 4 logging levels. We threw away the panic and fatal level as we don't want to DB error terminates the whole process.

func (*DriverTracer) Logger 

func (c *DriverTracer) Logger() *zap.Logger

Logger is a getter of logger.

func (*DriverTracer) Scope 

func (c *DriverTracer) Scope() tally.Scope

Scope is a getter of tally.Scope.

func (*DriverTracer) SetLogger 

func (c *DriverTracer) SetLogger(logger *zap.Logger)

SetLogger is a setter of logger.

func (*DriverTracer) SetScope 

func (c *DriverTracer) SetScope(scope tally.Scope)

SetScope is a setter of tally.Scope.

type QIDMetaData added in v1.1.11 

type QIDMetaData struct {
	QID string
	// contains filtered or unexported fields
}

QIDMetaData is the meta data for QID

type Rows 

type Rows struct {
	ResultOutput *athena.GetQueryResultsOutput
	// contains filtered or unexported fields
}

Rows defines rows in AWS Athena ResultSet.

func NewNonOpsRows added in v1.1.5 

func NewNonOpsRows(ctx context.Context, athenaAPI athenaiface.AthenaAPI, queryID string, driverConfig *Config,
	obs *DriverTracer) (*Rows, error)

NewNonOpsRows is to create a new Rows.

func NewRows 

func NewRows(ctx context.Context, athenaAPI athenaiface.AthenaAPI, queryID string, driverConfig *Config,
	obs *DriverTracer) (*Rows, error)

NewRows is to create a new Rows.

func (*Rows) Close 

func (r *Rows) Close() error

Close is to close Rows after reading all data.

func (*Rows) ColumnTypeDatabaseTypeName 

func (r *Rows) ColumnTypeDatabaseTypeName(index int) string

ColumnTypeDatabaseTypeName will be called by sql framework.

func (*Rows) Columns 

func (r *Rows) Columns() []string

Columns return Columns metadata.

func (*Rows) Next 

func (r *Rows) Next(dest []driver.Value) error

Next is to get next result set page.

type SQLConnector 

type SQLConnector struct {
	// contains filtered or unexported fields
}

SQLConnector is the connector for AWS Athena Driver.

func NoopsSQLConnector 

func NoopsSQLConnector() *SQLConnector

NoopsSQLConnector is to create a noops SQLConnector.

func (*SQLConnector) Connect 

func (c *SQLConnector) Connect(ctx context.Context) (driver.Conn, error)

Connect is to create an AWS session. The order to find auth information to create session is: 1. Manually set AWS profile in Config by calling config.SetAWSProfile(profileName) 2. AWS_SDK_LOAD_CONFIG 3. Static Credentials Ref: https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/configuring-sdk.html

func (*SQLConnector) Driver 

func (c *SQLConnector) Driver() driver.Driver

Driver is to construct a new SQLConnector.

type SQLDriver 

type SQLDriver struct {
	// contains filtered or unexported fields
}

SQLDriver is an implementation of sql/driver interface for AWS Athena. https://vyskocilm.github.io/blog/implement-sql-database-driver-in-100-lines-of-go/ https://golang.org/pkg/database/sql/driver/#Driver

func (*SQLDriver) Open 

func (d *SQLDriver) Open(dsn string) (driver.Conn, error)

Open returns a new connection to AWS Athena. The dsn is a string in a driver-specific format. the sql package maintains a pool of idle connections for efficient re-use. The returned connection is only used by one goroutine at a time.

func (*SQLDriver) OpenConnector 

func (d *SQLDriver) OpenConnector(dsn string) (driver.Connector, error)

OpenConnector will be called upon query execution. If a Driver implements DriverContext.OpenConnector, then sql.DB will call OpenConnector to obtain a Connector and then invoke that Connector's Conn method to obtain each needed connection, instead of invoking the Driver's Open method for each connection. The two-step sequence allows drivers to parse the name just once and also provides access to per-Conn contexts.

type ServiceLimitOverride added in v1.1.13 

type ServiceLimitOverride struct {
	// contains filtered or unexported fields
}

ServiceLimitOverride allows users to override service limits, hardcoded in constants.go. This assumes the service limits have been raised in the AWS account. https://docs.aws.amazon.com/athena/latest/ug/service-limits.html

func NewServiceLimitOverride added in v1.1.13 

func NewServiceLimitOverride() *ServiceLimitOverride

NewServiceLimitOverride is to create an empty ServiceLimitOverride. Values can be set using setters.

func (*ServiceLimitOverride) GetAsStringMap added in v1.1.13 

func (c *ServiceLimitOverride) GetAsStringMap() map[string]string

GetAsStringMap is to get the ServiceLimitOverride as a map of strings and aids in setting url.Values in Config

func (*ServiceLimitOverride) GetDDLQueryTimeout added in v1.1.13 

func (c *ServiceLimitOverride) GetDDLQueryTimeout() int

GetDDLQueryTimeout is to get the DDLQueryTimeout override.

func (*ServiceLimitOverride) GetDMLQueryTimeout added in v1.1.13 

func (c *ServiceLimitOverride) GetDMLQueryTimeout() int

GetDMLQueryTimeout is to get the DMLQueryTimeout override.

func (*ServiceLimitOverride) SetDDLQueryTimeout added in v1.1.13 

func (c *ServiceLimitOverride) SetDDLQueryTimeout(seconds int) error

SetDDLQueryTimeout is to set the DDLQueryTimeout override.

func (*ServiceLimitOverride) SetDMLQueryTimeout added in v1.1.13 

func (c *ServiceLimitOverride) SetDMLQueryTimeout(seconds int) error

SetDMLQueryTimeout is to set the DMLQueryTimeout override.

func (*ServiceLimitOverride) SetFromValues added in v1.1.13 

func (c *ServiceLimitOverride) SetFromValues(kvp url.Values)

SetFromValues is to set ServiceLimitOverride properties from a url.Values which might be a list of override and other ignored values from a dsn

type Statement 

type Statement struct {
	// contains filtered or unexported fields
}

Statement is to implement Go's database/sql Statement.

func (*Statement) Close 

func (s *Statement) Close() error

Close is to close an open statement.

func (*Statement) ColumnConverter 

func (s *Statement) ColumnConverter(idx int) driver.ValueConverter

ColumnConverter is to return driver's DefaultParameterConverter.

func (*Statement) Exec 

func (s *Statement) Exec(args []driver.Value) (driver.Result, error)

Exec is to execute a prepared statement.

func (*Statement) NumInput 

func (s *Statement) NumInput() int

NumInput returns the number of prepared arguments. It may also return -1, if the driver doesn't know its number of placeholders. In that case, the sql package will not sanity check Exec or Query argument counts. -- From Go `sql/driver`

func (*Statement) Query 

func (s *Statement) Query(args []driver.Value) (driver.Rows, error)

Query is to query based on a prepared statement.

type TContextKey 

type TContextKey string

TContextKey is a type for key in context.

type WGConfig 

type WGConfig struct {
	// contains filtered or unexported fields
}

WGConfig wraps WorkGroupConfiguration.

type WGTags 

type WGTags struct {
	// contains filtered or unexported fields
}

WGTags is a wrapper of []*athena.Tag.

func NewWGTags 

func NewWGTags() *WGTags

NewWGTags is to create a new WGTags.

func (*WGTags) AddTag 

func (t *WGTags) AddTag(k string, v string)

AddTag is to add tag.

func (*WGTags) Get 

func (t *WGTags) Get() []*athena.Tag

Get is a getter.

type Workgroup 

type Workgroup struct {
	Name   string
	Config *athena.WorkGroupConfiguration
	Tags   *WGTags
}

Workgroup is a wrapper of Athena Workgroup.

func NewDefaultWG 

func NewDefaultWG(name string, config *athena.WorkGroupConfiguration, tags *WGTags) *Workgroup

NewDefaultWG is to create new default Workgroup.

func NewWG 

func NewWG(name string, config *athena.WorkGroupConfiguration, tags *WGTags) *Workgroup

NewWG is to create a new Workgroup.

func (*Workgroup) CreateWGRemotely 

func (w *Workgroup) CreateWGRemotely(athenaService athenaiface.AthenaAPI) error

CreateWGRemotely is to create a Workgroup remotely.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.