sqlds

package module
v3.0.0-...-890dad5 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 11, 2024 License: Apache-2.0 Imports: 21 Imported by: 0

README

Build Status

sqlds

sqlds stands for SQL Datasource.

Most SQL-driven datasources, like Postgres, MySQL, and MSSQL share extremely similar codebases.

The sqlds package is intended to remove the repetition of these datasources and centralize the datasource logic. The only thing that the datasources themselves should have to define is connecting to the database, and what driver to use, and the plugin frontend.

Usage

if err := datasource.Manage("my-datasource", datasourceFactory, datasource.ManageOpts{}); err != nil {
  log.DefaultLogger.Error(err.Error())
  os.Exit(1)
}

func datasourceFactory(ctx context.Context, s backend.DataSourceInstanceSettings) (instancemgmt.Instance, error) {
  ds := sqlds.NewDatasource(&myDatasource{})
  return ds.NewDatasource(ctx, s)
}

Standardization

Macros

The sqlds package defines a set of default macros:

  • $__timeFilter(time_column): Filters by timestamp using the query period. Resolves to: time >= '0001-01-01T00:00:00Z' AND time <= '0001-01-01T00:00:00Z'
  • $__timeFrom(time_column): Filters by timestamp using the start point of the query period. Resolves to time >= '0001-01-01T00:00:00Z'
  • $__timeTo(time_column): Filters by timestamp using the end point of the query period. Resolves to time <= '0001-01-01T00:00:00Z'
  • $__timeGroup(time_column, period): To group times based on a period. Resolves to (minute example): "datepart(year, time), datepart(month, time)'"
  • $__table: Returns the table configured in the query.
  • $__column: Returns the column configured in the query.

Documentation

Index

Constants

View Source 
const MockDataFolder = "/mock-data"

MockDataFolder is the default folder that will contain data files

Variables

View Source 
var (
	// ErrorNotImplemented is returned if the function is not implemented by the provided Driver (the Completable pointer is nil)
	ErrorNotImplemented = errors.New("not implemented")
	// ErrorWrongOptions when trying to parse Options with a invalid JSON
	ErrorWrongOptions = errors.New("error reading query options")
)
View Source 
var (
	ErrorMissingMultipleConnectionsConfig = PluginError(errors.New("received connection arguments but the feature is not enabled"))
	ErrorMissingDBConnection              = PluginError(errors.New("unable to get default db connection"))
	HeaderKey                             = "grafana-http-headers"
	// Deprecated: ErrorMissingMultipleConnectionsConfig should be used instead
	MissingMultipleConnectionsConfig = ErrorMissingMultipleConnectionsConfig
	// Deprecated: ErrorMissingDBConnection should be used instead
	MissingDBConnection = ErrorMissingDBConnection
)
View Source 
var (
	// ErrorBadDatasource is returned if the data source could not be asserted to the correct type (this should basically never happen?)
	ErrorBadDatasource = errors.New("type assertion to datasource failed")
	// ErrorJSON is returned when json.Unmarshal fails
	ErrorJSON = errors.New("error unmarshaling query JSON the Query Model")
	// ErrorQuery is returned when the query could not complete / execute
	ErrorQuery = errors.New("error querying the database")
	// ErrorTimeout is returned if the query has timed out
	ErrorTimeout = errors.New("query timeout exceeded")
	// ErrorNoResults is returned if there were no results returned
	ErrorNoResults = errors.New("no results returned from query")
)
View Source 
var (
	// ErrorBadArgumentCount is returned from macros when the wrong number of arguments were provided
	ErrorBadArgumentCount = errors.New("unexpected number of arguments")
)

Functions

func CreateMockData 

func CreateMockData(table string, folder string, csvData string) error

CreateData will create a "table" (csv file) in the data folder that can be queried with SQL

func CreateMockTable 

func CreateMockTable(table string, folder string) error

Create will create a "table" (csv file) in the data folder that can be queried with SQL

func DownstreamError 

func DownstreamError(err error, override ...bool) error

func ErrorSource 

func ErrorSource(err error) backend.ErrorSource

func Interpolate 

func Interpolate(driver Driver, query *Query) (string, error)

Interpolate returns an interpolated query string given a backend.DataQuery

func PluginError 

func PluginError(err error, override ...bool) error

func QueryDB 

func QueryDB(ctx context.Context, db Connection, converterGenerator *sqlutil.ConverterGenerator, converters []sqlutil.Converter, fillMode *data.FillMissing, query *Query, args ...interface{}) (data.Frames, error)

QueryDB sends the query to the connection and converts the rows to a dataframe.

Types

type Completable 

type Completable interface {
	Schemas(ctx context.Context, options Options) ([]string, error)
	Tables(ctx context.Context, options Options) ([]string, error)
	Columns(ctx context.Context, options Options) ([]string, error)
}

Completable will be used to autocomplete Tables Schemas and Columns for SQL languages

type Connection 

type Connection interface {
	Close() error
	Ping() error
	PingContext(ctx context.Context) error
	QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
}

Connection represents a SQL connection and is satisfied by the *sql.DB type For now, we only add the functions that we need / actively use. Some other candidates for future use could include the ExecContext and BeginTxContext functions

type Driver 

type Driver interface {
	// Connect connects to the database. It does not need to call `db.Ping()`
	Connect(context.Context, backend.DataSourceInstanceSettings, json.RawMessage) (*sql.DB, error)
	// Settings are read whenever the plugin is initialized, or after the data source settings are updated
	Settings(context.Context, backend.DataSourceInstanceSettings) DriverSettings
	Macros() Macros
	Converters() []sqlutil.Converter
	ConverterGenerator() *sqlutil.ConverterGenerator
}

Driver is a simple interface that defines how to connect to a backend SQL datasource Plugin creators will need to implement this in order to create a managed datasource

type DriverSettings 

type DriverSettings struct {
	Timeout        time.Duration
	FillMode       *data.FillMissing
	Retries        int
	Pause          int
	RetryOn        []string
	ForwardHeaders bool
	Errors         bool
}

type FormatQueryOption 

type FormatQueryOption uint32

FormatQueryOption defines how the user has chosen to represent the data 

const (
	// FormatOptionTimeSeries formats the query results as a timeseries using "WideToLong"
	FormatOptionTimeSeries FormatQueryOption = iota
	// FormatOptionTable formats the query results as a table using "LongToWide"
	FormatOptionTable
	// FormatOptionLogs sets the preferred visualization to logs
	FormatOptionLogs
	// FormatOptionsTrace sets the preferred visualization to trace
	FormatOptionTrace
)

type Macro 

type Macro struct {
	Name string
	Args []string
}

type MacroFunc 

type MacroFunc func(*Query, []string) (string, error)

MacroFunc defines a signature for applying a query macro Query macro implementations are defined by users / consumers of this package

type Macros 

type Macros map[string]MacroFunc

Macros is a list of MacroFuncs. The "string" key is the name of the macro function. This name has to be regex friendly. 

var DefaultMacros Macros = Macros{
	"timeFilter": macroTimeFilter,
	"timeFrom":   macroTimeFrom,
	"timeGroup":  macroTimeGroup,
	"timeTo":     macroTimeTo,
	"table":      macroTable,
	"column":     macroColumn,
}

type Options 

type Options map[string]string

Options are used to query schemas, tables and columns. They will be encoded in the request body (e.g. {"database": "mydb"})

func ParseOptions 

func ParseOptions(rawOptions json.RawMessage) (Options, error)

type Query 

type Query struct {
	RawSQL         string            `json:"rawSql"`
	Format         FormatQueryOption `json:"format"`
	ConnectionArgs json.RawMessage   `json:"connectionArgs"`

	RefID         string            `json:"-"`
	Interval      time.Duration     `json:"-"`
	TimeRange     backend.TimeRange `json:"-"`
	MaxDataPoints int64             `json:"-"`
	FillMissing   *data.FillMissing `json:"fillMode,omitempty"`

	// Macros
	Schema string `json:"schema,omitempty"`
	Table  string `json:"table,omitempty"`
	Column string `json:"column,omitempty"`
}

Query is the model that represents the query that users submit from the panel / queryeditor. For the sake of backwards compatibility, when making changes to this type, ensure that changes are only additive.

func GetQuery 

func GetQuery(query backend.DataQuery, headers http.Header, setHeaders bool) (*Query, error)

GetQuery returns a Query object given a backend.DataQuery using json.Unmarshal

func (*Query) WithSQL 

func (q *Query) WithSQL(query string) *Query

WithSQL copies the Query, but with a different RawSQL value. This is mostly useful in the Interpolate function, where the RawSQL value is modified in a loop

type QueryArgSetter 

type QueryArgSetter interface {
	SetQueryArgs(ctx context.Context, headers http.Header) []interface{}
}

QueryArgSetter is an additional interface that could be implemented by driver. This adds the ability to the driver to optionally set query args that are then sent down to the database.

type QueryMutator 

type QueryMutator interface {
	MutateQuery(ctx context.Context, req backend.DataQuery) (context.Context, backend.DataQuery)
}

QueryMutator is an additional interface that could be implemented by driver. This adds ability to the driver it can mutate query before run.

type Response 

type Response struct {
	// contains filtered or unexported fields
}

func NewResponse 

func NewResponse(res *backend.QueryDataResponse) *Response

func (*Response) Response 

func (r *Response) Response() *backend.QueryDataResponse

func (*Response) Set 

func (r *Response) Set(refID string, res backend.DataResponse)

type ResponseMutator 

type ResponseMutator interface {
	MutateResponse(ctx context.Context, res data.Frames) (data.Frames, error)
}

ResponseMutator is an additional interface that could be implemented by driver. This adds ability to the driver, so it can mutate a response from the driver before its returned to the client.

type SQLDatasource 

type SQLDatasource struct {
	Completable

	backend.CallResourceHandler
	CustomRoutes map[string]func(http.ResponseWriter, *http.Request)
	// Enabling multiple connections may cause that concurrent connection limits
	// are hit. The datasource enabling this should make sure connections are cached
	// if necessary.
	EnableMultipleConnections bool
	// contains filtered or unexported fields
}

func NewDatasource 

func NewDatasource(c Driver) *SQLDatasource

NewDatasource initializes the Datasource wrapper and instance manager

func (*SQLDatasource) CheckHealth 

func (ds *SQLDatasource) CheckHealth(ctx context.Context, req *backend.CheckHealthRequest) (*backend.CheckHealthResult, error)

CheckHealth pings the connected SQL database

func (*SQLDatasource) Dispose 

func (ds *SQLDatasource) Dispose()

Dispose cleans up datasource instance resources. Note: Called when testing and saving a datasource

func (*SQLDatasource) DriverSettings 

func (ds *SQLDatasource) DriverSettings() DriverSettings

func (*SQLDatasource) GetDBFromQuery 

func (ds *SQLDatasource) GetDBFromQuery(ctx context.Context, q *Query, datasourceUID string) (*sql.DB, error)

func (*SQLDatasource) NewDatasource 

func (ds *SQLDatasource) NewDatasource(ctx context.Context, settings backend.DataSourceInstanceSettings) (instancemgmt.Instance, error)

NewDatasource creates a new `SQLDatasource`. It uses the provided settings argument to call the ds.Driver to connect to the SQL server

func (*SQLDatasource) QueryData 

func (ds *SQLDatasource) QueryData(ctx context.Context, req *backend.QueryDataRequest) (*backend.QueryDataResponse, error)

QueryData creates the Responses list and executes each query

type SQLMock 

type SQLMock struct {
	// contains filtered or unexported fields
}

SQLMock connects to a local folder with csv files

func (*SQLMock) Connect 

func (h *SQLMock) Connect(_ context.Context, _ backend.DataSourceInstanceSettings, msg json.RawMessage) (*sql.DB, error)

Connect opens a sql.DB connection using datasource settings

func (*SQLMock) Converters 

func (h *SQLMock) Converters() []sqlutil.Converter

Converters defines list of string convertors

func (*SQLMock) Macros 

func (h *SQLMock) Macros() Macros

Macros returns list of macro functions convert the macros of raw query

func (*SQLMock) Settings 

func (h *SQLMock) Settings(_ context.Context, _ backend.DataSourceInstanceSettings) DriverSettings

Source Files

View all Source files

Directories

Path Synopsis
csv

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.