redshift

README

redshift

Usage

source:
  name: redshift
  config:
    cluster_id: cluster_test
    db_name: testDB
    db_user: testUser
    aws_region: us-east-1
    exclude: secondaryDB

Inputs

Key	Value	Example	Description
cluster_id	string	cluster_test	cluster ID to access the redshift cluster	required
db_name	string	testDB	Default database name to access the redshift cluster	required
db_user	string	testUser	Database username to access the redshift cluster	required
aws_region	string	us-east-1	Aws region to access the redshift cluster	required
exclude	string	secondaryDB	This is a comma separated db list	optional

Outputs

Field	Sample Value
resource.urn	redshift::us-east-1/my_database/my_table
resource.name	my_table
resource.service	redshift
schema	[]Column

Column

Field	Sample Value
name	total_price
description	item's total price
data_type	decimal
is_nullable	true
length	1243

Contributing

Refer to the contribution guidelines for information on contributing to this module.

Documentation

Index

• type Config
• type Extractor
  • func New(logger log.Logger, opts ...Option) *Extractor
  • func (e *Extractor) Extract(_ context.Context, emit plugins.Emit) error
  • func (e *Extractor) GetColumn(dbName, tableName string) ([]*redshiftdataapiservice.ColumnMetadata, error)
  • func (e *Extractor) GetDBList() ([]string, error)
  • func (e *Extractor) GetTables(dbName string) ([]string, error)
  • func (e *Extractor) Init(ctx context.Context, config plugins.Config) error
• type Option
  • func WithClient(redshiftClient redshiftdataapiserviceiface.RedshiftDataAPIServiceAPI) Option

Types

type Config

type Config struct {
	ClusterID string `json:"cluster_id" yaml:"cluster_id" mapstructure:"cluster_id" validate:"required"`
	DBName    string `json:"db_name" yaml:"db_name" mapstructure:"db_name" validate:"required"`
	DBUser    string `json:"db_user" yaml:"db_user" mapstructure:"db_user" validate:"required"`
	AWSRegion string `json:"aws_region" yaml:"aws_region" mapstructure:"aws_region" validate:"required"`
	Exclude   string `json:"exclude" yaml:"exclude" mapstructure:"exclude"`
}

Config holds the set of configuration for the metabase extractor

type Extractor

type Extractor struct {
	plugins.BaseExtractor
	// contains filtered or unexported fields
}

Extractor manages the extraction of data from the redshift server

func New

func New(logger log.Logger, opts ...Option) *Extractor

New returns a pointer to an initialized Extractor Object

func (*Extractor) Extract

func (e *Extractor) Extract(_ context.Context, emit plugins.Emit) error

Extract collects metadata from the source. Metadata is collected through the emitter

func (*Extractor) GetColumn

func (e *Extractor) GetColumn(dbName, tableName string) ([]*redshiftdataapiservice.ColumnMetadata, error)

GetColumn returns the column metadata of particular table in a database

func (*Extractor) GetDBList

func (e *Extractor) GetDBList() ([]string, error)

GetDBList returns the list of databases in a cluster

func (*Extractor) GetTables

func (e *Extractor) GetTables(dbName string) ([]string, error)

GetTables return the list of tables name

func (*Extractor) Init

func (e *Extractor) Init(ctx context.Context, config plugins.Config) error

Init initializes the extractor

type Option

type Option func(*Extractor)

Option provides extension abstraction to Extractor constructor

func WithClient

func WithClient(redshiftClient redshiftdataapiserviceiface.RedshiftDataAPIServiceAPI) Option

WithClient assign custom client to the Extractor constructor