snowflake

README

snowflake

Usage

source:
  type: snowflake
  config:
    connection_url: user:password@my_organization-my_account/mydb
    exclude:
      databases:
        - database_a
        - database_b
      tables:
        - database_c.table_a

Inputs

Key	Value	Example	Description
connection_url	string	user:password@org22-acc123/mydb	URL to access the snowflake server	required
exclude.databases	[]string	[database_a, database_b]	List of databases to be excluded	optional
exclude.tables	[]string	[database_c.table_a, database_c.table_b]	List of tables to be excluded	optional

Outputs

Field	Sample Value
resource.urn	my_database.my_table
resource.name	my_table
resource.service	snowflake
description	table description
schema	[]column

Column

Field	Sample Value
name	total_price
description	item's total price
data_type	decimal
is_nullable	true
length	11

Contributing

Refer to the contribution guidelines for information on contributing to this module.

Documentation

Index

• type Config
• type Exclude
• type Extractor
  • func New(logger log.Logger, opts ...Option) *Extractor
  • func (e *Extractor) Extract(ctx context.Context, emit plugins.Emit) error
  • func (e *Extractor) Init(ctx context.Context, config plugins.Config) (err error)
• type Option
  • func WithHTTPTransport(htr http.RoundTripper) Option

Types

type Config

type Config struct {
	ConnectionURL string  `json:"connection_url" yaml:"connection_url" mapstructure:"connection_url" validate:"required"`
	Exclude       Exclude `json:"exclude" yaml:"exclude" mapstructure:"exclude"`
}

Config holds the connection URL for the extractor

type Exclude

type Exclude struct {
	Databases []string `json:"databases" yaml:"databases" mapstructure:"databases"`
	Tables    []string `json:"tables" yaml:"tables" mapstructure:"tables"`
}

type Extractor

type Extractor struct {
	plugins.BaseExtractor
	// contains filtered or unexported fields
}

Extractor manages the extraction of data from snowflake

func New

func New(logger log.Logger, opts ...Option) *Extractor

New returns a pointer to an initialized Extractor Object

func (*Extractor) Extract

func (e *Extractor) Extract(ctx context.Context, emit plugins.Emit) error

Extract collects metadata of the database through emitter

func (*Extractor) Init

func (e *Extractor) Init(ctx context.Context, config plugins.Config) (err error)

Init initializes the extractor

type Option

type Option func(*Extractor)

Option provides extension abstraction to Extractor constructor

func WithHTTPTransport

func WithHTTPTransport(htr http.RoundTripper) Option

WithHTTPTransport assign custom http client to the Extractor constructor