schema

package
v0.12.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 27, 2022 License: MPL-2.0 Imports: 20 Imported by: 344

Documentation

Index

Constants

View Source 
const (
	Postgres = DialectType("postgres")
	TSDB     = DialectType("timescale")
)
View Source 
const FetchIdMetaKey = "cq_fetch_id"

Variables

This section is empty.

Functions

func DeleteParentIdFilter added in v0.3.0 

func DeleteParentIdFilter(id string) func(meta ClientMeta, parent *Resource) []interface{}

DeleteParentIdFilter is mostly used for table relations to delete table data based on parent's cq_id

func ParentIdResolver 

func ParentIdResolver(_ context.Context, _ ClientMeta, r *Resource, c Column) error

ParentIdResolver resolves the cq_id from the parent if you want to reference the parent's primary keys use ParentResourceFieldResolver as required.

func ValidateTable added in v0.2.0 

func ValidateTable(t *Table) error

Types

type ClientIdentifier added in v0.9.2 

type ClientIdentifier interface {
	Identify() string
}

type ClientMeta 

type ClientMeta interface {
	Logger() hclog.Logger
}

type Column 

type Column struct {
	// Name of column
	Name string
	// Value Type of column i.e String, UUID etc'
	Type ValueType
	// Description about column, this description is added as a comment in the database
	Description string
	// Column Resolver allows to set you own data based on resolving this can be an API call or setting multiple embedded values etc'
	Resolver ColumnResolver
	// Creation options allow modifying how column is defined when table is created
	CreationOptions ColumnCreationOptions
	// IgnoreInTests is used to skip verifying the column is non-nil in integration tests.
	// By default, integration tests perform a fetch for all resources in cloudquery's test account, and
	// verify all columns are non-nil.
	// If IgnoreInTests is true, verification is skipped for this column.
	// Used when it is hard to create a reproducible environment with this column being non-nil (e.g. various error columns).
	IgnoreInTests bool
	// contains filtered or unexported fields
}

Column definition for Table

func SetColumnMeta added in v0.5.1 

func SetColumnMeta(c Column, m *ColumnMeta) Column

func (Column) Internal added in v0.7.0 

func (c Column) Internal() bool

func (Column) Meta added in v0.5.1 

func (c Column) Meta() *ColumnMeta

func (Column) ValidateType 

func (c Column) ValidateType(v interface{}) error

type ColumnCreationOptions 

type ColumnCreationOptions struct {
	Unique  bool
	NotNull bool
}

ColumnCreationOptions allow modification of how column is defined when table is created

type ColumnList added in v0.7.0 

type ColumnList []Column

func (ColumnList) Get added in v0.7.0 

func (c ColumnList) Get(name string) *Column

func (ColumnList) Names added in v0.7.0 

func (c ColumnList) Names() []string

func (ColumnList) Sift added in v0.7.2 

func (c ColumnList) Sift() (providerCols ColumnList, internalCols ColumnList)

Sift gets a column list and returns a list of provider columns, and another list of internal columns, cqId column being the very last one

type ColumnMeta added in v0.5.1 

type ColumnMeta struct {
	Resolver     *ResolverMeta
	IgnoreExists bool
}

type ColumnResolver 

type ColumnResolver func(ctx context.Context, meta ClientMeta, resource *Resource, c Column) error

ColumnResolver is called for each row received in TableResolver's data fetch. execution holds all relevant information regarding execution as well as the Column called. resource holds the current row we are resolving the column for.

func DateResolver added in v0.3.2 

func DateResolver(path string, rfcs ...string) ColumnResolver

DateResolver resolves the different date formats (ISODate - 2011-10-05T14:48:00.000Z is default) into *time.Time

Examples: DateResolver("Date") - resolves using RFC.RFC3339 as default DateResolver("InnerStruct.Field", time.RFC822) - resolves using time.RFC822 DateResolver("InnerStruct.Field", time.RFC822, "2011-10-05") - resolves using a few resolvers one by one

func DateUTCResolver added in v0.3.2 

func DateUTCResolver(path string, rfcs ...string) ColumnResolver

DateUTCResolver resolves the different date formats (ISODate - 2011-10-05T14:48:00.000Z is default) into *time.Time and converts the date to utc timezone

Examples: DateUTCResolver("Date") - resolves using RFC.RFC3339 as default DateUTCResolver("InnerStruct.Field", time.RFC822) - resolves using time.RFC822 DateUTCResolver("InnerStruct.Field", time.RFC822, "2011-10-05") - resolves using a few resolvers one by one

func IPAddressResolver added in v0.3.2 

func IPAddressResolver(path string) ColumnResolver

IPAddressResolver resolves the ip string value and returns net.IP

Examples: IPAddressResolver("IP")

func IPAddressesResolver added in v0.5.2 

func IPAddressesResolver(path string) ColumnResolver

IPAddressesResolver resolves the ip string value and returns net.IP

Examples: IPAddressesResolver("IP")

func IPNetResolver added in v0.3.2 

func IPNetResolver(path string) ColumnResolver

IPNetResolver resolves the network string value and returns net.IPNet

Examples: IPNetResolver("Network")

func IntResolver added in v0.3.2 

func IntResolver(path string) ColumnResolver

IntResolver tries to cast value into int

Examples: IntResolver("Id")

func MACAddressResolver added in v0.3.2 

func MACAddressResolver(path string) ColumnResolver

MACAddressResolver resolves the mac string value and returns net.HardwareAddr

Examples: MACAddressResolver("MAC")

func ParentPathResolver added in v0.3.0 

func ParentPathResolver(path string) ColumnResolver

ParentPathResolver resolves a field from the parent

func ParentResourceFieldResolver added in v0.3.0 

func ParentResourceFieldResolver(name string) ColumnResolver

ParentResourceFieldResolver resolves a field from the parent's resource, the value is expected to be set if name isn't set the field will be set to null

func PathResolver 

func PathResolver(path string) ColumnResolver

PathResolver resolves a field in the Resource.Item

Examples: PathResolver("Field") PathResolver("InnerStruct.Field") PathResolver("InnerStruct.InnerInnerStruct.Field")

func StringResolver added in v0.3.2 

func StringResolver(path string) ColumnResolver

StringResolver tries to cast value into string

Examples: StringResolver("Id")

func UUIDResolver added in v0.3.2 

func UUIDResolver(path string) ColumnResolver

UUIDResolver resolves the uuid string value and returns uuid.UUID

Examples: UUIDResolver("Resource.UUID")

type Dialect added in v0.7.0 

type Dialect interface {
	// PrimaryKeys returns the primary keys of table according to dialect
	PrimaryKeys(t *Table) []string

	// Columns returns the columns of table according to dialect
	Columns(t *Table) ColumnList

	// Constraints returns constraint definitions for table, according to dialect
	Constraints(t, parent *Table) []string

	// Extra returns additional definitions for table outside the CREATE TABLE statement, according to dialect
	Extra(t, parent *Table) []string

	// DBTypeFromType returns the database type from the given ValueType. Always lowercase.
	DBTypeFromType(v ValueType) string

	// GetResourceValues will return column values from the resource, ready to go in pgx.CopyFromSlice
	GetResourceValues(r *Resource) ([]interface{}, error)
}

func GetDialect added in v0.7.0 

func GetDialect(t DialectType) (Dialect, error)

GetDialect creates and returns a dialect specified by the DialectType

type DialectType added in v0.7.0 

type DialectType string

func (DialectType) MigrationDirectory added in v0.7.0 

func (t DialectType) MigrationDirectory() string

type IgnoreErrorFunc 

type IgnoreErrorFunc func(err error) bool

IgnoreErrorFunc checks if returned error from table resolver should be ignored.

type LengthTableValidator added in v0.2.0 

type LengthTableValidator struct{}

func (LengthTableValidator) Validate added in v0.2.0 

func (LengthTableValidator) Validate(t *Table) error

type Meta added in v0.7.0 

type Meta struct {
	LastUpdate time.Time `json:"last_updated"`
	FetchId    string    `json:"fetch_id,omitempty"`
}

type PostgresDialect added in v0.7.0 

type PostgresDialect struct{}

func (PostgresDialect) Columns added in v0.7.0 

func (PostgresDialect) Columns(t *Table) ColumnList

func (PostgresDialect) Constraints added in v0.7.0 

func (d PostgresDialect) Constraints(t, parent *Table) []string

func (PostgresDialect) DBTypeFromType added in v0.7.0 

func (PostgresDialect) DBTypeFromType(v ValueType) string

func (PostgresDialect) Extra added in v0.7.0 

func (PostgresDialect) Extra(_, _ *Table) []string

func (PostgresDialect) GetResourceValues added in v0.7.0 

func (d PostgresDialect) GetResourceValues(r *Resource) ([]interface{}, error)

func (PostgresDialect) PrimaryKeys added in v0.7.0 

func (PostgresDialect) PrimaryKeys(t *Table) []string

type ResolverMeta added in v0.5.1 

type ResolverMeta struct {
	Name    string
	Builtin bool
}

type Resource 

type Resource struct {
	// Original resource item that wa from prior resolve
	Item interface{}
	// Set if this is an embedded table
	Parent *Resource
	// contains filtered or unexported fields
}

Resource represents a row in it's associated table, it carries a reference to the original item, and automatically generates an Id based on Table's Columns. Resource data can be accessed by the Get and Set methods

func NewResourceData 

func NewResourceData(dialect Dialect, t *Table, parent *Resource, item interface{}, metadata map[string]interface{}, startTime time.Time) *Resource

func (*Resource) GenerateCQId added in v0.3.0 

func (r *Resource) GenerateCQId() error

func (*Resource) Get 

func (r *Resource) Get(key string) interface{}

func (Resource) GetMeta added in v0.8.7 

func (r Resource) GetMeta(key string) (interface{}, bool)

func (*Resource) Id 

func (r *Resource) Id() uuid.UUID

func (*Resource) PrimaryKeyValues added in v0.7.0 

func (r *Resource) PrimaryKeyValues() []string

func (*Resource) Set 

func (r *Resource) Set(key string, value interface{}) error

func (*Resource) TableName added in v0.7.0 

func (r *Resource) TableName() string

func (*Resource) Values 

func (r *Resource) Values() ([]interface{}, error)

type Resources added in v0.3.0 

type Resources []*Resource

func (Resources) ColumnNames added in v0.3.0 

func (rr Resources) ColumnNames() []string

func (Resources) GetIds added in v0.3.0 

func (rr Resources) GetIds() []uuid.UUID

func (Resources) TableName added in v0.3.0 

func (rr Resources) TableName() string

type RowResolver 

type RowResolver func(ctx context.Context, meta ClientMeta, resource *Resource) error

type TSDBDialect added in v0.7.0 

type TSDBDialect struct {
	// contains filtered or unexported fields
}

func (TSDBDialect) Columns added in v0.7.0 

func (TSDBDialect) Columns(t *Table) ColumnList

func (TSDBDialect) Constraints added in v0.7.0 

func (d TSDBDialect) Constraints(t, _ *Table) []string

func (TSDBDialect) DBTypeFromType added in v0.7.0 

func (d TSDBDialect) DBTypeFromType(v ValueType) string

func (TSDBDialect) Extra added in v0.7.0 

func (TSDBDialect) Extra(t, parent *Table) []string

func (TSDBDialect) GetResourceValues added in v0.7.0 

func (d TSDBDialect) GetResourceValues(r *Resource) ([]interface{}, error)

func (TSDBDialect) PrimaryKeys added in v0.7.0 

func (d TSDBDialect) PrimaryKeys(t *Table) []string

type Table 

type Table struct {
	// Name of table
	Name string
	// table description
	Description string
	// Columns are the set of fields that are part of this table
	Columns ColumnList
	// Relations are a set of related tables defines
	Relations []*Table
	// Resolver is the main entry point to fetching table data and
	Resolver TableResolver
	// Ignore errors checks if returned error from table resolver should be ignored.
	IgnoreError IgnoreErrorFunc
	// Multiplex returns re-purposed meta clients. The sdk will execute the table with each of them
	Multiplex func(meta ClientMeta) []ClientMeta
	// DeleteFilter returns a list of key/value pairs to add when truncating this table's data from the database.
	DeleteFilter func(meta ClientMeta, parent *Resource) []interface{}
	// Post resource resolver is called after all columns have been resolved, and before resource is inserted to database.
	PostResourceResolver RowResolver
	// Options allow modification of how the table is defined when created
	Options TableCreationOptions
	// AlwaysDelete will always delete table data on fetch regardless if delete is disabled on run,
	// use this only in specific cases, if you are unsure contact the CloudQuery Team.
	AlwaysDelete bool

	// IgnoreInTests is used to exclude a table from integration tests.
	// By default, integration tests fetch all resources from cloudquery's test account, and verifY all tables
	// have at least one row.
	// When IgnoreInTests is true, integration tests won't fetch from this table.
	// Used when it is hard to create a reproducible environment with a row in this table.
	IgnoreInTests bool
	// Global tables are usually the same regardless of the provider fetch configuration. Global table data gets fetched
	// and doesn't produce PK conflict errors instead data is replaced
	Global bool

	// Serial is used to force a signature change, which forces new table creation and cascading removal of old table and relations
	Serial string
}

func (Table) Column 

func (t Table) Column(name string) *Column

func (Table) Signature added in v0.10.0 

func (t Table) Signature(d Dialect) string

Signature returns a comparable string about the structure of the table (columns, options, relations)

func (Table) TableNames added in v0.10.0 

func (t Table) TableNames() []string

type TableCreationOptions added in v0.3.0 

type TableCreationOptions struct {
	// List of columns to set as primary keys. If this is empty, a random unique ID is generated.
	PrimaryKeys []string
}

TableCreationOptions allow modifying how table is created such as defining primary keys, indices, foreign keys and constraints.

type TableResolver 

type TableResolver func(ctx context.Context, meta ClientMeta, parent *Resource, res chan<- interface{}) error

TableResolver is the main entry point when a table fetch is called.

Table resolver has 3 main arguments: - meta(ClientMeta): is the client returned by the plugin.Provider Configure call - parent(Resource): resource is the parent resource in case this table is called via parent table (i.e. relation) - res(chan interface{}): is a channel to pass results fetched by the TableResolver

type TableValidator added in v0.2.0 

type TableValidator interface {
	Validate(t *Table) error
}

type ValueType 

type ValueType int
const (
	TypeInvalid ValueType = iota
	TypeBool
	TypeSmallInt
	TypeInt
	TypeBigInt
	TypeFloat
	TypeUUID
	TypeString
	TypeByteArray
	TypeStringArray
	TypeIntArray
	TypeTimestamp
	TypeJSON
	TypeUUIDArray
	TypeInet
	TypeInetArray
	TypeCIDR
	TypeCIDRArray
	TypeMacAddr
	TypeMacAddrArray
)

func ValueTypeFromString 

func ValueTypeFromString(s string) ValueType

ValueTypeFromString this function is mainly used by https://github.com/cloudquery/cq-gen

func (ValueType) String 

func (v ValueType) String() string

Source Files

View all Source files

Directories

Path Synopsis
Package mock is a generated GoMock package.
 Package mock is a generated GoMock package.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.