typesystem

README

typesystem

Package typesystem enables Data Transfer to be compatible with different type system versions and provides definitions of a Data Transfer type system.

See CHANGELOG.md for details on what each fallback does.

A version of a type system is a set of rules of conversion of specific source database types to Data Transfer types, as well as rule of their conversion to specific target database types. Type system is thus only about heterogenous transfers.

This package consists of the typesystem and document methods which define typesystem rules (the mentioned rules of conversion) and the fallback facility methods.

Fallbacks enable users of older versions of transfers (with typesystem version, a property of the transfer, set to some value different from the "current" one) to convert types differently than the newly created transfers do. This feature enables to introduce some non-backwards-compatible changes to Data Transfer without any extra settings.

Use AddFallbackSource and AddFallbackTarget methods to add fallbacks. For example, a fallback for PostgreSQL source is added as follows:

// In file `pkg/dataagent/pg/fallback_date_as_string.go`
func init() {
    typesystem.AddFallbackSource(typesystem.Fallback{
        To:           1,
        ProviderType: abstract.ProviderTypePostgreSQL,
        Function: func(ci *abstract.ChangeItem) (*abstract.ChangeItem, error) {
            if !ci.IsRowEvent() {
                switch ci.Kind {
                case abstract.InitTableLoad, abstract.DoneTableLoad:
                    // perform fallback
                default:
                    return ci, nil
                }
            }

            for i := 0; i < len(ci.TableSchema); i++ {
                switch ci.TableSchema[i].DataType {
                case schema.TypeDate.String(), schema.TypeDatetime.String(), schema.TypeTimestamp.String():
                    ci.TableSchema[i].DataType = schema.TypeString.String()
                default:
                    // do nothing
                }
            }
            return ci, nil
        },
    })
}

// ...

// In `typesystem` package:
const LatestVersion int = 2 // previously was `1`

See function descriptions and actual use examples for extra details.

Each type system change (update) requires an increment of the global "latest" type system version constant, located inside this package.

Documentation

Index

• Constants
• Variables
• func AddFallbackSourceFactory(factory FallbackFactory)
• func AddFallbackTargetFactory(factory FallbackFactory)
• func BuildSourceDoc(typ abstract.ProviderType, title string) string
• func BuildTargetDoc(typ abstract.ProviderType, title string) string
• func Doc(typ abstract.ProviderType, title string) string
• func ProviderType(typ abstract.ProviderType) func(m model.EndpointParams) bool
• func SourceRules(provider abstract.ProviderType, rules map[schema.Type][]string)
• func SupportedTypes() []schema.Type
• func TargetRule(provider abstract.ProviderType, rules map[schema.Type]string)
• type Fallback
  • func (f Fallback) Applies(version int, provider model.EndpointParams) bool
  • func (f Fallback) String() string
• type FallbackFactory
• type Rule
  • func RuleFor(provider abstract.ProviderType) Rule

Constants

const (
	RestPlaceholder         = "REST..."
	NotSupportedPlaceholder = "N/A"
)

const LatestVersion int = model.LatestVersion

const NewTransfersVersion int = model.NewTransfersVersion

Variables

var FallbackDoesNotApplyErr = xerrors.NewSentinel("this fallback does not apply")

var SourceFallbackFactories = make([]FallbackFactory, 0)

var TargetFallbackFactories = make([]FallbackFactory, 0)

Functions

func AddFallbackSourceFactory

func AddFallbackSourceFactory(factory FallbackFactory)

AddFallbackSourceFactory registers a fallbacks for a source of some type

This method is expected to be called in the `init()` function of a module which introduces a fallback when some additional behaviour is expected

func AddFallbackTargetFactory

func AddFallbackTargetFactory(factory FallbackFactory)

AddFallbackTargetFactory registers a fallbacks for a target of some type

This method is expected to be called in the `init()` function of a module which introduces a fallback when some additional behaviour is expected

func BuildSourceDoc

func BuildSourceDoc(typ abstract.ProviderType, title string) string

func BuildTargetDoc

func BuildTargetDoc(typ abstract.ProviderType, title string) string

func Doc

func Doc(typ abstract.ProviderType, title string) string

func ProviderType

func ProviderType(typ abstract.ProviderType) func(m model.EndpointParams) bool

func SourceRules

func SourceRules(provider abstract.ProviderType, rules map[schema.Type][]string)

func SupportedTypes

func SupportedTypes() []schema.Type

func TargetRule

func TargetRule(provider abstract.ProviderType, rules map[schema.Type]string)

Types

type Fallback

type Fallback struct {
	// To is the target typesystem version of this fallback
	To int
	// Picker limits the endpoints to which this fallback applies
	Picker func(m model.EndpointParams) bool
	// Function defines the transformation. Input is an item of any kind.
	//
	// If a fallback does not apply, this method MUST return an error containing FallbackDoesNotApplyErr
	Function func(ci *abstract.ChangeItem) (*abstract.ChangeItem, error)
}

Fallback defines a transformation from a newer version of typesystem to an older one.

func (Fallback) Applies

func (f Fallback) Applies(version int, provider model.EndpointParams) bool

func (Fallback) String

func (f Fallback) String() string

type FallbackFactory

type FallbackFactory func() Fallback

type Rule

type Rule struct {
	// `Target` stores the mapping of `schema.Type`-s to provider-specific types for each `Target` provider.
	// example:
	//	{schema.TypeString: "TEXT"}
	Target map[schema.Type]string
	// `Source` stores the conversion rules from the provider-specific type to the `schema.Type` for each `Source` provider.
	// example:
	//	{"TINYTEXT": schema.TypeString}
	Source map[string]schema.Type
}

func RuleFor

func RuleFor(provider abstract.ProviderType) Rule