sync

Documentation

Overview

Package sync manages step that synchronize datasets between datasources

Index

• func Load(ctx context.Context, log *logrus.Entry, recipePath string, name string, ...) (priority uint, steps []common.Steper, err error)
• type DestinationConfig
• type FilterConfig
• type SourceConfig
• type Step
  • func (st *Step) Cancel(log *logrus.Entry)
  • func (st *Step) Do(ctx context.Context, log *logrus.Entry) error
  • func (st *Step) Finish(log *logrus.Entry)
  • func (st *Step) Init(ctx context.Context, log *logrus.Entry) error
  • func (st *Step) PostLoad(log *logrus.Entry, superseed map[string]string) (err error)
  • func (st *Step) ToSkip(ctx context.Context, log *logrus.Entry) (bool, error)

Functions

func Load

func Load(ctx context.Context, log *logrus.Entry, recipePath string, name string, nameIndex int, v *viper.Viper, dss datasource.Datasourcers, provider provider.Provider, force bool, dryRun bool, limitedTags []string) (priority uint, steps []common.Steper, err error)

Load data from step file using its viper representation a return priority and list of steps.

Types

type DestinationConfig

type DestinationConfig struct {
	Tags    []string
	Engines []string
	Types   []string
	Table   string
	Key     string
	Mode    string
	Queries []string
}

DestinationConfig type for destination contain all possible fields without verification.

type FilterConfig

type FilterConfig struct {
	Aparameters []string
	Mparameters map[string]string
	Type        string
}

FilterConfig type for filter contain all possible fields without verification.

type SourceConfig

type SourceConfig struct {
	Tags    []string
	Engines []string
	Types   []string
	Table   string
	Where   string
}

SourceConfig type for source contain all possible fields without verification.

type Step

type Step struct {
	Name string
	// contains filtered or unexported fields
}

Step informations.

func (*Step) Cancel

func (st *Step) Cancel(log *logrus.Entry)

Cancel manage the cancellation of the step.

func (*Step) Do

func (st *Step) Do(ctx context.Context, log *logrus.Entry) error

Do manage the runnning of the step.

func (*Step) Finish

func (st *Step) Finish(log *logrus.Entry)

Finish manage the finish of the step (called after all other step of the same priority has ended their Do).

func (*Step) Init

func (st *Step) Init(ctx context.Context, log *logrus.Entry) error

Init manage the initialization of the step.

func (*Step) PostLoad

func (st *Step) PostLoad(log *logrus.Entry, superseed map[string]string) (err error)

PostLoad modify the loaded step values with the values provided in the map in argument.

func (*Step) ToSkip

func (st *Step) ToSkip(ctx context.Context, log *logrus.Entry) (bool, error)

ToSkip return true if the step must be skipped.