config

Documentation

Index

• Constants
• Variables
• func Validate(cfg Pipeline) error
• type Connector
• type DLQ
• type Parser
• type Pipeline
  • func Enrich(cfg Pipeline) Pipeline
• type Processor

Constants

const (
	StatusRunning   = "running"
	StatusStopped   = "stopped"
	TypeSource      = "source"
	TypeDestination = "destination"
)

Variables

var (
	ErrMandatoryField = cerrors.New("mandatory field not specified")
	ErrInvalidField   = cerrors.New("invalid field value")
	ErrDuplicateID    = cerrors.New("id already exists")
)

var (
	PipelineMutableFields = []string{"Name", "Description", "Connectors", "Processors", "DLQ"}
	PipelineIgnoredFields = []string{"Status"}

	ConnectorImmutableFields = []string{"Type", "Plugin"}
	ConnectorMutableFields   = []string{"Name", "Settings", "Processors"}

	ProcessorImmutableFields = []string{"Type"}
	ProcessorMutableFields   = []string{"Settings", "Workers", "Condition"}
)

Classify fields as immutable, mutable or ignored. This is used by the provisioning logic to identify if an entity needs to be updated, recreated or if no action needs to be taken in case a certain field is updated. All fields except "ID" need to be added to exactly one slice.

Functions

func Validate

func Validate(cfg Pipeline) error

Validate validates config field values for a pipeline

Types

type Connector

type Connector struct {
	ID         string
	Type       string
	Plugin     string
	Name       string
	Settings   map[string]string
	Processors []Processor
}

type DLQ

type DLQ struct {
	Plugin              string
	Settings            map[string]string
	WindowSize          *int
	WindowNackThreshold *int
}

type Parser

type Parser interface {
	Parse(ctx context.Context, reader io.Reader) ([]Pipeline, error)
}

Parser reads data from reader and parses all pipelines defined in the configuration.

type Pipeline

type Pipeline struct {
	ID          string
	Status      string
	Name        string
	Description string
	Connectors  []Connector
	Processors  []Processor
	DLQ         DLQ
}

func Enrich

func Enrich(cfg Pipeline) Pipeline

Enrich sets default values for pipeline config fields

type Processor

type Processor struct {
	ID        string
	Type      string
	Settings  map[string]string
	Workers   int
	Condition string
}