output

Documentation

Index

• type ElasticsearchIndexConfig
• type ElasticsearchOutput
  • func NewElasticsearchOutput(cfg ElasticsearchOutputConfig) *ElasticsearchOutput
  • func (o *ElasticsearchOutput) Create(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)
  • func (o *ElasticsearchOutput) Delete(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)
  • func (o *ElasticsearchOutput) Elements(repositories []gobulk.Repository, query interface{}, ...) ([]gobulk.Element, error)
  • func (o *ElasticsearchOutput) Repositories() []gobulk.Repository
  • func (o *ElasticsearchOutput) Setup() error
  • func (o *ElasticsearchOutput) Update(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)
• type ElasticsearchOutputConfig
• type FieldDescription
  • func (d *FieldDescription) ToMap() (map[string]interface{}, error)
• type GORMOutput
  • func NewGORMOutput(cfg GORMOutputConfig, relatedModels map[string]interface{}) GORMOutput
  • func (o *GORMOutput) Create(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)
  • func (o *GORMOutput) Delete(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)
  • func (o *GORMOutput) Elements(repositories []gobulk.Repository, query interface{}, ...) ([]gobulk.Element, error)
  • func (o *GORMOutput) Repositories() []gobulk.Repository
  • func (o *GORMOutput) Shutdown()
  • func (o *GORMOutput) Update(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)
• type GORMOutputConfig
• type GORMOutputQuery
• type MySQLOutput
  • func (o *MySQLOutput) Setup() error

Types

type ElasticsearchIndexConfig

type ElasticsearchIndexConfig struct {
	Settings map[string]interface{}
	Mappings map[string]interface{}
}

ElasticsearchIndexConfig represents the definition of configuration for a single index.

type ElasticsearchOutput

type ElasticsearchOutput struct {
	gobulk.BaseStorage
	Cfg ElasticsearchOutputConfig
	// contains filtered or unexported fields
}

ElasticsearchOutput represents an output that stores documents in Elasticsearch.

func NewElasticsearchOutput

func NewElasticsearchOutput(cfg ElasticsearchOutputConfig) *ElasticsearchOutput

NewElasticsearchOutput returns a new instance of the ElasticsearchOutput.

func (*ElasticsearchOutput) Create

func (o *ElasticsearchOutput) Create(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)

Create creates new operation data of output elements.

func (*ElasticsearchOutput) Delete

func (o *ElasticsearchOutput) Delete(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)

Delete removes operation data of existing output elementы.

func (*ElasticsearchOutput) Elements

func (o *ElasticsearchOutput) Elements(repositories []gobulk.Repository, query interface{}, unmarshal gobulk.UnmarshalOutputElement, expectedElementCount int) ([]gobulk.Element, error)

Elements provides the already existing elements of the output related to a given element (in most cases one or none, sometimes multiple).

func (*ElasticsearchOutput) Repositories

func (o *ElasticsearchOutput) Repositories() []gobulk.Repository

Repositories provides a list of all available repositories.

func (*ElasticsearchOutput) Setup

func (o *ElasticsearchOutput) Setup() error

Setup contains the storage preparations like connection etc. Is called only once at the very beginning of the work with the storage. As for the ElasticsearchOutput, it setups the internal client and indices.

func (*ElasticsearchOutput) Update

func (o *ElasticsearchOutput) Update(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)

Update modifies operation data of existing output elements.

type ElasticsearchOutputConfig

type ElasticsearchOutputConfig struct {
	// ServerURL is the ES server URL with protocol and port. E.g. https://my.es.instance:9200.
	ServerURL string `validate:"required,url"`
	// Indices represents indices that will be created in case they don't exist.
	// Base index config support: set Repository.Name to foo-base to make sure it's definitions will
	// be merged with all configs that have the name foo-*.
	Indices []gobulk.Repository
	// IndicesPath represents the path to a directory that contains *.json files with createIndex
	// payload (mappings and settings).
	// The file name will be used as index name.
	// If a file with foo-base.json exists, the settings and mappings of it will be merged into all
	// other foo-*.json index configurations.
	// The json files will be parsed and set to ElasticsearchOutput.Indices on setup.
	IndicesPath string
	// IndexSuffixes (prefix -> suffix) suffix will be appended to all index names that have a matching
	// prefix, this can be useful for versioning (foo-a-1, foo-b-1, ... can be later used as foo-*-1).
	IndexSuffixes map[string]string
}

ElasticsearchOutputConfig represents the ElasticsearchOutput configurable fields model.

type FieldDescription

type FieldDescription struct {
	Field   string `json:"-"`
	Type    string
	Null    string
	Key     string
	Default string
	Extra   string
}

FieldDescription represents a MySQL table field description.

func (*FieldDescription) ToMap

func (d *FieldDescription) ToMap() (map[string]interface{}, error)

ToMap returns the d representation as a map[string]interface.

type GORMOutput

type GORMOutput struct {
	gobulk.BaseStorage
	Cfg           GORMOutputConfig
	Client        *gorm.DB
	RelatedModels map[string]interface{}
	Tables        []gobulk.Repository
}

GORMOutput provides interface for general output operations using gorm library. The GORMOutput, while performing operations execition, doesn't use operations OutputIdentifier field. Instead, it relies on the passed data structures primary keys as the identifiers for rows to be created, updated or deleted. However, it's a good practice to have the OutputIdentifier field set for at least two reasons: 1) the GORMOutput then is easier to be replaced with other kind of gobulk.Output, because e.g. ElasticsearchOutput does use the OutputIdentifier field. 2) performed operations, no matter succeeded or issued, are tracked in the corresponding tracker repository, where you can see details of them and get to know what's gone wrong.

func NewGORMOutput

func NewGORMOutput(cfg GORMOutputConfig, relatedModels map[string]interface{}) GORMOutput

NewGORMOutput returns a new instance of the *GORMOutput. The relatedModels parameter represents a list of models (tables) that the output is going to use. The models map is supposed to be a map from a table name to its model as go structure. Example: map[string]interface{}{"users": User{}}

func (*GORMOutput) Create

func (o *GORMOutput) Create(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)

Create performs CREATE for all passed operations, creating operation.Data in the operation.OutputRepository table.

func (*GORMOutput) Delete

func (o *GORMOutput) Delete(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)

Delete performs DELETE for all passed operations, using the operations Data model primaryKey as the reference to the row to be updated and the operation.OutputRepository as the table name.

func (*GORMOutput) Elements

func (o *GORMOutput) Elements(repositories []gobulk.Repository, query interface{}, unmarshal gobulk.UnmarshalOutputElement, expectedElementCount int) ([]gobulk.Element, error)

Elements performs the query to the given list of repositories, transforms the result rows to elements using the unmarshal func, and returns the result list of elements.

The query parameter is expected to be of *GORMOutputQuery type, carrying the SQL condition and corresponding variables. Query application explanation: for all the given repositories: SELECT * FROM ${repository.Name} WHERE ${query.Condition}, ${query.Vars}. For the following result: SELECT * FROM users WHERE users.age > 40 AND users.name = "Kate", Use repositories = []gobulk.Repository{{Name: "users"}}, query = &output.GORMOutputQuery{Condition: "users.age > ? AND users.name = ?", Vars: []interface{}{40, "Kate"}}.

In the unmarshal func, you should consider the outputData to be of type map[string]interface{} with keys as the result field names and values as the actual field values. See the rowToMap GORMOutput method for more details.

func (*GORMOutput) Repositories

func (o *GORMOutput) Repositories() []gobulk.Repository

Repositories provides a list of all available repositories (tables).

func (*GORMOutput) Shutdown

func (o *GORMOutput) Shutdown()

Shutdown is called only once at the very end of the work with the storage. As for the GORMOutput, it closes the initially opened db connection.

func (*GORMOutput) Update

func (o *GORMOutput) Update(operations ...*gobulk.Operation) (*gobulk.OutputResponse, error)

Update performs UPDATE for all passed operations, creating operation.Data in the operation.OutputRepository table using the model primaryKey as the reference to the row to be updated.

type GORMOutputConfig

type GORMOutputConfig struct {
	Host     string           `validate:"required"`
	Database string           `validate:"required"`
	User     string           `validate:"required"`
	Password string           `validate:"required"`
	Port     string           `validate:"required"`
	Logger   logger.Interface `validate:"required"`
}

GORMOutputConfig represents the GORMOutput config structure.

type GORMOutputQuery

type GORMOutputQuery struct {
	Condition string
	Vars      []interface{}
}

GORMOutputQuery represents a structure containing parameters for GORMOutput queries. Example of the GORMOutputQuery usage: o.Client.Table(repository).Where(q.Condition, q.Vars...).

type MySQLOutput

type MySQLOutput struct {
	GORMOutput
}

MySQLOutput represents an output that stores documents to a MySQL database.

func (*MySQLOutput) Setup

func (o *MySQLOutput) Setup() error

Setup contains the storage preparations like connection etc. Is called only once at the very beginning of the work with the storage. As for the MySQLOutput, it tests the connection / read access of the tracker, prepares everything like migrations.