elasticsearch

README

Elasticsearch output

It sends events into Elasticsearch. It uses _bulk API to send events in batches. If a network error occurs, the batch will infinitely try to be delivered to the random endpoint.

Config params

endpoints []string required

The list of elasticsearch endpoints in the following format: SCHEMA://HOST:PORT

index_format string default=file-d-%

It defines the pattern of elasticsearch index name. Use % character as a placeholder. Use index_values to define values for the replacement. E.g. if index_format="my-index-%-%" and index_values="service,time" and event is {"service"="my-service"} then index for that event will be my-index-my-service-2020-01-05. First % replaced with service field of the event and the second replaced with current time(see time_format option)

index_values []string default=[@time]

A comma-separated list of event fields which will be used for replacement index_format. There is a special field time which equals the current time. Use the time_format to define a time format. E.g. [service, time]

time_format string default=2006-01-02

The time format pattern to use as value for the time placeholder.

Check out func Parse doc for details.

connection_timeout cfg.Duration default=5s

It defines how much time to wait for the connection.

workers_count cfg.Expression default=gomaxprocs*4

It defines how many workers will be instantiated to send batches.

batch_size cfg.Expression default=capacity/4

A maximum quantity of events to pack into one batch.

batch_flush_timeout cfg.Duration default=200ms

After this timeout batch will be sent even if batch isn't full.

Generated using insane-doc

Documentation

Index

• func Factory() (pipeline.AnyPlugin, pipeline.AnyConfig)
• type Config
• type Plugin
  • func (p *Plugin) Out(event *pipeline.Event)
  • func (p *Plugin) Start(config pipeline.AnyConfig, params *pipeline.OutputPluginParams)
  • func (p *Plugin) Stop()

Functions

func Factory

func Factory() (pipeline.AnyPlugin, pipeline.AnyConfig)

Types

type Config

type Config struct {
	//> @3@4@5@6
	//>
	//> The list of elasticsearch endpoints in the following format: `SCHEMA://HOST:PORT`
	Endpoints []string `json:"endpoints"  required:"true"` //*

	//> @3@4@5@6
	//>
	//> It defines the pattern of elasticsearch index name. Use `%` character as a placeholder. Use `index_values` to define values for the replacement.
	//> E.g. if `index_format="my-index-%-%"` and `index_values="service,@@time"` and event is `{"service"="my-service"}`
	//> then index for that event will be `my-index-my-service-2020-01-05`. First `%` replaced with `service` field of the event and the second
	//> replaced with current time(see `time_format` option)
	IndexFormat string `json:"index_format" default:"file-d-%"` //*

	//> @3@4@5@6
	//>
	//> A comma-separated list of event fields which will be used for replacement `index_format`.
	//> There is a special field `@@time` which equals the current time. Use the `time_format` to define a time format.
	//> E.g. `[service, @@time]`
	IndexValues []string `json:"index_values" default:"[@time]" slice:"true"` //*

	//> @3@4@5@6
	//>
	//> The time format pattern to use as value for the `@@time` placeholder.
	//> > Check out [func Parse doc](https://golang.org/pkg/time/#Parse) for details.
	TimeFormat string `json:"time_format" default:"2006-01-02"` //*

	//> @3@4@5@6
	//>
	//> It defines how much time to wait for the connection.
	ConnectionTimeout  cfg.Duration `json:"connection_timeout" default:"5s" parse:"duration"` //*
	ConnectionTimeout_ time.Duration

	//> @3@4@5@6
	//>
	//> It defines how many workers will be instantiated to send batches.
	WorkersCount  cfg.Expression `json:"workers_count" default:"gomaxprocs*4" parse:"expression"` //*
	WorkersCount_ int

	//> @3@4@5@6
	//>
	//> A maximum quantity of events to pack into one batch.
	BatchSize  cfg.Expression `json:"batch_size" default:"capacity/4"  parse:"expression"` //*
	BatchSize_ int

	//> @3@4@5@6
	//>
	//> After this timeout batch will be sent even if batch isn't full.
	BatchFlushTimeout  cfg.Duration `json:"batch_flush_timeout" default:"200ms" parse:"duration"` //*
	BatchFlushTimeout_ time.Duration
}

! config-params ^ config-params

type Plugin

type Plugin struct {
	// contains filtered or unexported fields
}

func (*Plugin) Out

func (p *Plugin) Out(event *pipeline.Event)

func (*Plugin) Start

func (p *Plugin) Start(config pipeline.AnyConfig, params *pipeline.OutputPluginParams)

func (*Plugin) Stop

func (p *Plugin) Stop()