opensearchexporter

README

OpenSearch Exporter

Status
Stability	development: logs
	alpha: traces
Distributions	contrib
Issues
Code Owners	@Aneurysm9, @MitchellGale, @MaxKsyunz, @YANG-DB

OpenSearch exporter supports sending OpenTelemetry signals as documents to OpenSearch.

The documents are sent using observability catalog schema.

Configuration options

Indexing Options

The Observability indices would follow the recommended pattern for immutable data stream ingestion using the data_stream concepts. Index pattern will follow the next naming template ss4o_{type}-{dataset}-{namespace}

• dataset (default=default) a user-provided label to classify source of telemetry. It is used to construct the name of the destination index or data stream.
• namespace (default=namespace) a user-provided label to group telemetry. It is used to construct the name of the destination index or data stream.

LogsIndex configures the index, index alias, or data stream name logs should be indexed in.

• logs_index a user-provided label to specify name of the destination index or data stream.

HTTP Connection Options

OpenSearch export supports standard HTTP client settings.

• http.endpoint (required) <url>:<port> of OpenSearch node to send data to.

TLS settings

Supports standard TLS settings as part of HTTP settings. See TLS Configuration/Client Settings.

Retry Options

• retry_on_failure: See retry_on_failure

Timeout Options

• timeout : See timeout

Bulk Indexer Options

• bulk_action (optional): the action for ingesting data. Only create and index are allowed here.

Example

extensions:
  basicauth/client:
  client_auth:
    username: username
    password: password
    
exporters:
  opensearch/trace:
    http:
      endpoint: https://opensearch.example.com:9200
      auth:
        authenticator: basicauth/client
# ······
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [opensearch/trace]
      processors: [batch]

Documentation

Overview

Package opensearchexporter contains an opentelemetry-collector exporter for OpenSearch.

Index

• func NewFactory() exporter.Factory
• type Config
  • func (cfg *Config) Validate() error
• type MappingMode
  • func (m MappingMode) String() string
• type MappingsSettings

Functions

func NewFactory

func NewFactory() exporter.Factory

NewFactory creates a factory for OpenSearch exporter.

Types

type Config

type Config struct {
	confighttp.ClientConfig        `mapstructure:"http"`
	configretry.BackOffConfig      `mapstructure:"retry_on_failure"`
	exporterhelper.TimeoutSettings `mapstructure:",squash"`
	MappingsSettings               `mapstructure:"mapping"`

	// The Observability indices would follow the recommended for immutable data stream ingestion pattern using
	// the data_stream concepts. See https://opensearch.org/docs/latest/dashboards/im-dashboards/datastream/
	// Index pattern will follow the next naming template ss4o_{type}-{dataset}-{namespace}
	Dataset   string `mapstructure:"dataset"`
	Namespace string `mapstructure:"namespace"`

	// LogsIndex configures the index, index alias, or data stream name logs should be indexed in.
	// https://opensearch.org/docs/latest/im-plugin/index/
	// https://opensearch.org/docs/latest/dashboards/im-dashboards/datastream/
	LogsIndex string `mapstructure:"logs_index"`

	// BulkAction configures the action for ingesting data. Only `create` and `index` are allowed here.
	// If not specified, the default value `create` will be used.
	BulkAction string `mapstructure:"bulk_action"`
}

Config defines configuration for OpenSearch exporter.

func (*Config) Validate

func (cfg *Config) Validate() error

Validate validates the opensearch server configuration.

type MappingMode

type MappingMode int

const (
	MappingSS4O MappingMode = iota
	MappingECS
	MappingFlattenAttributes
)

Enum values for MappingMode.

func (MappingMode) String

func (m MappingMode) String() string

type MappingsSettings

type MappingsSettings struct {
	// Mode configures the field mappings.
	// Supported modes are the following:
	//
	//   ss4o: exports logs in the Simple Schema for Observability standard.
	//   This mode is enabled by default.
	//   See: https://opensearch.org/docs/latest/observing-your-data/ss4o/
	//
	//   ecs: maps fields defined in the OpenTelemetry Semantic Conventions
	//   to the Elastic Common Schema.
	//   See: https://www.elastic.co/guide/en/ecs/current/index.html
	//
	//   flatten_attributes: uses the ECS mapping but flattens all resource and
	//   log attributes in the record to the top-level.
	Mode string `mapstructure:"mode"`

	// Additional field mappings.
	Fields map[string]string `mapstructure:"fields"`

	// File to read additional fields mappings from.
	File string `mapstructure:"file"`

	// Field to store timestamp in.  If not set uses the default @timestamp
	TimestampField string `mapstructure:"timestamp_field"`

	// Whether to store timestamp in Epoch miliseconds
	UnixTimestamp bool `mapstructure:"unix_timestamp"`

	// Try to find and remove duplicate fields
	Dedup bool `mapstructure:"dedup"`

	Dedot bool `mapstructure:"dedot"`
}