lokiexporter

README

Loki Exporter

Status
Stability	beta
Supported pipeline types	logs
Distributions	contrib

Exports data via HTTP to Loki.

Getting Started

The following settings are required:

• endpoint (no default): The target URL to send Loki log streams to (e.g.: http://loki:3100/loki/api/v1/push).

The following options are now deprecated:

• labels.{attributes/resource}. Deprecated and will be removed by v0.59.0. See the Labels section for more information.
• labels.record. Deprecated and will be removed by v0.59.0. See the Labels section for more information.
• tenant: Deprecated and will be removed by v0.59.0. See the Labels section for more information.
• format Deprecated without replacement. If you rely on this, let us know by opening an issue before v0.59.0 and we'll assist you in finding a solution.

Example:

receivers:
  otlp:

exporters:
  loki:
    endpoint: https://loki.example.com:3100/loki/api/v1/push

processors:
  attributes:
    actions:
    - action: insert
      key: event_domain
      from_attribute: event.domain
    - action: insert
      key: loki.attribute.labels
      value: event_domain

  resource:
    attributes:
    - action: insert
      key: service_name
      from_attribute: service.name
    - action: insert
      key: service_namespace
      from_attribute: service.namespace
    - action: insert
      key: loki.resource.labels
      value: service_name, service_namespace

extensions:

service:
  extensions:
  pipelines:
    logs:
      receivers: [otlp]
      processors: [resource, attributes]
      exporters: [loki]

The full list of settings exposed for this exporter are documented here with detailed sample configurations here.

More information on how to send logs to Grafana Loki using the OpenTelemetry Collector could be found here

Labels

The Loki exporter can convert OTLP resource and log attributes into Loki labels, which are indexed. For that, you need to configure hints, specifying which attributes should be placed as labels. The hints are themselves attributes and will be ignored when exporting to Loki. The following example uses the attributes processor to hint the Loki exporter to set the event.domain attribute as label and the resource processor to give a hint to the Loki exporter to set the service.name as label.

processors:
  attributes:
    actions:
      - action: insert
        key: event_domain
        from_attribute: event.domain
      - action: insert
        key: loki.attribute.labels
        value: event_domain

  resource:
    attributes:
      - action: insert
        key: service_name
        from_attribute: service.name
      - action: insert
        key: loki.resource.labels
        value: service_name

Currently, Loki does not support labels with dots. That’s why to add Loki label based on event.domain OTLP attribute we need to specify two actions. The first one inserts a new attribute event_domain from the OTLP attribute event.domain. The second one is a hint for Loki, specifying that the event_domain attribute should be placed as a Loki label. The same approach is applicable to placing Loki labels from resource attribute service.name.

Tenant information

It is recommended to use the header_setter extension to configure the tenant information to send to Loki. In case a static tenant should be used, you can make use of the headers option for regular HTTP client settings, like the following:

exporters:
  loki:
    endpoint: http://localhost:3100/loki/api/v1/push
    headers:
      "X-Scope-OrgID": acme

It is also possible to provide the loki.tenant attribute hint that specifies which resource or log attributes value should be used as a tenant. For example:

processors:
  resource:
    attributes:
    - action: insert
      key: loki.tenant
      value: host_name
    - action: insert
      key: host_name
      from_attribute: host.name

In this case the value of the host.name resource attribute is used to group logs by tenant and send requests with the X-Scope-OrgID header set to relevant tenants.

If the loki.tenant hint attribute is present in both resource or log attributes, then the look-up for a tenant value from resource attributes takes precedence.

Severity

OpenTelemetry uses record.severity to track log levels where loki uses record.attributes.level for the same. The exporter automatically maps the two, except if a "level" attribute already exists.

Advanced Configuration

Several helper files are leveraged to provide additional capabilities automatically:

• HTTP settings
• Queuing and retry settings

Documentation

Overview

Package lokiexporter implements an exporter that sends log data to Loki.

Index

• func NewFactory() exporter.Factory
• type Config
  • func (c *Config) Validate() error
• type LabelsConfig
• type Tenant

Functions

func NewFactory

func NewFactory() exporter.Factory

NewFactory creates a factory for the legacy Loki exporter.

Types

type Config

type Config struct {
	confighttp.HTTPClientSettings `mapstructure:",squash"` // squash ensures fields are correctly decoded in embedded struct.
	exporterhelper.QueueSettings  `mapstructure:"sending_queue"`
	exporterhelper.RetrySettings  `mapstructure:"retry_on_failure"`

	// TenantID defines the tenant ID to associate log streams with.
	// Deprecated: [v0.57.0] use the attribute processor to add a `loki.tenant` hint.
	// See this component's documentation for more information on how to specify the hint.
	TenantID *string `mapstructure:"tenant_id"`

	// Labels defines how labels should be applied to log streams sent to Loki.
	// Deprecated: [v0.57.0] use the attribute processor to add a `loki.attribute.labels` hint.
	// See this component's documentation for more information on how to specify the hint.
	Labels *LabelsConfig `mapstructure:"labels"`

	// Allows you to choose the entry format in the exporter.
	// Deprecated: [v0.57.0] Only the JSON format will be supported in the future. If you rely on the
	// "body" format and can't change to JSON, let us know before v0.59.0 by opening a GitHub issue
	// and we'll work with you to find a solution.
	Format *string `mapstructure:"format"`

	// Tenant defines how to obtain the tenant ID
	// Deprecated: [v0.57.0] use the attribute processor to add a `loki.tenant` hint.
	// See this component's documentation for more information on how to specify the hint.
	Tenant *Tenant `mapstructure:"tenant"`
}

Config defines configuration for Loki exporter.

func (*Config) Validate

func (c *Config) Validate() error

type LabelsConfig

type LabelsConfig struct {
	// Attributes are the log record attributes that are allowed to be added as labels on a log stream.
	Attributes map[string]string `mapstructure:"attributes"`

	// ResourceAttributes are the resource attributes that are allowed to be added as labels on a log stream.
	ResourceAttributes map[string]string `mapstructure:"resource"`

	// RecordAttributes are the attributes from the record that are allowed to be added as labels on a log stream. Possible keys:
	// traceID, spanID, severity, severityN.
	RecordAttributes map[string]string `mapstructure:"record"`
}

LabelsConfig defines the labels-related configuration Deprecated: [v0.57.0] will be removed without replacement by v0.61.0. See the Config#Labels for alternatives.

type Tenant

type Tenant struct {
	// Source defines where to obtain the tenant ID. Possible values: static, context, attribute.
	Source string `mapstruct:"source"`

	// Value will be used by the tenant source provider to lookup the value. For instance,
	// when the source=static, the value is a static value. When the source=context, value
	// should be the context key that holds the tenant information.
	Value string `mapstruct:"value"`
}

Deprecated: [v0.57.0] will be removed without replacement by v0.61.0. See the Config#Tenant for alternatives.