azureblobreceiver

README

Azure Blob Receiver

Status
Stability	development: logs, traces
Distributions	contrib
Issues
Code Owners	@eedorenko, @mx-psi

This receiver reads logs and trace data from Azure Blob Storage.

Configuration

The following settings are required:

• event_hub: endpoint: (no default): Azure Event Hub endpoint triggering on the Blob Create event

The following settings can be optionally configured:

• auth (default = connection_string): Specifies the used authentication method. Supported values are connection_string, service_principal.
• cloud (default = "AzureCloud"): Defines which Azure Cloud to use when using the service_principal authentication method. Value is either AzureCloud or AzureUSGovernment.
• logs: container_name: (default = "logs"): Name of the blob container with the logs
• traces: container_name: (default = "traces"): Name of the blob container with the traces

Authenticating using a connection string requires configuration of the following additional setting:

• connection_string: Azure Blob Storage connection key, which can be found in the Azure Blob Storage resource on the Azure Portal.

Authenticating using service principal requires configuration of the following additional settings:

• service_principal: tenant_id client_id client_secret
• storage_account_url: Azure Storage Account url

The service principal method also requires the Storage Blob Data Contributor role on the logs and traces containers.

Example configurations

Using connection string for authentication:

receivers:
  azureblob:
    connection_string: DefaultEndpointsProtocol=https;AccountName=accountName;AccountKey=+idLkHYcL0MUWIKYHm2j4Q==;EndpointSuffix=core.windows.net
    event_hub:
      endpoint: Endpoint=sb://oteldata.servicebus.windows.net/;SharedAccessKeyName=otelhubbpollicy;SharedAccessKey=mPJVubIK5dJ6mLfZo1ucsdkLysLSQ6N7kddvsIcmoEs=;EntityPath=otellhub

Using service principal for authentication:

receivers:
  azureblob:
    auth: service_principal
    service_principal:
      tenant_id: "${tenant_id}"
      client_id: "${client_id}"
      client_secret: "${env:CLIENT_SECRET}"
    storage_account_url: https://accountName.blob.core.windows.net
    event_hub:
      endpoint: Endpoint=sb://oteldata.servicebus.windows.net/;SharedAccessKeyName=otelhubbpollicy;SharedAccessKey=mPJVubIK5dJ6mLfZo1ucsdkLysLSQ6N7kddvsIcmoEs=;EntityPath=otellhub

The receiver subscribes on the events published by Azure Blob Storage and handled by Azure Event Hub. When it receives Blob Create event, it reads the logs or traces from a corresponding blob and deletes it after processing.

Documentation

Index

• Constants
• func NewFactory() receiver.Factory
• type AuthType
  • func (e *AuthType) UnmarshalText(text []byte) error
• type CloudType
  • func (e *CloudType) UnmarshalText(text []byte) error
• type Config
  • func (c Config) Validate() (err error)
• type EventHubConfig
• type LogsConfig
• type ServicePrincipalConfig
• type TracesConfig

Constants

const (
	AzureCloudType           = "AzureCloud"
	AzureGovernmentCloudType = "AzureUSGovernment"
)

Functions

func NewFactory

func NewFactory() receiver.Factory

NewFactory returns a factory for Azure Blob receiver.

Types

type AuthType

type AuthType string

const (
	ServicePrincipalAuth AuthType = "service_principal"
	ConnectionStringAuth AuthType = "connection_string"
)

func (*AuthType) UnmarshalText

func (e *AuthType) UnmarshalText(text []byte) error

type CloudType

type CloudType string

func (*CloudType) UnmarshalText

func (e *CloudType) UnmarshalText(text []byte) error

type Config

type Config struct {
	// Type of authentication to use
	Authentication AuthType `mapstructure:"auth"`
	// Azure Blob Storage connection key,
	// which can be found in the Azure Blob Storage resource on the Azure Portal. (no default)
	ConnectionString string `mapstructure:"connection_string"`
	// Storage Account URL, used with Service Principal authentication
	StorageAccountURL string `mapstructure:"storage_account_url"`
	// Configuration for the Service Principal credentials
	ServicePrincipal ServicePrincipalConfig `mapstructure:"service_principal"`
	// Azure Cloud to authenticate against, used with Service Principal authentication
	Cloud CloudType `mapstructure:"cloud"`
	// Configurations of Azure Event Hub triggering on the `Blob Create` event
	EventHub EventHubConfig `mapstructure:"event_hub"`
	// Logs related configurations
	Logs LogsConfig `mapstructure:"logs"`
	// Traces related configurations
	Traces TracesConfig `mapstructure:"traces"`
}

func (Config) Validate

func (c Config) Validate() (err error)

Validate validates the configuration by checking for missing or invalid fields

type EventHubConfig

type EventHubConfig struct {
	// Azure Event Hub endpoint triggering on the `Blob Create` event
	EndPoint string `mapstructure:"endpoint"`
}

type LogsConfig

type LogsConfig struct {
	// Name of the blob container with the logs (default = "logs")
	ContainerName string `mapstructure:"container_name"`
}

type ServicePrincipalConfig

type ServicePrincipalConfig struct {
	// Tenant ID, used with Service Principal authentication
	TenantID string `mapstructure:"tenant_id"`
	// Client ID, used with Service Principal authentication
	ClientID string `mapstructure:"client_id"`
	// Client secret, used with Service Principal authentication
	ClientSecret configopaque.String `mapstructure:"client_secret"`
}

type TracesConfig

type TracesConfig struct {
	// Name of the blob container with the traces (default = "traces")
	ContainerName string `mapstructure:"container_name"`
}